Skip to content

[infra] select the evaluation suite per pipeline #153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vikuznetsov-nvidia merged 3 commits into from
Sep 8, 2026
+8 −4
Merged

[infra] select the evaluation suite per pipeline #153

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0c04d6b
[infra] select the evaluation suite per pipeline
vikuznetsov-nvidia Sep 7, 2026
a7bfa3f
[clean] trim suite comments that restate the value
vikuznetsov-nvidia Sep 8, 2026
c9e95f2
Merge branch 'main' into vikuznetsov/eval-suite-ci
vikuznetsov-nvidia Sep 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/workflows/nightly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ env:
EVAL_REPORT_ARTIFACT: eval-reports-staging
AWS_DEFAULT_REGION: ${{ vars.AWS_DEFAULT_REGION }}
S3_DATASETS_BUCKET: ${{ vars.S3_DATASETS_BUCKET }}
# Jetson configurations inherit this but never read it: their eval steps are
# gated on matrix.eval.
EVAL_SUITE: full

jobs:
check-changes:
Expand Down Expand Up @@ -197,7 +200,7 @@ jobs:
run: |
docker run --rm --network host \
-e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e AWS_DEFAULT_REGION -e S3_DATASETS_BUCKET \
-e RUNNER_STORAGE_ROOT -e RUNNER_LOCAL_DATASETS_ROOT \
-e RUNNER_STORAGE_ROOT -e RUNNER_LOCAL_DATASETS_ROOT -e EVAL_SUITE \
-v "$(pwd):/cuvslam:ro" \
-v "$RUNNER_LOCAL_DATASETS_ROOT:$RUNNER_LOCAL_DATASETS_ROOT" \
-w /cuvslam \
Expand All @@ -212,7 +215,7 @@ jobs:
mkdir -p "$RUNNER_LOCAL_DATASETS_ROOT"
docker run --rm --network host \
-e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e AWS_DEFAULT_REGION -e S3_DATASETS_BUCKET \
-e RUNNER_LOCAL_DATASETS_ROOT \
-e RUNNER_LOCAL_DATASETS_ROOT -e EVAL_SUITE \
-v "$(pwd):/cuvslam:ro" \
-v "$RUNNER_LOCAL_DATASETS_ROOT:$RUNNER_LOCAL_DATASETS_ROOT" \
-w /cuvslam \
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/pr-verify.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ env:
S3_DATASETS_BUCKET: ${{ vars.S3_DATASETS_BUCKET }}
# PR eval runs on x86_64 with the build script's default CUDA/Ubuntu.
EVAL_CONFIG: x86_64-cuda12.6.3-ubuntu24.04
EVAL_SUITE: smoke

jobs:
lint:
Expand Down Expand Up @@ -149,7 +150,7 @@ jobs:
run: |
docker run --rm --network host \
-e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e AWS_DEFAULT_REGION -e S3_DATASETS_BUCKET \
-e RUNNER_STORAGE_ROOT -e RUNNER_LOCAL_DATASETS_ROOT \
-e RUNNER_STORAGE_ROOT -e RUNNER_LOCAL_DATASETS_ROOT -e EVAL_SUITE \
-v "$(pwd):/cuvslam:ro" \
-v "$RUNNER_LOCAL_DATASETS_ROOT:$RUNNER_LOCAL_DATASETS_ROOT" \
-w /cuvslam \
Expand All @@ -163,7 +164,7 @@ jobs:
mkdir -p "$RUNNER_LOCAL_DATASETS_ROOT"
docker run --rm --network host \
-e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e AWS_DEFAULT_REGION -e S3_DATASETS_BUCKET \
-e RUNNER_LOCAL_DATASETS_ROOT \
-e RUNNER_LOCAL_DATASETS_ROOT -e EVAL_SUITE \
-v "$(pwd):/cuvslam:ro" \
-v "$RUNNER_LOCAL_DATASETS_ROOT:$RUNNER_LOCAL_DATASETS_ROOT" \
-w /cuvslam \
Expand Down
Loading