Repository navigation
Conversation
(cherry picked from commit f00d958)
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
— codex GPT-6 on behalf of Ralf — Separate local and wheel test dependency groups so CuPy test setup does not install component wheels that shadow the local Toolkit. Keep the existing pinned dependencies for wheel-based jobs. Include cuRAND in the shared mini Toolkit and recognize a loadable native cuRAND library for bare sample requirements. Versioned, extras and URL requirements continue to use distribution metadata. The component-list change refreshes the mini-Toolkit caches for all action consumers. Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
github-actions
Bot
added
CI/CD
|
/ok to test c17a310 |
|
— codex GPT-6 on behalf of Ralf — PyTorch can load cuRAND into the pytest parent on Windows without making that DLL discoverable by sample subprocesses. Probe with the current interpreter in a fresh process so parent-only libraries do not admit samples that cannot run. Preserve diagnostics for unexpected loader errors. Cover the process boundary with real child imports and retain local Toolkit sample execution. Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
— codex GPT-6 on behalf of Ralf — The CUDA 13.5 CUPTI wheel uses cupti64_13.dll rather than the older year-version names. Recognize it first while preserving the existing fallback names. Local core jobs exposed this in their subsequent pathfinder all_must_work phase after installing the latest wheel extras. Extend the search regression to cover the new name and old-name fallback. Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
github-actions
Bot
added
the
cuda.pathfinder
|
/ok to test 64107b7 |
— codex GPT-6 on behalf of Ralf — An editable pathfinder finder can locate the real module despite the fixture's empty cuda package. Supply an explicit missing-module stub so the test models absence independently of the developer's installation. All 78 sample-runner tests now pass with both editable and wheel installs. Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
|
/ok to test 413814c |
mdboom
left a comment
mdboom
left a comment
There was a problem hiding this comment.
I asked my klanker why we have a broad (major-only) pin for cuda-toolkit in the pyproject.toml and then we narrow it (in multiple places) in run-tests. It responded:
===
The pyproject groups can't carry the narrow pin, because the minor differs per CI row. ci/test-matrix.yml runs the same major at different minors. For example, CUDA_VER is 13.0.2 on some rows and 13.4.2 on others, and 12.9.1 is separate. A dependency group is static, so test-cu13 would have to say ==13.0.* and ==13.4.* at once. The only way to do that in pyproject is one group per minor (test-cu13-0, test-cu13-4, ...). Each Toolkit bump would then need a new group, a new [tool.uv] conflicts entry, and a new test-matrix reference. That is more churn than the one-line pin in run-tests. The broad ==13.* in pyproject is also correct for the other consumers: local dev, toolshed/collect_site_packages_*, and users running pip install --group test-cu13.
The duplication is real, though. run-tests repeats cuda-toolkit==${TEST_CUDA_MAJOR_MINOR}.* at lines 79, 131, 141, 186, 199 and 214. A cleaner version sets it once as a pip constraint, which is what cuda_core/third_party/cccl/ci/pyenv_helper.sh:113 already does:
echo "cuda-toolkit==${TEST_CUDA_MAJOR_MINOR}.*" > "${PIP_CONSTRAINT}"
export PIP_CONSTRAINTThat one line covers every pip install in the script, extras like [curand] included. It also removes the TOOLKIT_PIN array that this PR adds. Suggest it as a follow-up, or in this PR if the author is open to it.
===
I think we should make that simplification.
| @@ -0,0 +1,176 @@ | |||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | |||
There was a problem hiding this comment.
These tests are so monkeypatch'ed as to be useless. I would just remove this. If run-tests does the wrong thing, we will see it in CI anyway.
| # Local jobs get CuPy's headers and libraries from the mini CTK; installing | ||
| # component wheels would take precedence over those local libraries. |
There was a problem hiding this comment.
Nit: Doesn't make sense to mention CuPy here -- it's true, but that's just an artifact of what our test dependencies are (which may change in the future).
| # Local jobs get CuPy's headers and libraries from the mini CTK; installing | |
| # component wheels would take precedence over those local libraries. | |
| # Local jobs get cuda-core's test dependencies from the mini CTK; installing | |
| # component wheels would take precedence over those local libraries. |
| test-cu12-local = [ {include-group = "ml-dtypes" }, {include-group = "test" }, "cupy-cuda12x; python_version < '3.15'"] | ||
| test-cu13-local = [ {include-group = "ml-dtypes" }, {include-group = "test" }, "cupy-cuda13x; python_version < '3.15'"] | ||
| test-cu12 = [ {include-group = "test-cu12-local" }, "cuda-toolkit[cudart]==12.*"] # runtime headers needed by CuPy | ||
| test-cu13 = [ {include-group = "test-cu13-local" }, "cuda-toolkit[cudart]==13.*"] # runtime headers needed by CuPy |
There was a problem hiding this comment.
Consider a comment on the groups explaining why -local exists, because the pyproject has no hint of it.
— codex GPT-6 on behalf of Ralf —
Description
A CUDA Toolkit minor bump builds cuda-core with newer headers before matching cuda-bindings are available on PyPI. Older-runtime CI rows then install the published bindings floor and fail cuda-core's header compatibility check.
Use the bindings wheel built in the same run while the floor trails the build Toolkit minor, and resume literal-floor testing after the floor advances. Cover standard core tests and PyTorch/Numba nightlies using the newly built core, while preserving the nightly job testing released core. The initial change is extracted from f00d958 in #3368.
The local Toolkit follow-up is isolated in c17a310 and c460771, so it can be reviewed or reverted separately. Local core jobs install bare CuPy without component wheels that shadow the selected Toolkit. The mini Toolkit supplies cuRAND, and the sample dependency check accepts native cuRAND for bare requirements while retaining distribution checks for version, extras and URL constraints. The native probe runs in a fresh interpreter because libraries loaded by PyTorch in pytest are not necessarily available to sample subprocesses. Existing wheel-job pins remain. Adding cuRAND refreshes shared mini-Toolkit caches and expands their downloads.
Commit 64107b7 adds CUDA 13.5's Windows CUPTI name,
cupti64_13.dll, while preserving older names. CI exposed this when the post-core pathfinder tests installed the newly published 13.5 wheel extras.Validation:
This branch retains main's Toolkit matrix; #3368 supplies the CUDA 13.5 integration case.
Checklist