Repository navigation
Compare GPU block-argmin objective up to FMA rounding (test only) - #155
Merged
ChrisRackauckas merged 2 commits intoOct 17, 2026
Merged
Conversation
`sol.objective` is `f` recomputed on the host at the chosen position, while the particles' `best_cost` comes from the GPU kernel, where multiply-adds may be contracted to FMA. The two differ by 1 ulp on CUDA CI. Check the chosen candidate exactly and the objective with rtol = 8eps(). Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code Agent-Model: claude-opus-5-5[1m] Agent-Session: https://claude.ai/code/session_011aK5Xvp9NpdkzKuEwtiMGd
Each ordering of an n-term sum of nonnegative squares has relative error at most about n*eps/2, so host and FMA-contracted kernel values differ by at most about n*eps. The sampled 4-ulp maximum was not a bound: x = (1, y, ..., y) with y = sqrt(8.5eps) gives a 9eps gap, which passes rtol = 10eps. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude <noreply@anthropic.com> Agent-Harness: Claude Code Agent-Model: claude-opus-5-5[1m] Agent-Session: https://claude.ai/code/session_011aK5Xvp9NpdkzKuEwtiMGd
ChrisRackauckas
marked this pull request as ready for review
October 17, 2026 05:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ignore this PR until @ChrisRackauckas has reviewed it.
What and why
On main, the "block argmin matches brute force" test in
test/gpu.jlfails on CUDA CI. It compares two numbers with==:sol.objective, which isprob.f(gbest.position, prob.p)recomputed on the host (src/solve.jl);best_cost, which the GPU kernel computed for the same particle.The kernel's
sum(abs2, x)may be contracted to FMA, so the two values can differ by a few ulp even when the kernel picks the right particle. This PR changes only that test:sol.u == best.best_position;isapprox(...; rtol = length(sol.u) * eps()).Evidence
The CI gaps are all 1 ulp. GPU Tests on main has failed on every run since 2026-10-09 (593a4a2, ef0c3ad, df60a25) and was green through 2026-10-03 (956709f). The only repo change in that window is a JET compat bump. CUDA went from 6.2.2 to 6.4.1 and GPUCompiler from 1.23.0 to 2.10.0. Examples from the failing logs:
A wrong candidate would almost never come out within 1 ulp of the true minimum.
FMA contraction explains a gap of this size. I ran a CPU script comparing a plain left-to-right
sum(abs2, x)with an FMA-chained one, for D = 10 and 100k random points:That sampled maximum is empirical, not a bound, so the tolerance is derived from the floating-point error bound instead. Summing n nonnegative squares in either order (plain or FMA-chained) gives a relative error of at most about n·u, with u = eps/2. The two results therefore differ by at most about n·eps, so the test uses
rtol = length(sol.u) * eps(), which is 10eps for D = 10.Review probe: for x = (1, y, …, y) with y = sqrt(8.5eps), the plain and FMA sums differ by 9.0eps relative. That fails
rtol = 8eps()and passesrtol = 10eps:All four CI pairs above are 1 ulp, well within the bound.
Local verification
PPS_TEST_GROUP=Core julia --project -e 'using Pkg; Pkg.test()'passes on CPU. The Core group runsgpu.jlwith the CPU backend, including this testset:Runic and typos report nothing on
test/gpu.jl.Not verified
Overlap with #151
#151 changes the same assertion, as part of its scalar-cost argmin work. This PR leaves out that behaviour change so main can go green on its own. Whichever PR merges second needs a trivial rebase of this hunk.
Risk assessment
test/gpu.jl. No source, public API or downstream impact.🤖 Generated with Claude Code (model: claude-opus-5-5[1m])
https://claude.ai/code/session_011aK5Xvp9NpdkzKuEwtiMGd
Risk assessment (independent review, codex-cli gpt-6-astra, reviewed head 2846e8f)
Head reviewed:
2846e8f7804f0bb2697628b444a92cef026f72c2. Recorded before review and unchanged at the final check. Initial low-risk rating was made from the diff before reading the author's risk claims.Authorship: commit trailers identify Claude Code / claude-opus-5-5[1m], with Co-Authored-By Claude; the body also identifies Claude Code. Reviewed normally under the explicit non-Claude-reviewer exception.
🤖 Posted by an AI agent — harness: codex-cli · model: gpt-6-astra (independent reviewer; appended by Claude Code 2.1.301 · claude-opus-5-5)
Conversation: https://claude.ai/code/session_01VZjwf7Ux9YtTzwHGHMqnsq