Skip to content

FE-1686: Stream a sweep's first GPU point by probing after start and sharing one setup across its points - #9665

Open
kube wants to merge 10 commits into
claude/opt-proto-sweep-objective-chartfrom
claude/opt-proto-gpu-first-stream
Open

FE-1686: Stream a sweep's first GPU point by probing after start and sharing one setup across its points#9665
kube wants to merge 10 commits into
claude/opt-proto-sweep-objective-chartfrom
claude/opt-proto-gpu-first-stream

Conversation

@kube

@kube kube commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Important

Experimental
Behind the WebGPU and Parameter sweeps feature flags.

Summary

Before this PR, the first point a sweep computed on the GPU showed nothing until it was done, while every later rung of the same point streamed from its first chunk. Two causes compounded. The derived-capacity probe ran while the experiment handle was being created, before anything subscribed to its metrics store, so its frames surfaced as one complete picture once the handle existed. And a point selection carried no per-run parameter buffer, so every swept value was baked into the shader and its cache key: each parameter change destroyed the device, recompiled the shader and probed from scratch, paying the first cause again.

This PR moves the probes into the run phase, after start(), so their chunks stream like the attempt's, and carries a sweep's swept parameters through the per-run buffer for every selection, so all points of a sweep share one compiled shader, one device and one calibration. Rungs that start on the same marking share a single in-flight probe, and a shared calibration that a new point outgrows grows once and re-probes from the grown slabs. Measured on real WebGPU, the first frames of a fresh point now draw within about 50 to 160 ms of the click on every bundled model tried, against 250 to 450 ms and a full-picture jump before; a parameter change no longer destroys or requests a device.

9665.mp4

Links

Changes

GPU handle

  • Probes run as the first phase of start()

    New gpu-experiment-handle/run-phase.ts orders the capacity probe, the window probe and the calibrated attempt over one executeAttempt, pure over the device so it is unit-tested without one. Creation keeps every static refusal.

  • Probe refusals end the run with a reason

    A net whose slabs overflow past seven growths, or whose heavy tail needs more than 64 KB per run, ends with an error asking to switch the experiment to the CPU backend. The silent first-batch fallback only ever covered the first batch.

  • One probe in flight per marking

    gpu-experiment-handle/shared-calibration.ts lets a rung wait for the probe in flight on its marking and adopt its result, only when the marking needs a probe, so a declared-slab net pipelines as before.

  • A shared calibration that overflows grows once, then re-probes from the grown slabs

    slabsFromProbe takes a slab floor, so a re-probe never shrinks below slabs that already overflowed.
    An attempt a metric halted is handed back without the re-probe.

  • Probe attempts publish no progress of their own

    completedRuns no longer jumps to the probe's run count and back.

  • slabsFromProbe sizes its heavy-tail arena check from the uncapped observation

    The observed maximum plus margin is measured against the arena threshold before the place's slab ceiling caps the capacity.

Sweep batches

  • Swept parameters ride the per-run buffer for every selection

    sweptNetParameterIds derives the ids once per experiment; constantRunPlan fills the buffer for a point selection; translateRangeDraws always emits the swept ids. Every point of a sweep hits the same gpuBackendSetupKey.

Review fixes

  • Waiters re-read the calibration share after each wait, so a probe that stored nothing wakes one new probe

    A failed or cancelled probe woke every waiting rung at once and each claimed the key and probed. The lookup now loops: the first rung to wake claims, the rest find that claim and wait on it.

  • Cached calibration's attempt a metric halted skips the fresh probe

    Outgrowing a cached calibration sent the run back through the probe even when a metric had halted a run, which the same seeds halt again, so the probe could only repeat the failure the handle was about to report.
    runCalibratedExperiment re-probes only while no metric halted; run-phase.test.ts pins one attempt, unchanged slabs, nothing remembered and the halted counts handed back.

  • Capacity probe a metric halted is not remembered for later batches on its marking

    remember ran before the halted check, so a later batch on the marking adopted the halted probe's calibration, skipped its own probe and met the halt only after a full attempt.
    runCalibratedExperiment returns the failure before remembering; run-phase.test.ts pins one attempt, the halted reason and nothing remembered for a probe a metric halted.

Known issues

  • A point whose dynamics need much larger slabs than the first point refines later than before

    Its first frames draw at once, then the shared calibration grows and re-probes; the satellite launcher at 18000 frames folded its first runs at 22.6 s against 11.2 s before.

  • Axes that shape the initial marking still miss the setup key per point

    The marking cannot vary per run on the GPU.

  • Two rungs adopting the same stale calibration re-probe separately
  • A renderer memory cliff at 18000 frames × 1000 runs on the second point is pre-existing

Next steps

  • Publish cost at long maxTime
  • Progress that does not step back when a full attempt grows its slabs

Test coverage

  • run-phase.test.ts:

    Probe order, the window-probe branch, adoption of a cached calibration, the grow-once-then-re-probe path with the slab floor, cancellation and disposal between phases.

  • shared-calibration.test.ts:

    Claim, wait, adopt and release; no claim when no probe is needed.

  • gpu-experiment-handle.test.ts:

    Two batches on one marking run at once when neither needs a probe.
    A batch waits for the probe another runs on its marking, then adopts it.
    A probe's chunks publish no progress.
    Three batches on one marking whose first probe is cancelled run exactly one further probe, then both remaining attempts share its slabs.

  • calibration.test-helpers.ts:

    Shared shader, place, session and outcome builders for the three GPU handle suites.

  • gpu-backend-cache.test.ts:

    Setup-key equality for a point batch at another rate.

  • sweep-run-overrides.test.ts:

    Swept ids derived from the axes, the constant run plan for a point, ids always emitted for ranges.

  • Measured on real WebGPU:

    First data after a click on four bundled models, device requests per parameter change, shader modules per point, ladder completion times, recorded in the PR summary and the diagnosis notes.

How to test

  • Open Petrinaut preview on Vercel in a WebGPU browser
  • Viewport controls > Settings > Simulation > WebGPU, Parameter sweeps
  • Menu > Load example > Probabilistic Satellite Launcher
  • Simulate > Experiments > Create > Backend GPU, Sweep on launch_rate and satellite_initial_altitude, Add metric > Place tokens > Run
  • Click the Surface

    Expect the metric chart to start drawing within a fraction of a second and fill progressively, the Compute column reading GPU

  • Move a slider to a new point

    Expect the chart to move at once, no pause for a device or shader

  • Repeat with the SIR Model

    Expect the same immediate streaming

@vercel

vercel Bot commented Sep 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hash Ready Ready Preview Sep 12, 2026 4:32pm UTC
petrinaut Ready Ready Preview Sep 12, 2026 4:32pm UTC
petrinaut-docs Ready Ready Preview Sep 12, 2026 4:32pm UTC
1 Skipped Deployment
Project Deployment Actions Updated
hashdotdesign-tokens Ignored Ignored Preview Sep 12, 2026 4:32pm UTC

Request Review

@kube kube self-assigned this Sep 11, 2026
@kube
kube added this pull request to stack #9549 September 11, 2026 04:35
@github-actions github-actions Bot added area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team type/eng > backend Owned by the @backend team labels Sep 11, 2026
@kube
kube force-pushed the claude/opt-proto-gpu-first-stream branch from f061eb5 to 3fe57fb Compare September 11, 2026 13:03
@kube
kube force-pushed the claude/opt-proto-gpu-first-stream branch from 3fe57fb to a6e0be2 Compare September 11, 2026 13:07
@kube kube changed the title Stream a sweep's first GPU point by probing after start and sharing one setup across its points FE-1686: Stream a sweep's first GPU point by probing after start and sharing one setup across its points Sep 11, 2026
@kube
kube force-pushed the claude/opt-proto-gpu-first-stream branch from a6e0be2 to 2f2669c Compare September 11, 2026 18:30
@kube
kube marked this pull request as ready for review September 11, 2026 22:11
Copilot AI balanced review requested due to automatic review settings September 11, 2026 22:11
@cursor

cursor Bot commented Sep 11, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes GPU experiment lifecycle, concurrent calibration sharing, and sweep backend caching—behavioral shifts in streaming, errors, and sweep performance, though heavily unit-tested.

Overview
Moves GPU capacity and histogram calibration from handle creation into the first phase of start(), so probe frames stream to charts like any other attempt, while static eligibility still fails at create.... Probe chunks no longer advance run progress (probe flag on attempts), and probe-only failures (heavy tails, unbounded overflow) end the experiment with an error asking to switch to the CPU instead of refusing at creation.

Adds run-phase.ts to order probes and the full run, shared-calibration.ts plus GpuBackend.calibrating so concurrent batches on the same marking share one in-flight probe, and CACHED_RUN_POLICY with slab floors when a reused calibration still overflows (re-probe without shrinking slabs). slabsFromProbe checks arena size before applying the slab ceiling.

For parameter sweeps, every batch now carries swept net parameters in the per-run buffer (sweptNetParameterIds, constantRunPlan, fixed id set in translateRangeDraws), so point and range selections share one gpuBackendSetupKey, device, shader, and calibration across slider moves.

Reviewed by Cursor Bugbot for commit 129492d. Bugbot is set up for automated code reviews on this repo. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Sweep dependency detection misses cross-axis expressions, failed probes can trigger duplicate concurrent probes, and required changesets are absent.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Moves WebGPU calibration into the run phase for immediate streaming and reuses GPU setup across sweep points.

Changes:

  • Adds streamed, shared GPU calibration with cached-calibration growth.
  • Sends swept parameters through per-run buffers.
  • Updates tests and user/architecture documentation.
File summaries
File Description
libs/@local/petrinaut-arch-docs/content/simulation/gpu-capacity-calibration.mdx Documents run-phase and shared calibration.
libs/@local/petrinaut-arch-docs/content/experiments/sweep-orchestration.mdx Documents uniform sweep buffers.
libs/@local/petrinaut-arch-docs/content/experiments/parameter-sweeps.mdx Documents GPU setup reuse.
libs/@local/petrinaut-arch-docs/content/experiments/backend-selection.mdx Clarifies runtime probe failures.
libs/@local/petrinaut-arch-docs/content/diagrams/gpu-capacity-calibration.d2 Updates calibration failure flow.
libs/@hashintel/petrinaut/src/react/experiments/provider/sweep-batch-instantiation/sweep-run-overrides.ts Derives and builds sweep run plans.
libs/@hashintel/petrinaut/src/react/experiments/provider/sweep-batch-instantiation/sweep-run-overrides.test.ts Tests sweep parameter plans.
libs/@hashintel/petrinaut/src/react/experiments/provider/sweep-batch-instantiation.ts Applies uniform plans to batches.
libs/@hashintel/petrinaut/src/react/experiments/provider.tsx Supplies sweep axes.
libs/@hashintel/petrinaut/docs/experiments.md Updates user-facing GPU behavior.
libs/@hashintel/petrinaut-core/src/webgpu/gpu-experiment-handle/shared-calibration.ts Coordinates in-flight probes.
libs/@hashintel/petrinaut-core/src/webgpu/gpu-experiment-handle/shared-calibration.test.ts Tests calibration sharing.
libs/@hashintel/petrinaut-core/src/webgpu/gpu-experiment-handle/run-phase.ts Orchestrates probe and run phases.
libs/@hashintel/petrinaut-core/src/webgpu/gpu-experiment-handle/run-phase.test.ts Tests run-phase sequencing.
libs/@hashintel/petrinaut-core/src/webgpu/gpu-experiment-handle/calibration.ts Adds probe metadata and slab floors.
libs/@hashintel/petrinaut-core/src/webgpu/gpu-experiment-handle/calibration.test.ts Tests revised calibration behavior.
libs/@hashintel/petrinaut-core/src/webgpu/gpu-experiment-handle/calibration.test-helpers.ts Shares calibration test fixtures.
libs/@hashintel/petrinaut-core/src/webgpu/gpu-experiment-handle.ts Runs and shares calibration after start.
libs/@hashintel/petrinaut-core/src/webgpu/gpu-experiment-handle.test.ts Tests concurrent handle behavior.
libs/@hashintel/petrinaut-core/src/webgpu/gpu-backend-cache.test.ts Tests setup-key reuse.
libs/@hashintel/petrinaut-core/src/webgpu/backend.ts Stores in-flight calibration state.
Review details
  • Files reviewed: 21/21 changed files
  • Comments generated: 3
  • Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.


💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread libs/@hashintel/petrinaut-core/src/webgpu/gpu-experiment-handle.ts Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 2f2669c. Configure here.

@kube
kube force-pushed the claude/opt-proto-gpu-first-stream branch from 2f2669c to 7f6a3c8 Compare September 12, 2026 01:19
drdma
drdma previously approved these changes Sep 12, 2026
@kube
kube added this pull request to the merge queue Sep 12, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue because a pull request earlier in the stack was removed Sep 12, 2026
@kube
kube added this pull request to the merge queue Sep 12, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue because a pull request earlier in the stack was removed Sep 12, 2026
@kube
kube added this pull request to the merge queue Sep 12, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue because a pull request earlier in the stack was removed Sep 12, 2026
@kube
kube added this pull request to the merge queue Sep 12, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue because a pull request earlier in the stack was removed Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

4 participants