Skip to content

Add a sleep tool that really waits, and make wait report the time it took - #223

Closed
handstandsam wants to merge 1 commit into
mainfrom
upstream-2026.08.02
Closed

Add a sleep tool that really waits, and make wait report the time it took#223
handstandsam wants to merge 1 commit into
mainfrom
upstream-2026.08.02

Conversation

@handstandsam

Copy link
Copy Markdown
Collaborator

Summary

Syncs the latest upstream changes through 2026.08.02.

A sleep tool that actually waits

wait and waitForChange are settles: they return the moment the UI goes quiet, so on a
static screen they come back in milliseconds no matter what duration you asked for. That makes
them the wrong tool when you need time to genuinely pass — e.g. letting a server-side value
propagate before navigating to the screen that reads it.

New sleep tool blocks for a fixed wall-clock duration and never returns early. It runs
in-process for every agent, so the duration is driver-independent. It is hidden from the LLM's
toolbox (surfaceToLlm = false) — a fixed sleep is right when an author knows about an
off-screen dependency and almost always wrong when picked autonomously — but is callable from
hand-authored trail YAML and scripted tools, and is recorded like any other tool. A duration
outside 100ms–5min is an error rather than a silent clamp, since clamping would return early
while reporting success.

Waits now report the time they actually took

wait used to log "Waited 30 seconds" whether it waited 30 seconds or 150ms, and
waitForChange's unsupported-driver fallback made the same claim. Both now report measured
elapsed time alongside the ceiling that was requested, so a log no longer asserts a wait that
never happened. The tool descriptions say "ceiling, not a duration" too, and point at asserting
on the element you're actually waiting for.

Report timeline: see the tool calls that were folded away

A traceId is allocated per LLM request — one turn's whole tool batch — not per tool call, so a
turn's calls all fold onto one timeline row. The other calls weren't just collapsed, they were
absent from the payload entirely, and nothing in the row hinted they existed. They now appear as
expandable children in dispatch order. Repeated polls keep their ×N count instead of expanding
into N rows.

Recording-resolution census fixes

The one-line census printed buckets that overlapped, so summing the numbers a reader saw gave the
wrong total: a matched-but-empty step counted as both exact and zero-tool no-op. The buckets
now partition, the trailhead is named separately so step numbers line up with every other
artifact, and conditional is parenthesised because it's a property of the matched steps rather
than a bucket of its own. A self-guarding tool (probes, no-ops when its target is absent) now
counts as conditional alongside the wrapper form, so a device that spells the guard differently
is no longer reported as having lost one.

Implementation detail

  • SleepTrailblazeTool (HostLocalExecutableTrailblazeTool, so BaseTrailblazeAgent.runTrailblazeTools
    runs it before driver-specific dispatch), registered as CoreTools.SLEEP in the
    core_interaction toolset. Bounds held under DaemonClient.RUN_POLL_TIMEOUT_MS — a host-local
    sleep emits no progress, so a longer one is indistinguishable from a wedged run.
  • WaitForIdleSyncTrailblazeTool / WaitForChangeTrailblazeTool measure with
    TimeSource.Monotonic around the driver call.
  • toolChildren in run-report-extract.ts merges executed TrailblazeToolLogs with
    DelegatingTrailblazeToolLog declarations, matching on name and args so a repeated
    primitive with one unlogged dispatch still surfaces.
  • RecordingResolution / TrailRecordingResolution bucket partitioning and
    CONDITIONAL_TOOL_NAMES.
  • testRailUrl trail-metadata read dropped from the runner paths.

Test plan

  • :trailblaze-common:check — new SleepTrailblazeToolTest, WaitForIdleSyncTrailblazeToolTest, WaitForChangeTrailblazeToolTest, ToolSerializationTest
  • :trailblaze-models:checkRecordingResolutionTest, apiCheck
  • :trailblaze-report:checkrun-report-core.test.ts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant