Skip to content

fix(android): bound the scroll-hint dumpsys probe and skip it in wait polling (#1270)#1288

Merged
thymikee merged 2 commits into
mainfrom
fix/1270-scroll-hint-budget
Jul 16, 2026
Merged

fix(android): bound the scroll-hint dumpsys probe and skip it in wait polling (#1270)#1288
thymikee merged 2 commits into
mainfrom
fix/1270-scroll-hint-budget

Conversation

@thymikee

Copy link
Copy Markdown
Member

Summary

Fixes the latent hazard identified in the diagnosis comment on #1270: deriveScrollableContentHintsIfNeeded's dumpsys activity top probe (dumpActivityTop in src/platforms/android/snapshot.ts) ran with timeoutMs: 8_000 — a whole typical wait/get budget — inside every poll iteration, and the call's latency was measured ranging from 37ms to ~5.9s under contention (10/10 manual samples pathological on the same unchanged screen). One slow call could eat the entire timeout budget of an otherwise-successful capture.

Per the diagnosis's own rescope: this is a timeout-hygiene fix, not a capture-completeness bug. The originally-filed "viewport-bottom element missing" framing was ruled out — the node is always present in the raw dump; the cost comes entirely from this auxiliary scroll-hint derivation call running after the real capture already succeeded.

Changes

  1. Cap the dumpsys calldumpActivityTop's timeoutMs drops from 8_000 to 1_500 (healthy latency is ~37ms, so 1.5s is generous headroom while bounding the pathological tail). It already returns null on failure/timeout and callers already degrade gracefully to an empty hint map — verified via existing + new tests.
  2. Skip hint derivation during find ... wait polling — a presence check never consumes scroll hints. The wait poll loop turned out to live in waitForFindMatch (src/commands/interaction/runtime/selector-read.ts), not daemon/handlers/find.ts's handleFindWait as the diagnosis comment named — dispatchFindReadOnlyViaRuntime now always intercepts read-only find actions (wait/exists/get_text/get_attrs) before handleFindWait is ever reached, so that loop is dead code on main today (confirmed empirically). The new includeHiddenContentHints option threads through the same existing channel other snapshot flags use: BackendSnapshotOptionscaptureSelectorSnapshotCommandFlags/contextFromFlagsDispatchContexthandleSnapshotCommandInteractor.snapshotsnapshotAndroid's pre-existing includeHiddenContentHints option (already used by alert.ts for system-dialog captures).
  3. Tests, matching the existing android-snapshot and selector-runtime idiom:
    • snapshotAndroid caps the activity-top scroll-hint probe at 1.5s — scrollable-typed node, XML without can-scroll-* attributes → dumpActivityTop fires with timeoutMs: 1500.
    • snapshotAndroid skips the activity-top probe when the helper XML already carries scroll-action attributes — pins the existing short-circuit behavior explicitly.
    • runtime find wait skips hidden-content hint derivation on every poll (#1270) — every poll iteration of waitForFindMatch passes includeHiddenContentHints: false down to the backend capture call.

Test plan

  • pnpm typecheck
  • pnpm exec oxlint --deny-warnings
  • pnpm format:check
  • pnpm check:affected --base origin/main --run (green modulo known-flaky-under-contention tests below, all confirmed passing in isolation)
  • pnpm check:fallow --base origin/main (0 new findings; pre-existing handleSettingsCommand complexity + a selector-read-shared.ts/resolution.ts duplicate clone group correctly excluded as inherited baseline findings)
  • Targeted suites green: src/platforms/android/__tests__/snapshot.test.ts (40/40), src/commands/interaction/runtime/selector-read.test.ts (17/17)
  • runtime-hints.test.ts, runner-client.test.ts, apple index.test.ts failed once under heavy host contention (load avg ~25) during a full check:affected run, unrelated to this diff's files — reran each in isolation and all pass cleanly
  • Live confirmation rides the shared wave-3 validation flow (bottom-wait probe, n=20, helper path) per the diagnosis comment's suggested closing evidence

@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.7 MB 1.7 MB +531 B
JS gzip 557.3 kB 557.4 kB +129 B
npm tarball 672.9 kB 673.1 kB +130 B
npm unpacked 2.4 MB 2.4 MB +531 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 26.4 ms 26.4 ms -0.0 ms
CLI --help 56.0 ms 56.7 ms +0.8 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/runtime.js +146 B +35 B
dist/src/selector-runtime.js +96 B +30 B
dist/src/context.js +134 B +23 B
dist/src/tv-remote.js +99 B +21 B
dist/src/internal/daemon.js +1 B 0 B

thymikee added a commit that referenced this pull request Jul 16, 2026
…ndler (#1290)

dispatchFindReadOnlyViaRuntime intercepts every read-only find action
(exists/wait/get_text/get_attrs) inside handleFindCommands and always
returns a response for them, so the legacy handleFindWait polling loop,
handleFindExists/GetText/GetAttrs, attachIssuedRefsGeneration, and the
sessionless-find device resolution behind them were unreachable
(verified empirically in #1288 and again here by planting throws in all
four handlers — the full find suite stays green).

With only mutating actions (click/fill/focus/type) able to reach the
snapshot path, requiresRect is constant true; fold it and drop the
now-dead scope threading, including the legacyIosSparse recovery scope
field that was only ever populated with undefined.
@thymikee
thymikee marked this pull request as ready for review July 16, 2026 15:12
@thymikee

Copy link
Copy Markdown
Member Author

Review at exact head 1550a1d9faff8f6cb7dfd0cc8885ce63b3c7a343:

The 1.5s cap and the skip for the distinct find … wait action are sound, but the issue’s motivating public path is still uncovered.

wait 'label="Battery"' 8000 parses as a standalone selector wait and polls waitForSelector(), whose snapshot capture does not set includeHiddenContentHints: false. Explicit text/ref waits likewise reach capture paths without that override. Android therefore defaults hidden-content hints on and can still await the auxiliary dumpsys activity top probe for up to 1.5s on every poll, charged to the caller’s timeout budget.

The new regression directly exercises only device.selectors.find({ action: 'wait' }). It is revert-sensitive for that separate command, but does not cover the daemon/public wait route or the reported repro.

Please disable hidden-content-hint derivation for the presence-only standalone selector/text/ref wait paths (stable wait may retain full snapshot semantics), and add a daemon/public-route regression matching the exact repro.

All current CI checks are green, but the live n=20 helper-path evidence is still absent. The branch is also behind current main, which now contains overlapping Android-helper changes, so rebase and rerun the relevant validation before the next review.

thymikee added 2 commits July 16, 2026 17:19
… polling (#1270)

deriveScrollableContentHintsIfNeeded's dumpsys activity top probe ran
with an 8s timeout equal to a whole wait/get budget, and measured
latency ranges from 37ms to ~5.9s under contention. Cap it at 1.5s so
one pathological call can't starve a caller's timeout, and skip hint
derivation entirely during find ... wait polling, since a presence
check never consumes scroll hints.

The wait polling loop actually lives in
commands/interaction/runtime/selector-read.ts's waitForFindMatch
(daemon/handlers/find.ts's own handleFindWait is unreachable for wait
today — dispatchFindReadOnlyViaRuntime always intercepts read-only
find actions first), so the skip-hints option threads through that
capture path down to snapshotAndroid via the existing
flags -> contextFromFlags -> dispatchCommand -> interactor.snapshot
channel.
…too (#1270)

The issue's motivating repro — wait 'label="Battery"' 8000 — polls
waitForSelector, not the find-wait loop, and text/ref waits poll
waitForText (backend.findText -> captureWaitSnapshot on the daemon, or
the snapshotContainsText fallback). All three presence-only polling
captures now disable hidden-content-hint derivation, matching the
Android alert-wait capture which already did. Stable wait keeps full
snapshot semantics.

Adds daemon-route regressions for the exact repro shape on both the
selector-wait and text-wait routes, asserting every per-poll snapshot
dispatch carries snapshotIncludeHiddenContentHints: false.
@thymikee
thymikee force-pushed the fix/1270-scroll-hint-budget branch from 1550a1d to 5046190 Compare July 16, 2026 15:33
@thymikee

Copy link
Copy Markdown
Member Author

Addressed at 5046190f7 (rebased onto current main first — clean rebase; the #1281 helper consolidation touched src/platforms/android/snapshot.ts but not the dumpActivityTop/hints region, and #1290's removal of the daemon find handler's dead read-only path composed cleanly with this branch; diff re-verified file-by-file against the new base).

Wait polling paths — full enumeration and now-state

Path Route it serves Now-state
waitForFindMatch (src/commands/interaction/runtime/selector-read.ts) find <q> wait [ms] Skips hints (previous commit)
waitForSelector (same file) wait 'label="Battery"' 8000 — the issue's repro Skips hints (this commit)
waitForText (same file), arm A: backend.findText → daemon findTextcaptureWaitSnapshot (src/daemon/selector-runtime-backend.ts) wait "text" [ms] and wait @ref [ms] on Android (ref resolves its label from the stored session snapshot, then polls as text) Skips hints (this commit)
waitForText, arm B: snapshotContainsText fallback (same file) same commands on SDK-runtime backends without findText Skips hints (this commit)
waitForStablerunStableCaptureLoop (stable-capture.ts, shared with --settle) wait stable Retains full snapshot semantics — untouched per review scope; it passes no hints override, so the new option stays undefined there
handleFindWait (src/daemon/handlers/find.ts) none Dead code, already deleted on main by #1290 — gone after rebase
waitForNativeAlert (snapshot-alert.ts) alert wait No snapshot captures on the Apple-runner poll; the Android alert capture already passed includeHiddenContentHints: false on main (src/platforms/android/alert.ts:130) — the pre-existing precedent for this approach

Post-timeout diagnostics (wait-current-surface.ts, Android timeout-evidence screenshots) are one-shot captures after the wait already failed, not polls — left with full semantics.

New daemon/public-route regressions

In src/daemon/handlers/__tests__/snapshot-handler.test.ts, exercising handleSnapshotCommands with command: 'wait' (the exact daemon surface of the repro):

  • wait selector polling skips hidden-content hint derivation on every poll (#1270) — positionals ['label="Battery"', '8000'], miss-then-match over two polls, asserts every snapshot dispatch context carries snapshotIncludeHiddenContentHints: false.
  • wait text polling skips hidden-content hint derivation on every poll (#1270) — same shape for the text/ref route (['Battery', '8000']), which flows through the separate captureWaitSnapshot site.

Revert-sensitivity verified: with the skips stripped from waitForSelector and captureWaitSnapshot, both tests fail (expected undefined to be false); restored, 49/49 in the file pass.

Consolidation-candidate note (not done here — single concern)

The enumeration surfaced five distinct hand-rolled polling loops for wait-shaped commands (waitForFindMatch, waitForSelector, waitForText, runStableCaptureLoop, waitForNativeAlert), each re-implementing the deadline/poll/sleep skeleton with its own capture policy — this fix had to be applied at three separate capture sites as a result. Filing as evidence for a future poll-loop consolidation; deliberately not attempted in this PR.

Gates at 5046190f7

pnpm typecheck, oxlint --deny-warnings, format:check, check:fallow --base origin/main (0 new findings), and the full check:affected --base origin/main --run all green; known-flaky trio (runtime-hints, runner-client, apple index) additionally rerun in isolation, 177/177.

Live n=20 helper-path evidence rides the shared wave-3 validation flow, as discussed.

@thymikee

Copy link
Copy Markdown
Member Author

Current-head re-review (5046190f7): code review is clean. The prior wait-path gap is fixed across all polling routes: find-wait and selector wait explicitly disable hidden-content hint derivation; text/ref wait does so through the daemon captureWaitSnapshot route or the SDK snapshot fallback; wait stable retains full snapshot semantics. The new public daemon-route selector/text regressions are revert-sensitive, the internal flag propagates through dispatch to Android capture correctly, the rebase onto current main is clean, and all current CI checks are green.

Residual merge evidence: the requested live Android n=20 helper-path run is still pending. Please attach that evidence before merge; it is not a code-review finding.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Jul 16, 2026
@thymikee
thymikee merged commit bd62502 into main Jul 16, 2026
22 checks passed
@thymikee
thymikee deleted the fix/1270-scroll-hint-budget branch July 16, 2026 16:03
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-07-16 16:04 UTC

@thymikee

Copy link
Copy Markdown
Member Author

Post-merge live confirmation — n=20 replay matrix + Leg E overlay acceptance

Full n=20 replay of the Android Settings native-list flow (flowA-settings.ad) on the production helper path, varied across four environment buckets.

bucket runs clean false-divergence
plain 8 7 1
relaunch 4 4 0
daemon-restart 4 4 0
theme 4 4 0
total 20 19 1

FDR(runs) = 0.05, median clean wall 8.0s.

Per-class:

The lone divergence (plain run #2, step 6 scroll bottom) is a non-identity scroll-verify artifact, not a #1288 dumpsys-probe starvation and not a mis-binding: kind: action-failure, cause: COMMAND_FAILED "Failed to verify scroll bottom state", hint "scroll bottom needs a snapshot showing hidden content below before it will move." Reads as scroll-state carryover in a single plain run. All identity classes (get + press) are 20/20 clean.

Leg E — #1264/#1265 overlay acceptance

Resume-replay from the get step against the archived plan digest 0168fcf2….

(a) Volume dialog — not reproduced (timing/hittability limited, not a burial regression). Across plain, timed, and aggressive-pump (VOLUME_UP every 0.3s spanning the whole replay) attempts, the async divergence fast-capture never contained a systemui/volume node (0 systemui in screen.refs and in suggestions). Two compounding reasons on this AVD: the dialog is transient (~1.5s) and faded before the multi-step async capture fired; and its nodes are all non-clickable (clickable: null, only "Sound settings" labeled) — essentially no hittable dismiss target to rank. A manual snapshot --raw right after a fresh keyevent did capture the volume window (50 systemui + 8 volume-id nodes), confirming the capture path sees it when timing aligns and the ranking code (isForeignOverlayDismissTarget) is present.

(b) Quick-settings shade — captured-but-filtered (candidate new issue). cmd statusbar expand-settings (persistent) → divergence fast-capture returns screen.state: unavailable, reason: capture-failed, 0 refs. The archived raw capture tree distinguishes never-captured vs filtered: the helper captured 191 nodes, all systemui, 97 meaningful (windowType 3, applicationWindowRootCount: 0), but the Android content-recovery guard rejects the whole capture as content-poor-app-windowupstream of the divergence overlay-ranking. So a full-cover shade yields empty screen.refs despite 191 captured tiles. This does not violate the #1265 safety invariant (a plain snapshot also fails content-poor under the full shade, so both fail identically and the agent is never shown a false-healthy app), but the aspirational goal "shade tiles must survive into screen.refs" is not met for a full-cover overlay: the ranking only helps a partial-cover overlay where the app window is co-captured. Flagging as a follow-up.


Disclosure: combined tree cae03de8d, tree-hash-identical (ad3cf045…) to main 1fdbf80c3. Device: emulator-5556, Pixel 9 Pro XL API 37 (sdk_gphone16k_arm64, Android release 17 / API/SDK 37), production helper path (backend: android-helper, helper 0.19.3 / versionCode 19003). Methodology honesty: an earlier attempt was stopped at system load 101–166 (2 runs, excluded as env artifacts); this run was executed in a low-contention window (~7.5).

thymikee added a commit that referenced this pull request Jul 17, 2026
#1270) (#1314)

The snapshot helper is the only capture backend and emits `can-scroll-forward`/
`can-scroll-backward` for exactly the nodes Android reports as scrollable. Their
absence therefore means nothing on screen scrolls, not that scroll state is
unknown — so the `dumpsys activity top` probe only ever ran when there was no
scrollable content for it to describe.

#1288 bounded that probe at 1.5s rather than removing it, leaving every capture
of a screen with a scrollable-typed but non-scrollable node (a list short enough
to fit) paying the cap for hints that cannot exist. `dumpsys activity top`
serializes a view dump of every top activity through each app's main thread, so
one busy app stalls the call until Android's own 10s service-dump timeout — the
mechanism behind the 37ms-to-5.9s spread in the issue's evidence.

Hints for genuinely scrollable nodes are unaffected: they come from the parsed
helper attributes, which is where they already came from on every screen that
reports a scroll action.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant