Summary
First data point on driving a Capacitor (WKWebView) iOS app with agent-device. Full-fidelity accessibility snapshots work well; the one real gap is keyboard handling for web forms — there's no supported keyboard-dismiss or programmatic blur, so elements sitting under the on-screen keyboard mis-tap and text routes into the wrong field.
Environment: agent-device 0.17.2, iOS 26.5 / Xcode 26.5, iOS Simulator, a Capacitor app (Next.js static export in a WKWebView).
What works well (positive)
- Full-fidelity accessibility snapshots against the WKWebView: large trees (~336 nodes), including modifier/drawer overlays, serialize correctly.
- The
#701 serialization path never hard-failed across our runs.
- Roughly 1 in 50 snapshots came back keyboard-only/transient, and recovered cleanly on a retry — low-frequency, self-healing, not a blocker.
This is, as far as we can tell, the first published account of agent-device against a Capacitor/WKWebView target, so sharing it as a reference point.
The gap: keyboard dismiss / blur on web forms
When a web <input> is focused, the on-screen keyboard covers the lower portion of the webview, and we have no reliable way to dismiss it or blur the field:
keyboard dismiss → UNSUPPORTED_OPERATION on iOS.- There is no reliable programmatic blur primitive for a WKWebView form field.
- Consequence: taps aimed at elements underneath the keyboard mis-target, and typed text lands in the still-focused form field instead of the intended control.
keyboard enter → UNSUPPORTED, though type $'\n' (a literal newline) does work as a submit, so submit is achievable; dismiss/blur is the missing piece.
Suggestion
A supported keyboard-dismiss / blur primitive for WKWebView forms — e.g. an operation that resigns the first responder / dismisses the keyboard (equivalent to tapping "Done" or resigning focus) so that elements previously occluded by the keyboard become tappable again. This would close the main friction point we hit automating web forms inside a Capacitor app.
Happy to provide more detail or test a fix against our Capacitor app.
Summary
First data point on driving a Capacitor (WKWebView) iOS app with
agent-device. Full-fidelity accessibility snapshots work well; the one real gap is keyboard handling for web forms — there's no supported keyboard-dismiss or programmatic blur, so elements sitting under the on-screen keyboard mis-tap and text routes into the wrong field.Environment:
agent-device0.17.2, iOS 26.5 / Xcode 26.5, iOS Simulator, a Capacitor app (Next.js static export in a WKWebView).What works well (positive)
#701serialization path never hard-failed across our runs.This is, as far as we can tell, the first published account of
agent-deviceagainst a Capacitor/WKWebView target, so sharing it as a reference point.The gap: keyboard dismiss / blur on web forms
When a web
<input>is focused, the on-screen keyboard covers the lower portion of the webview, and we have no reliable way to dismiss it or blur the field:keyboard dismiss→UNSUPPORTED_OPERATIONon iOS.keyboard enter→UNSUPPORTED, thoughtype $'\n'(a literal newline) does work as a submit, so submit is achievable; dismiss/blur is the missing piece.Suggestion
A supported keyboard-dismiss / blur primitive for WKWebView forms — e.g. an operation that resigns the first responder / dismisses the keyboard (equivalent to tapping "Done" or resigning focus) so that elements previously occluded by the keyboard become tappable again. This would close the main friction point we hit automating web forms inside a Capacitor app.
Happy to provide more detail or test a fix against our Capacitor app.