Skip to content

fix(chatgpt): adapt to 2026-08 chatgpt.com UI and zero-size bridge viewports - #2436

Open
RusianHu wants to merge 3 commits into
jackwener:mainfrom
RusianHu:fix/chatgpt-2026-08-viewport-and-model
Open

fix(chatgpt): adapt to 2026-08 chatgpt.com UI and zero-size bridge viewports#2436
RusianHu wants to merge 3 commits into
jackwener:mainfrom
RusianHu:fix/chatgpt-2026-08-viewport-and-model

Conversation

@RusianHu

@RusianHu RusianHu commented Aug 29, 2026

Copy link
Copy Markdown

Summary

Fixes a cluster of chatgpt adapter failures against the 2026-08 chatgpt.com web UI (root cause report: #2435). All findings were reproduced against a real logged-in account (zh-CN UI) and verified end-to-end after the fix; the adapter's unit tests pass 180/180.

Root causes found (verified live, see #2435 for details)

  1. Zero-size viewport in bridge windows. The OpenCLI browser bridge runs tabs in a window where window.innerWidth/innerHeight === 0 while document.visibilityState === 'visible' (reproduced with --window foreground too). The new chatgpt.com UI sizes message/image containers from the viewport, so fluid-width nodes report rect.width === 0 while their text renders fine. Every isVisible that gates on rect.width > N then filters out exactly the nodes we need → ask TIMEOUT, read/detail EMPTY, image finds no generated images.
  2. Sidebar anchors are gone. Conversation items now render as bare ul > li > button with no attributes and no <a href="/c/..."> anywhere in the document → history EMPTY_RESULT.
  3. Model picker restructured. The trigger is now an unlabeled button showing a short effort label (observed extended → 高, min → 中, zh-CN), and the thinking-effort options live in a 选择模型 submenu that ignores synthetic pointer/keyboard events entirely (full pointerdown/mousedown/pointerup/mouseup/click sequence, trusted-key attempts — all no-ops), so the picker path cannot reach intelligence options.
  4. Upload preview chips are aria-label-only ([Bug]: chatgpt image --image exits 1 with 'image upload preview did not appear' after a successful backend upload (waitForChatGPTUploadPreview misses aria-label-only preview chip) #2302) — filename matching against body.innerText misses them.

Changes (all in clis/chatgpt/utils.js)

  • Viewport tolerance for layout-based visibility (15 isVisible sites): when document.documentElement.clientWidth <= 0, skip the rect gate; CSS display/visibility checks remain, so genuinely hidden aria-shadow copies are still excluded. On any normal viewport behavior is unchanged.
  • Conversation discovery: getConversationList now prefers the backend API (/api/auth/session token → /backend-api/conversations) executed in the page context; the anchor extraction is kept as fallback.
  • Model selection: added a last-resort effort patch via the settings API (/backend-api/settings/user_last_used_model_config) run in the page context (the CLI host may not be able to reach chatgpt.com directly; the browser session always can). It keeps the account's current model family from the oai-last-model-config cookie and patches only the effort (fast→min, balanced→standard, advanced→extended, very-high→xhigh — vocabulary confirmed by the API's own enum validation). The visible picker stays the primary path; the API fallback only fires when the picker cannot reach the option or the postcondition fails. Server-side eligibility rejects invalid combos with the config untouched.
  • Model trigger discovery: fallback to the composer's only unlabeled short-text button; clicks dispatch the full pointer/mouse event sequence (plain .click() does not open the menu) with coordinates returned as before so native clicks still work on real viewports.
  • Upload preview ([Bug]: chatgpt image --image exits 1 with 'image upload preview did not appear' after a successful backend upload (waitForChatGPTUploadPreview misses aria-label-only preview chip) #2302): match file names against aria-labels as well as visible text; media nodes in zero-size viewports are accepted via naturalWidth/videoWidth/background-image presence.
  • Terminal no-image detection (policy refusals): when ChatGPT ends the turn with a text-only reply instead of an image (policy refusals finish in ~1 min), waitForChatGPTImages now fails fast with the refusal text in the error instead of burning the full --timeout on a state that will never produce images. Conservative gating: requires two consecutive quiet polls with substantive (≥16 char) assistant text; if message extraction is unavailable it falls back to the existing deadline behavior. Covered by two new unit tests (182/182 pass).
  • Robustness: getCookies() calls race an 8s timeout (the full-jar call was observed to hang the daemon bridge); the post-PATCH reload is scheduled via setTimeout (reloading synchronously inside evaluate destroys the execution context and leaves the promise pending forever); the session fetch in buildChatGPTBackendHeaders catches network aborts instead of surfacing AbortError.

Verification

  • Live: ask (new + continuation), history, detail, read, whoami, status, new, image (text-to-image), image --image (edit), model (fast → advanced round trip, confirmed server-side via the config cookie).
  • npx vitest run clis/chatgpt → 180/180 pass.
  • opencli validate chatgpt → PASS (14 commands).
  • Refusal fast-fail covered by dedicated unit tests (text-only terminal state).

Notes for reviewers: the zero-size viewport is a property of our bridge environment (Windows, extension-connected daemon); the clientWidth <= 0 guards are no-ops on normal viewports. The history API path is strictly more robust than DOM anchors but the anchor fallback is retained in case the UI regresses or another locale renders anchors.

Update 2026-09-03: /c/WEB:<uuid> temporary route ids (new commit)

Re-testing ask against the live site on 2026-09-03 surfaced a further frontend change that also breaks upstream main, so it belongs in this PR:

Symptom. ask sends the message and the model replies (server-side conversation is created), but the CLI fails with ChatGPT did not create a conversation URL after sending the message.

Root cause (verified live). Brand-new conversations now route to /c/WEB:<uuid>:

  • <uuid> is a client-side temporary id — it never appears in backend-api, and it is not the real conversation id. Navigating directly to /c/WEB:<uuid> bounces back to the home page.
  • The id segment contains :, which the old parseChatGPTConversationId character class ([A-Za-z0-9_-]) rejects, so waitForConversationUrl times out after 30 s.
  • Observed post-condition: once generation finishes, the frontend itself swaps the URL for the real /c/<server-id>.

Fix (second commit, clis/chatgpt/utils.js + clis/chatgpt/ask.js):

  • parseChatGPTConversationId: the id character class now includes : so WEB: ids parse.
  • ask: when the parsed id is a WEB: temporary id, wait for the response first, then re-read the URL — if the frontend has already swapped in the real id, use it; otherwise fall back to the newest entry of the backend-api conversation listing (new resolveWebConversationId helper, reuses the existing page-context fetch from getConversationList).
  • The leave-conversation URL guard is skipped on that path: the WEB: → server-id swap is a legitimate navigation, and the old guard false-fires on it.

Verification (same battery as above, re-run after this commit):

  • Live: ask (new conversation) → returns the real server-side id and correct response; ask --conversation <real-id> continuation works.
  • npx vitest run --project adapter clis/chatgpt/ → 164/164 pass.
  • npx tsc --noEmit → clean.
  • npx tsx src/main.ts validate chatgpt → PASS (14 commands).

Fixes #2435

@RusianHu
RusianHu force-pushed the fix/chatgpt-2026-08-viewport-and-model branch 2 times, most recently from f52041b to d4f6ec0 Compare August 29, 2026 17:19
…ewports

- Conversation discovery: sidebar entries are now attribute-less buttons
  (no /c/ anchors); getConversationList prefers the page-context
  backend-api route and keeps anchor extraction as a fallback
- Message/image extraction: when the bridge window viewport is 0x0
  (visibilityState still 'visible'), fluid-width node rects collapse to 0
  and every rect-based visibility check fails; add clientWidth<=0 early
  exits to the isVisible implementations while keeping CSS
  display/visibility checks
- Model switching: the new UI's model submenu is immune to synthetic
  events; the selector path stays as a fallback. Effort tiers now go
  through the page-context settings API (preserving the current model
  family); triggers gain a no-aria-label short-text button fallback with
  a full pointer-event sequence click
- Upload preview: filename also matches aria-label (jackwener#2302); media nodes
  fall back to naturalWidth/videoWidth/background-image when the viewport
  is collapsed
- Robustness: getCookies races a timeout; reload defers via setTimeout to
  avoid a synchronous-navigation hang inside evaluate; session fetch
  catches network interruptions

Verified live against a logged-in zh-CN account: ask/history/detail/
read/image/image --image/model round-trips; chatgpt unit tests 180/180.

Fixes jackwener#2435
…26-09 frontend

- parseChatGPTConversationId: the id character class now includes ':';
  brand-new conversations route to /c/WEB:<uuid> where <uuid> is a
  client-side temporary id that never appears in the backend API
- ask: when the parsed id is a WEB: temporary id, wait for the response
  first, then re-read the URL - the frontend swaps it for the real
  /c/<server-id> once generation finishes; fall back to the newest
  backend-api conversation listing (new resolveWebConversationId) when
  it still carries the WEB: prefix
- ask: skip the leave-conversation URL guard on that path because the
  WEB: -> server-id swap is a legitimate navigation
…ecovery

When ask times out (or otherwise fails mid-wait) after the message was
sent, the conversation already exists on the server, but the thrown
TimeoutError carried no conversation context - callers could not tell
re-sending from waiting, and had to dig through chatgpt history to find
the id.

- attachConversationContext(): append [conversationId=... conversationUrl=...]
  to the error message and a ready-to-run 'opencli chatgpt detail <id>'
  recovery hint, mutating the caught error in place so code/exitCode
  (TIMEOUT/75) stay intact for script callers
- readConversationContext(): re-read the current URL on failure, preferring
  the real server id if the frontend has already swapped out WEB:<uuid>
- covered: both waitForChatGPTResponse branches (WEB: and regular) and the
  waitForConversationUrl 30s timeout (URL appended for diagnostics)

Verified: ask --timeout 8 on a long-form prompt now reports the
conversation id on TIMEOUT; 'detail <id>' then reads the session directly
(no history detour). Success path regression-checked.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: chatgpt history/ask broken — chatgpt.com sidebar no longer renders conversations as <a href="/c/..."> anchors

1 participant