Skip to content

fix(agent): skip invalid JSON candidates - #2683

Open
alectimison-maker wants to merge 2 commits into
webbrain-one:mainfrom
alectimison-maker:fix/skip-invalid-json-candidates
Open

fix(agent): skip invalid JSON candidates#2683
alectimison-maker wants to merge 2 commits into
webbrain-one:mainfrom
alectimison-maker:fix/skip-invalid-json-candidates

Conversation

@alectimison-maker

Copy link
Copy Markdown
Contributor

Summary

  • keep scanning model output after a balanced object-shaped candidate fails JSON.parse
  • preserve fenced-output priority, string escaping, and first-valid-object behavior
  • mirror the parser change across Chrome and Firefox and cover both planners with a regression test

Problem

Planner and classifier responses sometimes include an illustrative object before the final JSON result. When that first balanced candidate was not valid JSON, extractFirstJsonObject stopped scanning the candidate entirely, so a later valid plan was discarded.

Testing

  • node test/run.js: 1450 passed, 1 pre-existing repository failure (package.json is 26.0.10, newest CHANGELOG.md entry is 26.0.0)
  • node test/security/injection-corpus.mjs: 60/60 checks passed
  • Chrome/Firefox parser files are byte-identical

Scope

This does not relax plan validation or choose a later object when an earlier candidate is valid. It only advances after the earlier balanced candidate cannot be parsed as JSON.

@vercel

vercel Bot commented Aug 4, 2026

Copy link
Copy Markdown

@alectimison-maker is attempting to deploy a commit to the esokullu's projects Team on Vercel.

A member of the Team first needs to authorize it.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates shared planner/classifier JSON extraction to skip malformed balanced objects and continue to the first valid object.

Changes:

  • Continues scanning after JSON.parse failures.
  • Mirrors behavior across Chrome and Firefox.
  • Adds regression coverage for both planners.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
src/chrome/src/agent/json-extract.js Adds resilient candidate scanning.
src/firefox/src/agent/json-extract.js Mirrors Chrome parser behavior.
test/run.js Tests malformed-candidate recovery.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Skipping unparseable candidates covered the illustrative-object case but
not the neighbouring one: a `{` that never closes still abandoned the
whole candidate, so a stray brace in prose ahead of the real object
discarded it. Scanning now resumes one character past an unbalanced
opener, capped at 16 restarts so a "{{{{..." response stays linear.

This mirrors the scanner in tool-call-parser.js so the two agree on
recovery, and adds coverage for the brace flood plus the still-standing
limitation that an earlier *valid* non-plan object wins.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@esokullu

esokullu commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Pushed a commit extending the recovery to one more case. No Copilot findings on this one — its review was a summary only.

What I added. Skipping unparseable candidates covered the illustrative-object case but not its neighbour: a { that never closed still abandoned the whole candidate, so a stray brace in prose ahead of the real object discarded it. Scanning now resumes at start + 1 on an unbalanced opener, capped at 16 restarts to keep a pathological "{{{{…" response linear.

This is the same fix as #2682, deliberately. The two scanners had drifted apart on exactly this behaviour — each handled one half of the problem — which is worth knowing about since it's how the same bug ended up in both trees. They agree now.

Worth noting for reviewers, since the description mentions only the planner: this function also backs three classifier paths — agent.js:11454 (progress-intent), _isSafetyRefusalTerminal, and _looksLikePlanOnlyTerminal. I walked all three and none looks like a downgrade; every consumer shape-checks (normalizePlan requires summary, the refusal detector requires confidence === 0 plus refusal wording, the plan-only check requires ≥5 policy keys), and for the refusal detector the change makes a refusal more likely to be caught, which is the safe direction. Still, the blast radius is wider than "planner" suggests.

Tests. An unbalanced opener followed by two broken candidates then the real plan; the bounded brace flood; and one pinning the limitation this PR correctly scopes out — an earlier valid non-plan object still wins and returns null. That last one asserts current behaviour on purpose, so if anyone later moves to "first object that normalises," the test fails and forces the decision to be explicit rather than silent.

1453 passed, security corpus 60/60, Chrome and Firefox copies cmp-verified identical. The one failure is the pre-existing package.json 26.0.10 vs CHANGELOG 26.0.0 mismatch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants