fix: reject incomplete research reports before publication - #177
Merged
Conversation
Reject truncated or malformed research Markdown before publication. Validate citation provenance and Sources parity, and increase synthesis headroom.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
iamjr15
added a commit
that referenced
this pull request
Aug 5, 2026
## Summary - Publishes a successful deep-research tool's validated Markdown directly as assistant text in the same durable Workflow step that publishes its artifact. - Treats successful `research_deep` and `research_fanout` calls as terminal response producers, so no second model turn can summarize or rewrite the report. - Suppresses model pre-tool narration for terminal research calls, making the visible report body and PDF input identical by construction. ## Why Production QA for PR #177 proved that the PDF was complete and valid, but the outer agent ignored its prompt and posted a 1,386-character summary instead of the canonical report. Prompt-only enforcement cannot guarantee content identity. This moves ownership to the durable runtime boundary. ## Architecture The research tool continues to return `{ artifact, report }`, with `report` already validated against collected citation provenance. The Workflow projects that strict output into both destinations: 1. `artifact` becomes the durable deliverable part. 2. `report` becomes the assistant text part. 3. The run completes immediately after the successful tool turn. Both parts share the deterministic `tool:<model-step>:<tool-index>` publication receipt, so Workflow replay cannot duplicate either one. ## Decisions Made | Decision | Choice | Alternatives considered | Reasoning | |---|---|---|---| | Response ownership | Durable Workflow publishes canonical report | Ask the outer model to copy tool output | The production model summarized despite explicit instructions. | | Completion | Successful research tool is terminal | Run another model turn | A second turn can rewrite, truncate, or append content. | | Progress prose | Suppress research-call model text | Keep preamble before the report | Ensures visible report content and PDF source are the same Markdown. | | Failure path | Continue agent loop on tool error | Treat every research attempt as terminal | The model still needs to explain provider or validation failures. | | Replay behavior | Report and artifact share tool publication | Separate non-durable append | Existing atomic receipts already guarantee exactly-once transcript publication. | ## Edge Cases Handled | Scenario | Handling | |---|---| | Research tool fails | No canonical response is projected; the next model turn explains the error. | | Non-terminal search tool runs | Existing model loop is unchanged. | | Research output shape is invalid | Strict schema parsing fails the run rather than publishing divergent content. | | Worker/DO replays publication | Deterministic event receipt prevents duplicate report or artifact parts. | | Model emits a preamble with the tool call | It stays in model context but is not published to the user. | ## How to Review 1. Review `agent-run-research-response-support.ts` for terminal-tool classification and strict output projection. 2. Review `agent-run-workflow.ts` for suppression, atomic report publication, and terminal completion. 3. Confirm the agent-worker README and system prompt document the same ownership boundary. ## Verification - `pnpm lint` - `pnpm typecheck` - `pnpm turbo build --force` - `pnpm deadcode` - `pnpm architecture:check` - `pnpm turbo skills:build` - Direct projection contract: successful report preserved byte-for-byte; failed, non-terminal, and invalid outputs handled as designed. - Production evidence before this fix: reload recovery completed one run with one tool and one artifact, but persisted assistant text was a short model summary rather than the PDF's report.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Architecture
The synthesis model still produces the canonical Markdown displayed in chat and rendered into the PDF. A deterministic validation boundary now runs before
ResearchReportSchemasucceeds and before the artifact renderer or generated-output store is invoked. Invalid output uses the existing bounded in-memory retry; a second invalid result fails safely instead of publishing a broken report.Decisions Made
Edge Cases Handled
How to Review
research-markdown.tsfor the deterministic publication contract.deep-research-workflow.tsfor finish-reason wiring, output headroom, and synthesis instructions.packages/agent-core/README.mdmatches the runtime boundary.Verification
pnpm lintpnpm typecheckpnpm turbo build --forcepnpm deadcodepnpm architecture:checkpnpm turbo skills:build/listed it; visual inspection exposed the truncated final Markdown link this PR prevents.