fix: render research PDFs from canonical Markdown (#164)

## Summary

- Render the validated research report Markdown directly into the PDF
instead of flattening it into a separate section/paragraph model.
- Preserve the same headings, prose, lists, task items, tables, code,
links, citations, and ordering shown in chat.
- Keep structured claim provenance internal while returning a smaller
user-facing research tool result.

## Architecture

The research synthesis remains the single content source. The Worker
parses that bounded Markdown with a pinned GFM parser, stages the
Markdown and its semantic tokens in the existing hidden artifact-input
path, and the sandbox's existing React PDF runtime renders those tokens
into a durable workspace/R2 PDF.

No database, storage, sandbox snapshot, or deployment-topology change is
required.

## Decisions Made

| Decision | Choice | Alternatives considered | Reasoning |
|---|---|---|---|
| Content source | Use `report.report` verbatim | Rebuild a document
from claims and sources | Prevents chat/PDF drift and removes synthetic
appendices the user did not see in chat. |
| Markdown parsing | Pin `marked` in `agent-core` | Regex parsing; add
runtime dependencies | Gives standards-based GFM tokens without
requiring a new Daytona snapshot. |
| PDF runtime | Reuse React PDF and baked Liberation fonts | Browser
print-to-PDF; new PDF vendor | Keeps rendering deterministic,
sandbox-local, and compatible with the current document boundary. |
| Tool result | Return report + artifact only | Return all findings,
claims, and sources | Provenance is validated internally; the smaller
result makes verbatim chat presentation reliable. |

## Edge Cases Handled

| Scenario | Handling |
|---|---|
| Long multi-page reports | A4 wrapping, widow/orphan controls, and
heading presence rules prevent clipping and orphaned headings. |
| Nested, ordered, and task lists | Semantic list tokens retain nesting,
numbering, and task state. |
| Comparison tables | Header, alignment, cell borders, and row-safe
pagination are preserved. |
| Citations | Safe HTTP(S)/mailto links remain visible and clickable PDF
annotations. |
| Inline HTML or unsafe URLs | HTML is rendered as inert text and
unsupported URL schemes are not made clickable. |

## How to Review

1. Start with `research-report-document-support.ts` and
`research-tools.ts` for the one-source data flow.
2. Review `execute.ts` and `scripts.ts` for parsing, staging, and
semantic PDF rendering.
3. Review the synthesis/system prompts and README for the chat/PDF
contract.

## Verification

- [x] `pnpm lint`
- [x] `pnpm typecheck`
- [x] `pnpm turbo build --force`
- [x] `pnpm deadcode`
- [x] `pnpm architecture:check`
- [x] `pnpm turbo skills:build`
- [x] Rendered a five-page hostile GFM fixture and visually inspected
every page.
- [x] Verified extracted text, A4 metadata, and clickable link
annotations with Poppler.

Author: iamjr15

packages/agent-core/README.md

@@ -71,9 +71,11 @@ primary result. A single tool-free model pass structures each provider evidence
 pack, and workflow steps do not retry failed model output inside the same Worker
 invocation. Claim citations and the final synthesis are schema-validated against
 that evidence; prose URL scraping is not an accepted provenance boundary.
-Successful top-level deep-research and fan-out tools deterministically package
-that validated report, its claim-to-source map, and its source list as a PDF
-artifact. The project workspace is resolved only after remote research succeeds;
+Successful top-level deep-research and fan-out tools render the validated report's
+canonical GitHub-flavored Markdown directly into a PDF artifact. The chat response
+and PDF therefore preserve the same headings, prose, lists, tables, links, citations,
+and ordering; only print-safe pagination and document chrome differ. The project
+workspace is resolved only after remote research succeeds;
 the PDF is then stored both in the live project files and the durable
 generated-output store. Document generators stage their bounded structured input
 in a hidden, project-local temporary file instead of embedding it in the sandbox

packages/agent-core/package.json

@@ -43,6 +43,7 @@
     "@openrouter/ai-sdk-provider": "catalog:",
     "ai": "catalog:",
     "arquero": "catalog:",
+    "marked": "catalog:",
     "zod": "catalog:"
   },
   "devDependencies": {

packages/agent-core/src/mastra/system-prompt.ts

@@ -233,7 +233,7 @@ Load the generate-media skill before creating or editing an image or generating
 
 const RESEARCH_MODULE = `## Research
 
-Gather sources with search_web / search_web_advanced / search_company / search_web_content, then use search_scrape or search_extract for source retrieval. For an explicit deep-research request, cited report, market analysis, due diligence, or comprehensive investigation of one topic, use research_deep with 3 queries for a concise/narrow report, 4 by default, and 5-6 only when the user explicitly asks for deeper coverage. Use research_fanout when the request compares many entities or independent angles. Call the selected research workflow once per user request; if it fails, explain the failure instead of immediately rerunning it. Those workflows produce the complete cited PDF deliverable automatically, so do not regenerate it with a document tool. Treat search snippets as leads, not sources — open the real pages and cross-check. Cite as you go: attribute each claim to its source inline with the page title and its URL, and make sure every citation resolves. End a research answer with a short Sources list of the URLs you actually used.`;
+Gather sources with search_web / search_web_advanced / search_company / search_web_content, then use search_scrape or search_extract for source retrieval. For an explicit deep-research request, cited report, market analysis, due diligence, or comprehensive investigation of one topic, use research_deep with 3 queries for a concise/narrow report, 4 by default, and 5-6 only when the user explicitly asks for deeper coverage. Use research_fanout when the request compares many entities or independent angles. Call the selected research workflow once per user request; if it fails, explain the failure instead of immediately rerunning it. Those workflows produce the complete cited PDF deliverable automatically, so do not regenerate it with a document tool. After a research workflow succeeds, present its returned report Markdown as the response body without summarizing, restructuring, or appending hidden claim-map data; the PDF is rendered from that same Markdown. Treat search snippets as leads, not sources — open the real pages and cross-check. Cite as you go: attribute each claim to its source inline with the page title and its URL, and make sure every citation resolves. End a research answer with a short Sources list of the URLs you actually used.`;
 
 /** Compact all-domains pointer for an ambiguous general request — keeps the model aware without the full modules. */
 const GENERALIST_MODULE = `## Choosing your approach

packages/agent-core/src/mastra/tool-defs/research-report-document-support.ts

@@ -1,116 +1,21 @@
-import type { GenerateDocumentInput } from "../../tools/docs/schemas";
-import type { ResearchReport, ResearchSource } from "../workflows/research-schemas";
+import type { GenerateMarkdownPdfInput } from "../../tools/docs/schemas";
 
-const MAX_NARRATIVE_SECTIONS = 60;
-const MAX_APPENDIX_SECTIONS = 10;
-const MAX_PARAGRAPH_LENGTH = 5_000;
-const PARAGRAPHS_PER_SECTION = 20;
-
-interface DocumentSection {
-  heading: string;
-  paragraphs: string[];
-}
-
-export function buildResearchReportDocument(
-  report: ResearchReport,
+export function buildResearchMarkdownPdfInput(
+  markdown: string,
   topic: string,
-): GenerateDocumentInput {
-  const title = `Research report: ${cleanInlineMarkdown(topic)}`;
-  const sourceById = new Map(report.sources.map((source) => [source.id, source]));
-  const narrative = narrativeSections(report.report).slice(0, MAX_NARRATIVE_SECTIONS);
-  const evidence = appendixSections(
-    "Evidence map",
-    report.claims.map((claim) => evidenceParagraph(claim.claim, claim.sourceIds, sourceById)),
-  );
-  const sources = appendixSections("Sources", report.sources.map(sourceParagraph));
-
+): GenerateMarkdownPdfInput {
+  const fallbackTitle = cleanTitle(topic) || "Research";
   return {
     filename: researchFilename(topic),
-    sections: [...narrative, ...evidence, ...sources],
-    title: clampText(title),
+    markdown,
+    title: markdownTitle(markdown) ?? `Research report: ${fallbackTitle}`,
   };
 }
 
-function narrativeSections(markdown: string): DocumentSection[] {
-  const sections: DocumentSection[] = [];
-  let heading = "Research findings";
-  let paragraphs: string[] = [];
-
-  for (const block of markdown.split(/\n\s*\n/u)) {
-    const lines = block.split("\n").map((line) => line.trim());
-    const firstLine = lines[0] ?? "";
-    const headingMatch = /^(?:#{1,6})\s+(.+)$/u.exec(firstLine);
-    if (headingMatch) {
-      appendSection(sections, heading, paragraphs);
-      heading = cleanInlineMarkdown(headingMatch[1] ?? "Research findings");
-      paragraphs = lines.slice(1).flatMap(cleanDocumentLine);
-      continue;
-    }
-    paragraphs.push(...lines.flatMap(cleanDocumentLine));
-  }
-
-  appendSection(sections, heading, paragraphs);
-  return sections.length > 0
-    ? sections
-    : [{ heading: "Research findings", paragraphs: [clampText(cleanInlineMarkdown(markdown))] }];
-}
-
-function appendSection(sections: DocumentSection[], heading: string, paragraphs: string[]): void {
-  for (const [index, chunk] of chunks(paragraphs, PARAGRAPHS_PER_SECTION).entries()) {
-    sections.push({
-      heading: index === 0 ? clampText(heading) : clampText(`${heading} (continued)`),
-      paragraphs: chunk,
-    });
-  }
-}
-
-function appendixSections(heading: string, paragraphs: string[]): DocumentSection[] {
-  return chunks(paragraphs, PARAGRAPHS_PER_SECTION)
-    .slice(0, MAX_APPENDIX_SECTIONS)
-    .map((chunk, index) => ({
-      heading: index === 0 ? heading : `${heading} (continued)`,
-      paragraphs: chunk,
-    }));
-}
-
-function evidenceParagraph(
-  claim: string,
-  sourceIds: string[],
-  sourceById: ReadonlyMap<string, ResearchSource>,
-): string {
-  const sources = sourceIds
-    .map((sourceId) => sourceById.get(sourceId))
-    .filter((source): source is ResearchSource => source !== undefined)
-    .map(sourceLabel);
-  return clampText(`${cleanInlineMarkdown(claim)}\nSources: ${sources.join("; ")}`);
-}
-
-function sourceParagraph(source: ResearchSource): string {
-  return clampText(`${source.title?.trim() || source.url}\n${source.url}`);
-}
-
-function sourceLabel(source: ResearchSource): string {
-  return source.title?.trim() ? `${source.title.trim()} (${source.url})` : source.url;
-}
-
-function cleanDocumentLine(line: string): string[] {
-  const cleaned = cleanInlineMarkdown(line)
-    .replace(/^[-*+]\s+/u, "• ")
-    .replace(/^\d+[.)]\s+/u, (prefix) => `${prefix} `)
-    .replace(/^\|(.+)\|$/u, "$1")
-    .replace(/\s*\|\s*/gu, " — ")
-    .trim();
-  return cleaned && !/^[-: ]+$/u.test(cleaned) ? [clampText(cleaned)] : [];
-}
-
-function cleanInlineMarkdown(value: string): string {
-  return value
-    .replace(/!\[([^\]]*)\]\([^)]*\)/gu, "$1")
-    .replace(/\[([^\]]+)\]\((https?:\/\/[^)]+)\)/gu, "$1 ($2)")
-    .replace(/`([^`]+)`/gu, "$1")
-    .replace(/[*_~]+/gu, "")
-    .replace(/\s+/gu, " ")
-    .trim();
+function markdownTitle(markdown: string): string | undefined {
+  const heading = /^#\s+(.+)$/mu.exec(markdown)?.[1];
+  const title = heading ? cleanTitle(heading) : "";
+  return title || undefined;
 }
 
 function researchFilename(topic: string): string {
@@ -123,17 +28,12 @@ function researchFilename(topic: string): string {
   return `research-${slug || "report"}.pdf`;
 }
 
-function clampText(value: string): string {
-  const normalized = value.trim();
-  return normalized.length <= MAX_PARAGRAPH_LENGTH
-    ? normalized
-    : `${normalized.slice(0, MAX_PARAGRAPH_LENGTH - 1)}…`;
-}
-
-function chunks<T>(values: T[], size: number): T[][] {
-  const output: T[][] = [];
-  for (let index = 0; index < values.length; index += size) {
-    output.push(values.slice(index, index + size));
-  }
-  return output;
+function cleanTitle(value: string): string {
+  return value
+    .replace(/!\[([^\]]*)\]\([^)]*\)/gu, "$1")
+    .replace(/\[([^\]]+)\]\([^)]+\)/gu, "$1")
+    .replace(/[`*_~]+/gu, "")
+    .replace(/\s+/gu, " ")
+    .trim()
+    .slice(0, 5_000);
 }

packages/agent-core/src/mastra/tool-defs/research-tools.ts

@@ -1,6 +1,7 @@
 import { createTool, type ToolExecutionContext } from "@mastra/core/tools";
-import { executeGeneratePdf } from "../../tools/docs/execute";
-import { GeneratePdfOutputSchema } from "../../tools/docs/schemas";
+import { z } from "zod/v4";
+import { executeGenerateMarkdownPdf } from "../../tools/docs/execute";
+import { GenerateMarkdownPdfOutputSchema } from "../../tools/docs/schemas";
 import {
   ExaSearchInputSchema,
   ExaSearchOutputSchema,
@@ -27,12 +28,13 @@ import {
   firecrawlSource,
   registerResearchSources,
 } from "../workflows/research-provenance";
-import { buildResearchReportDocument } from "./research-report-document-support";
+import { buildResearchMarkdownPdfInput } from "./research-report-document-support";
 import { researchRuntimeFromContext, workspaceRuntimeFromContext } from "./tool-runtime-context";
 import { WorkflowResultSchema } from "./tool-schemas";
 
-const ResearchReportArtifactSchema = ResearchReportSchema.extend({
-  artifact: GeneratePdfOutputSchema,
+const ResearchReportArtifactSchema = z.strictObject({
+  artifact: GenerateMarkdownPdfOutputSchema,
+  report: ResearchReportSchema.shape.report,
 });
 
 type RequestContextReader = { get(key: string): unknown };
@@ -297,11 +299,11 @@ async function createResearchReportArtifact(
   context: ToolExecutionContext,
 ) {
   context.abortSignal?.throwIfAborted();
-  const artifact = await executeGeneratePdf(
-    buildResearchReportDocument(report, topic),
+  const artifact = await executeGenerateMarkdownPdf(
+    buildResearchMarkdownPdfInput(report.report, topic),
     await workspaceRuntimeFromContext(context),
   );
-  return ResearchReportArtifactSchema.parse({ ...report, artifact });
+  return ResearchReportArtifactSchema.parse({ artifact, report: report.report });
 }
 
 async function executeExaTool(input: unknown, context: ToolExecutionContext) {

packages/agent-core/src/mastra/workflows/deep-research-workflow.ts

@@ -305,6 +305,8 @@ function researchSynthesisPrompt(
     config.synthesisPrompt(findings),
     "Consolidate overlapping evidence into at most 16 distinct claims with no more than 4 source IDs per claim.",
     "Keep the report focused and complete within 2,000 words while retaining actionable findings and citations.",
+    "Write report as polished GitHub-flavored Markdown for direct display and PDF rendering. Preserve a clear heading hierarchy, lists, and comparison tables where useful.",
+    "Cite factual claims with descriptive Markdown links to the exact source URLs in the findings, and finish with a Sources heading containing only sources used in the report.",
   ].join("\n");
 }
 

packages/agent-core/src/tools/docs/execute.ts

@@ -4,12 +4,17 @@ import type {
   ArtifactUploadResult,
   CodeRuntimeContext,
 } from "@cheatcode/sandbox-contracts";
+import { lexer } from "marked";
 import { z } from "zod";
 import {
   type GenerateDocumentInput,
   GenerateDocumentInputSchema,
   type GenerateDocxOutput,
   GenerateDocxOutputSchema,
+  type GenerateMarkdownPdfInput,
+  GenerateMarkdownPdfInputSchema,
+  type GenerateMarkdownPdfOutput,
+  GenerateMarkdownPdfOutputSchema,
   type GeneratePdfOutput,
   GeneratePdfOutputSchema,
   type GenerateSlidesInput,
@@ -21,7 +26,13 @@ import {
   type GenerateXlsxOutput,
   GenerateXlsxOutputSchema,
 } from "./schemas";
-import { buildDocxScript, buildPdfScript, buildSlidesScript, buildXlsxScript } from "./scripts";
+import {
+  buildDocxScript,
+  buildMarkdownPdfScript,
+  buildPdfScript,
+  buildSlidesScript,
+  buildXlsxScript,
+} from "./scripts";
 
 const SandboxArtifactSchema = z.strictObject({
   base64: z.string().min(1),
@@ -84,6 +95,26 @@ export async function executeGeneratePdf(
   });
 }
 
+export async function executeGenerateMarkdownPdf(
+  input: GenerateMarkdownPdfInput,
+  runtimeContext: CodeRuntimeContext,
+): Promise<GenerateMarkdownPdfOutput> {
+  const parsed = GenerateMarkdownPdfInputSchema.parse(input);
+  const filename = normalizeFilename(parsed.filename ?? parsed.title ?? "research-report", "pdf");
+  const tokens = lexer(parsed.markdown, { gfm: true });
+  const artifact = await runArtifactScript(
+    { markdown: parsed.markdown, title: parsed.title, tokens },
+    runtimeContext,
+    "pdf",
+    (inputPath) => buildMarkdownPdfScript(inputPath, filename),
+  );
+  return GenerateMarkdownPdfOutputSchema.parse({
+    ...artifact,
+    blockCount: tokens.filter((token) => token.type !== "space").length,
+    kind: "pdf",
+  });
+}
+
 export async function executeGenerateXlsx(
   input: GenerateSpreadsheetInput,
   runtimeContext: CodeRuntimeContext,

packages/agent-core/src/tools/docs/schemas.ts

@@ -34,6 +34,12 @@ export const GenerateDocumentInputSchema = z.strictObject({
   title: TextValueSchema,
 });
 
+export const GenerateMarkdownPdfInputSchema = z.strictObject({
+  filename: z.string().trim().min(1).max(160).optional(),
+  markdown: z.string().trim().min(1).max(120_000),
+  title: TextValueSchema.optional(),
+});
+
 const SpreadsheetCellSchema = z.union([z.string(), z.number(), z.boolean(), z.null()]);
 const SpreadsheetRowSchema = z.record(z.string().min(1).max(80), SpreadsheetCellSchema);
 
@@ -61,6 +67,12 @@ export const GeneratePdfOutputSchema = z.strictObject({
   sectionCount: z.number().int().positive(),
 });
 
+export const GenerateMarkdownPdfOutputSchema = z.strictObject({
+  ...ArtifactOutputSchema.shape,
+  blockCount: z.number().int().positive(),
+  kind: z.literal("pdf"),
+});
+
 export const GenerateSlidesOutputSchema = z.strictObject({
   ...ArtifactOutputSchema.shape,
   kind: z.literal("slide"),
@@ -75,6 +87,8 @@ export const GenerateXlsxOutputSchema = z.strictObject({
 
 export type GenerateDocumentInput = z.input<typeof GenerateDocumentInputSchema>;
 export type GenerateDocxOutput = z.output<typeof GenerateDocxOutputSchema>;
+export type GenerateMarkdownPdfInput = z.input<typeof GenerateMarkdownPdfInputSchema>;
+export type GenerateMarkdownPdfOutput = z.output<typeof GenerateMarkdownPdfOutputSchema>;
 export type GeneratePdfOutput = z.output<typeof GeneratePdfOutputSchema>;
 export type GenerateSlidesInput = z.input<typeof GenerateSlidesInputSchema>;
 export type GenerateSlidesOutput = z.output<typeof GenerateSlidesOutputSchema>;