feat: generate PDF deliverables for deep research (#155)

iamjr15 · web-flow · commit 558e9882cb9e · 2026-08-05T02:37:15.000+05:30
## Summary

- makes explicit deep-research and fan-out runs return a durable PDF
deliverable by default
- stores the same PDF in the live project workspace so it remains
available from Files and slash file references
- keeps quick factual research remote-only and projectless
- updates the bundled Research skill and agent guidance so models
consistently select the deterministic workflow

## Architecture

The evidence-validated research workflow still owns search, provenance,
and synthesis. After it succeeds, the top-level research tool
deterministically converts that validated result into a structured
document containing the narrative, claim-to-source evidence map, and
authoritative source list. The existing PDF generator writes the file to
the lazily resolved project workspace and durable generated-output
storage, and the capability catalog emits it through the normal
Deliverables stream.

## Decisions Made

| Decision | Choice | Alternatives considered | Reasoning |
|---|---|---|---|
| PDF creation | Generate inside the top-level research tools | Ask the
model to call a second document tool | Makes delivery deterministic and
prevents model-specific omissions |
| Workspace lifecycle | Resolve the project after research succeeds |
Create a project when research begins | Avoids leaving empty projects
after provider or synthesis failures |
| Report content | Package validated narrative, evidence map, and
sources | Render only the chat prose | Keeps the downloadable report
evidence-bound and independently useful |
| Quick research | Leave direct search tools unchanged | Generate a PDF
for every lookup | Preserves low latency for simple questions |

## Edge Cases Handled

| Scenario | Handling |
|---|---|
| Long reports | Sections and paragraphs are bounded to the document
schema limits |
| Repeated or unsafe filenames | Existing normalized artifact filenames
and unique output IDs remain authoritative |
| Missing project selection | The existing lazy workspace resolver
creates and attaches a project only when the PDF is ready |
| Cancellation after research | The abort signal is checked before
workspace and PDF work starts |
| Markdown report formatting | Headings, links, lists, and tables are
normalized into readable PDF sections |

## Verification

- [x] `pnpm lint`
- [x] `pnpm typecheck`
- [x] `pnpm turbo build --force`
- [x] `pnpm deadcode`
- [x] `pnpm architecture:check`
- [x] `pnpm turbo skills:build`
- [x] exercised the report converter with representative headings,
citations, evidence, and sources
- [ ] production Research flow after deployment
diff --git a/packages/agent-core/README.md b/packages/agent-core/README.md
@@ -68,7 +68,11 @@ the Mastra workflow run, forward each workflow step signal through every nested
 Each concurrent research pass gets an isolated evidence collector populated only
 from parsed Exa result IDs/URLs and Firecrawl result URLs. Claim citations and the
 final synthesis are schema-validated against that evidence; prose URL scraping
-is not an accepted provenance boundary.
+is not an accepted provenance boundary. Successful top-level deep-research and
+fan-out tools deterministically package that validated report, its claim-to-source
+map, and its source list as a PDF artifact. The project workspace is resolved only
+after remote research succeeds; the PDF is then stored both in the live project
+files and the durable generated-output store.
 
 Composio REST tool discovery and execution responses are byte-bounded before
 parsing, then projected into bounded, valid JSON before entering model context.
diff --git a/packages/agent-core/src/mastra/system-prompt.ts b/packages/agent-core/src/mastra/system-prompt.ts
@@ -232,7 +232,7 @@ Load the generate-media skill before creating or editing an image or generating
 
 const RESEARCH_MODULE = `## Research
 
-Gather sources with search_web / search_web_advanced / search_company / search_web_content, then use search_scrape or search_extract for source retrieval; use research_deep and research_fanout for cited multi-source reports. Treat search snippets as leads, not sources — open the real pages and cross-check. Cite as you go: attribute each claim to its source inline with the page title and its URL, and make sure every citation resolves. End a research answer with a short Sources list of the URLs you actually used.`;
+Gather sources with search_web / search_web_advanced / search_company / search_web_content, then use search_scrape or search_extract for source retrieval. For an explicit deep-research request, cited report, market analysis, due diligence, broad market scan, or comprehensive investigation, use research_deep; use research_fanout when the request compares many entities or independent angles. Those workflows produce the complete cited PDF deliverable automatically, so do not regenerate it with a document tool. Treat search snippets as leads, not sources — open the real pages and cross-check. Cite as you go: attribute each claim to its source inline with the page title and its URL, and make sure every citation resolves. End a research answer with a short Sources list of the URLs you actually used.`;
 
 /** Compact all-domains pointer for an ambiguous general request — keeps the model aware without the full modules. */
 const GENERALIST_MODULE = `## Choosing your approach
@@ -242,7 +242,7 @@ Pick the path that fits and load the matching skill (skill_invoke) for its full
 - Slides or documents → build with pptxgenjs / docx / @react-pdf / exceljs (or docs_generate_*), then render and eyeball every page; the file lands in the Deliverables.
 - Data → profile it (data_analyze_csv, or pandas / Node) and chart it (data_chart) when it adds insight; verify the numbers.
 - Image or video → load generate-media, then use generate_or_edit_media; the asset lands in the project and Deliverables.
-- Research → gather and cross-check real sources (search_web / firecrawl_* / research_deep); cite everything.
+- Research → gather and cross-check real sources (search_web / firecrawl_* / research_deep); cite everything, and use the deep-research workflow when the user asks for a report so its PDF is delivered automatically.
 - Acting in the user's connected apps → composio_list_tools then composio_execute, only when they ask.`;
 
 const FINISHING = `## Finishing
diff --git a/packages/agent-core/src/mastra/tool-defs/research-report-document-support.ts b/packages/agent-core/src/mastra/tool-defs/research-report-document-support.ts
@@ -0,0 +1,139 @@
+import type { GenerateDocumentInput } from "../../tools/docs/schemas";
+import type { ResearchReport, ResearchSource } from "../workflows/research-schemas";
+
+const MAX_NARRATIVE_SECTIONS = 60;
+const MAX_APPENDIX_SECTIONS = 10;
+const MAX_PARAGRAPH_LENGTH = 5_000;
+const PARAGRAPHS_PER_SECTION = 20;
+
+interface DocumentSection {
+  heading: string;
+  paragraphs: string[];
+}
+
+export function buildResearchReportDocument(
+  report: ResearchReport,
+  topic: string,
+): GenerateDocumentInput {
+  const title = `Research report: ${cleanInlineMarkdown(topic)}`;
+  const sourceById = new Map(report.sources.map((source) => [source.id, source]));
+  const narrative = narrativeSections(report.report).slice(0, MAX_NARRATIVE_SECTIONS);
+  const evidence = appendixSections(
+    "Evidence map",
+    report.claims.map((claim) => evidenceParagraph(claim.claim, claim.sourceIds, sourceById)),
+  );
+  const sources = appendixSections("Sources", report.sources.map(sourceParagraph));
+
+  return {
+    filename: researchFilename(topic),
+    sections: [...narrative, ...evidence, ...sources],
+    title: clampText(title),
+  };
+}
+
+function narrativeSections(markdown: string): DocumentSection[] {
+  const sections: DocumentSection[] = [];
+  let heading = "Research findings";
+  let paragraphs: string[] = [];
+
+  for (const block of markdown.split(/\n\s*\n/u)) {
+    const lines = block.split("\n").map((line) => line.trim());
+    const firstLine = lines[0] ?? "";
+    const headingMatch = /^(?:#{1,6})\s+(.+)$/u.exec(firstLine);
+    if (headingMatch) {
+      appendSection(sections, heading, paragraphs);
+      heading = cleanInlineMarkdown(headingMatch[1] ?? "Research findings");
+      paragraphs = lines.slice(1).flatMap(cleanDocumentLine);
+      continue;
+    }
+    paragraphs.push(...lines.flatMap(cleanDocumentLine));
+  }
+
+  appendSection(sections, heading, paragraphs);
+  return sections.length > 0
+    ? sections
+    : [{ heading: "Research findings", paragraphs: [clampText(cleanInlineMarkdown(markdown))] }];
+}
+
+function appendSection(sections: DocumentSection[], heading: string, paragraphs: string[]): void {
+  for (const [index, chunk] of chunks(paragraphs, PARAGRAPHS_PER_SECTION).entries()) {
+    sections.push({
+      heading: index === 0 ? clampText(heading) : clampText(`${heading} (continued)`),
+      paragraphs: chunk,
+    });
+  }
+}
+
+function appendixSections(heading: string, paragraphs: string[]): DocumentSection[] {
+  return chunks(paragraphs, PARAGRAPHS_PER_SECTION)
+    .slice(0, MAX_APPENDIX_SECTIONS)
+    .map((chunk, index) => ({
+      heading: index === 0 ? heading : `${heading} (continued)`,
+      paragraphs: chunk,
+    }));
+}
+
+function evidenceParagraph(
+  claim: string,
+  sourceIds: string[],
+  sourceById: ReadonlyMap<string, ResearchSource>,
+): string {
+  const sources = sourceIds
+    .map((sourceId) => sourceById.get(sourceId))
+    .filter((source): source is ResearchSource => source !== undefined)
+    .map(sourceLabel);
+  return clampText(`${cleanInlineMarkdown(claim)}\nSources: ${sources.join("; ")}`);
+}
+
+function sourceParagraph(source: ResearchSource): string {
+  return clampText(`${source.title?.trim() || source.url}\n${source.url}`);
+}
+
+function sourceLabel(source: ResearchSource): string {
+  return source.title?.trim() ? `${source.title.trim()} (${source.url})` : source.url;
+}
+
+function cleanDocumentLine(line: string): string[] {
+  const cleaned = cleanInlineMarkdown(line)
+    .replace(/^[-*+]\s+/u, "• ")
+    .replace(/^\d+[.)]\s+/u, (prefix) => `${prefix} `)
+    .replace(/^\|(.+)\|$/u, "$1")
+    .replace(/\s*\|\s*/gu, " — ")
+    .trim();
+  return cleaned && !/^[-: ]+$/u.test(cleaned) ? [clampText(cleaned)] : [];
+}
+
+function cleanInlineMarkdown(value: string): string {
+  return value
+    .replace(/!\[([^\]]*)\]\([^)]*\)/gu, "$1")
+    .replace(/\[([^\]]+)\]\((https?:\/\/[^)]+)\)/gu, "$1 ($2)")
+    .replace(/`([^`]+)`/gu, "$1")
+    .replace(/[*_~]+/gu, "")
+    .replace(/\s+/gu, " ")
+    .trim();
+}
+
+function researchFilename(topic: string): string {
+  const slug = topic
+    .trim()
+    .toLowerCase()
+    .replace(/[^a-z0-9]+/gu, "-")
+    .replace(/^-+|-+$/gu, "")
+    .slice(0, 110);
+  return `research-${slug || "report"}.pdf`;
+}
+
+function clampText(value: string): string {
+  const normalized = value.trim();
+  return normalized.length <= MAX_PARAGRAPH_LENGTH
+    ? normalized
+    : `${normalized.slice(0, MAX_PARAGRAPH_LENGTH - 1)}…`;
+}
+
+function chunks<T>(values: T[], size: number): T[][] {
+  const output: T[][] = [];
+  for (let index = 0; index < values.length; index += size) {
+    output.push(values.slice(index, index + size));
+  }
+  return output;
+}
diff --git a/packages/agent-core/src/mastra/tool-defs/research-tools.ts b/packages/agent-core/src/mastra/tool-defs/research-tools.ts
@@ -1,4 +1,6 @@
 import { createTool, type ToolExecutionContext } from "@mastra/core/tools";
+import { executeGeneratePdf } from "../../tools/docs/execute";
+import { GeneratePdfOutputSchema } from "../../tools/docs/schemas";
 import {
   ExaSearchInputSchema,
   ExaSearchOutputSchema,
@@ -25,9 +27,14 @@ import {
   firecrawlSource,
   registerResearchSources,
 } from "../workflows/research-provenance";
-import { researchRuntimeFromContext } from "./tool-runtime-context";
+import { buildResearchReportDocument } from "./research-report-document-support";
+import { researchRuntimeFromContext, workspaceRuntimeFromContext } from "./tool-runtime-context";
 import { WorkflowResultSchema } from "./tool-schemas";
 
+const ResearchReportArtifactSchema = ResearchReportSchema.extend({
+  artifact: GeneratePdfOutputSchema,
+});
+
 type RequestContextReader = { get(key: string): unknown };
 type MutableRequestContext = RequestContextReader & {
   delete(key: string): boolean;
@@ -241,31 +248,50 @@ export const mastraFirecrawlExtract = createTool({
 export const mastraDeepResearch = createTool({
   id: "research_deep",
   description:
-    "Run the Deep Research workflow for a complex topic. It fans out focused research queries and returns a cited report.",
+    "Run the Deep Research workflow for a complex topic. It returns a cited report and saves the complete report as a PDF deliverable in the project.",
   inputSchema: DeepResearchInputSchema,
-  outputSchema: ResearchReportSchema,
-  execute: async (input, context) =>
-    runResearchWorkflow({
+  outputSchema: ResearchReportArtifactSchema,
+  execute: async (input, context) => {
+    const parsedInput = DeepResearchInputSchema.parse(input);
+    const report = await runResearchWorkflow({
       context,
-      inputData: DeepResearchInputSchema.parse(input),
+      inputData: parsedInput,
       workflowName: "deepResearch",
-    }),
+    });
+    return createResearchReportArtifact(report, parsedInput.topic, context);
+  },
 });
 
 export const mastraResearchFanout = createTool({
   id: "research_fanout",
   description:
-    "Run the Deep Research fan-out workflow across multiple entities or angles and return a comparison matrix style report.",
+    "Run the Deep Research fan-out workflow across multiple entities or angles. It returns a cited comparison report and saves the complete report as a PDF deliverable in the project.",
   inputSchema: DeepResearchFanoutInputSchema,
-  outputSchema: ResearchReportSchema,
-  execute: async (input, context) =>
-    runResearchWorkflow({
+  outputSchema: ResearchReportArtifactSchema,
+  execute: async (input, context) => {
+    const parsedInput = DeepResearchFanoutInputSchema.parse(input);
+    const report = await runResearchWorkflow({
       context,
-      inputData: DeepResearchFanoutInputSchema.parse(input),
+      inputData: parsedInput,
       workflowName: "deepResearchFanout",
-    }),
+    });
+    return createResearchReportArtifact(report, parsedInput.goal, context);
+  },
 });
 
+async function createResearchReportArtifact(
+  report: ResearchReport,
+  topic: string,
+  context: ToolExecutionContext,
+) {
+  context.abortSignal?.throwIfAborted();
+  const artifact = await executeGeneratePdf(
+    buildResearchReportDocument(report, topic),
+    await workspaceRuntimeFromContext(context),
+  );
+  return ResearchReportArtifactSchema.parse({ ...report, artifact });
+}
+
 async function executeExaTool(input: unknown, context: ToolExecutionContext) {
   const output = await executeExaSearch(
     ExaSearchInputSchema.parse(input),
diff --git a/packages/types/src/capabilities.ts b/packages/types/src/capabilities.ts
@@ -62,8 +62,18 @@ export const TOOL_CAPABILITIES = [
   tool("research", "search_extract", "Extract structured data with Firecrawl.", REMOTE_TOOL),
   tool("research", "search_scrape", "Scrape a known URL with Firecrawl.", REMOTE_TOOL),
   tool("research", "search_web_content", "Search and scrape with Firecrawl.", REMOTE_TOOL),
-  tool("research", "research_deep", "Run the deep research workflow.", REMOTE_TOOL),
-  tool("research", "research_fanout", "Run the deep research fan-out workflow.", REMOTE_TOOL),
+  tool(
+    "research",
+    "research_deep",
+    "Run deep research and generate a cited PDF report.",
+    ARTIFACT_TOOL,
+  ),
+  tool(
+    "research",
+    "research_fanout",
+    "Run deep research fan-out and generate a cited PDF report.",
+    ARTIFACT_TOOL,
+  ),
   tool("research", "search_company", "Search company intel with Exa.", REMOTE_TOOL),
   tool("research", "search_web", "Search the web with Exa.", REMOTE_TOOL),
   tool("research", "search_web_advanced", "Search the web with Exa filters.", REMOTE_TOOL),
diff --git a/skills/deep-research/SKILL.md b/skills/deep-research/SKILL.md
@@ -9,22 +9,21 @@ compatibility: Requires Exa and Firecrawl research tools; fan-out mode requires
 
 # Deep Research
 
-Answer complex questions with sourced synthesis. The output should read like an analyst brief: clear thesis, cited evidence, disagreement handling, and confidence notes. When the question spans many entities or angles, fan out parallel probes first (see Fan-out Mode), then synthesize.
+Answer complex questions with sourced synthesis. The output should read like an analyst brief: clear thesis, cited evidence, disagreement handling, and confidence notes. Every run produces a complete PDF deliverable in the user's project in addition to a concise chat summary. When the question spans many entities or angles, use Fan-out Mode.
 
 ## Quick Start
 
-1. Rewrite the user's ask into 3-6 research questions.
-2. Create a source matrix with question, claim, evidence, URL, date, and confidence columns.
-3. Search with `search_web_advanced`; use date/domain filters when appropriate.
-4. Scrape authoritative sources with `search_scrape`.
-5. Synthesize with inline citations and explicit uncertainty.
+1. Scope the user's ask, timeframe, geography, and decision use.
+2. Run the request through `research_deep`; it performs the parallel research, validates citations, and creates the PDF deliverable.
+3. Return the key conclusion and important caveats in chat, with a short source list.
+4. Refer to the PDF naturally as ready below. Do not call a separate document tool or recreate the report.
 
 ## Fan-out Mode
 
-Use breadth first when the ask covers many independent entities (companies, tools, policies, markets): survey a population, compare many companies, or scan a market across 10-25 angles. Run it through the `research_fanout` workflow tool.
+Use breadth first when the ask covers many independent entities (companies, tools, policies, markets): survey a population, compare many companies, or scan a market across many angles. Run it through the `research_fanout` workflow tool; it creates the cited comparison PDF automatically.
 
 1. Identify the population to cover and the comparison criteria.
-2. Define up to 25 independent probe slots with clear per-probe questions and source expectations.
+2. Define up to 12 independent probe slots with clear per-probe questions and source expectations.
 3. Fan out the probes; keep them independent so slow or failed branches do not block the answer.
 4. Use the same fields for every entity so the comparison is fair; track source URLs per cell.
 5. Deduplicate facts, aliases, and repeated articles before synthesis.
@@ -55,8 +54,9 @@ Use breadth first when the ask covers many independent entities (companies, tool
 
 ## Deliverables
 
-- Markdown report
-- Source matrix
+- PDF research report saved in the project and shown in Deliverables
+- Concise chat summary with the main conclusion and caveats
+- Claim-to-source evidence map and source list inside the PDF
 - Confidence and gaps
 
 ## References
