Skip to content

feat(langgraph): Add tool support to LangGraph templates#130

Open
andreiborza wants to merge 6 commits intomainfrom
ab/add-langgraph-tool-calls
Open

feat(langgraph): Add tool support to LangGraph templates#130
andreiborza wants to merge 6 commits intomainfrom
ab/add-langgraph-tool-calls

Conversation

@andreiborza
Copy link
Copy Markdown
Member

This PR adds tool calls to LangGraph templates. It creates tools using @langchain/core/tools with zod schemas.

This PR adds tool calls to LangGraph templates. It creates tools using
`@langchain/core/tools` with zod schemas.
@constantinius
Copy link
Copy Markdown
Collaborator

Looks good!

This reminded me that createReactAgent is deprecated. Not sure what the replacement is. We should tackle that as well. This PR could the be the opportunity for doing that.

@andreiborza
Copy link
Copy Markdown
Member Author

Yep, the alternative is https://docs.langchain.com/oss/javascript/releases/langchain-v1#createagent, we'll need to instrument that too at some point.

Copy link
Copy Markdown
Member

@nicohrubec nicohrubec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 2, 2026

🔴 AI SDK Integration Test Results

Status: 2 regressions detected

Summary

Metric main PR Change
Total Tests 667 667
Passed 386 388 +2 ✅
Failed 163 163

🔴 Regressions

These tests were passing on main but are now failing:

browser/openai :: Multi-Turn LLM Test (blocking)

Error: Browser test timed out (60s)

Browser test timed out (60s)
node/google-genai :: Conversation ID LLM Test (blocking)

Error: 2 check(s) failed:

2 check(s) failed:
Attribute validation failed:
  Span 50c37a2f: Attribute 'gen_ai.usage.input_tokens' must exist but is missing
  Span 50c37a2f: Attribute 'gen_ai.usage.output_tokens' must exist but is missing
Token usage validation failed:
  input_tokens must exist
  output_tokens must exist
  total_tokens must exist
gen_ai.response.model is missing (optional but recommended)

✅ Fixed

These tests were failing on main but are now passing:

  • cloudflare/vercel :: Conversation ID Agent Test (blocking, function, openai)
  • browser/google-genai :: Basic LLM Test (blocking)
  • cloudflare/google-genai :: Multi-Turn LLM Test (blocking)
  • cloudflare/langchain :: Multi-Turn LLM Test (streaming)

Test Matrix

Agent Tests

SDK Basic Agent Test Conversation ID Agent Test Long Input Agent Test Tool Call Agent Test Tool Error Agent Test Vision Agent Test
browser/langgraph blk, combinedblk, compiledblk, custom-stateblk, graphblk, langchainstr, combinedstr, compiledstr, custom-statestr, graphstr, langchain blk, combinedblk, compiledblk, custom-stateblk, graphblk, langchainstr, combinedstr, compiledstr, custom-statestr, graphstr, langchain blk, combinedblk, compiledblk, custom-stateblk, graphblk, langchainstr, combinedstr, compiledstr, custom-statestr, graphstr, langchain blk, combinedblk, compiledblk, custom-stateblk, graphblk, langchainstr, combinedstr, compiledstr, custom-statestr, graphstr, langchain blk, combinedblk, compiledblk, custom-stateblk, graphblk, langchainstr, combinedstr, compiledstr, custom-statestr, graphstr, langchain blk, combinedblk, compiledblk, custom-stateblk, graphblk, langchainstr, combinedstr, compiledstr, custom-statestr, graphstr, langchain
cloudflare/langgraph
cloudflare/vercel blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropic ✅🔧blk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai
nextjs/mastra
nextjs/vercel blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai
node/langgraph
node/manual
node/mastra
node/vercel blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai blk, class, anthropicblk, class, openaiblk, function, anthropicblk, function, openaistr, class, anthropicstr, class, openaistr, function, anthropicstr, function, openai
python/langgraph as as as as as as
python/manual as as as as as as
python/openai-agents
python/pydantic-ai a, fallbacka, single a, fallbacka, single a, fallbacka, single a, fallbacka, single a, fallbacka, single a, fallbacka, single

Embedding Tests

SDK Basic Embeddings Test
browser/google-genai
browser/langchain
browser/openai
cloudflare/google-genai
cloudflare/langchain
cloudflare/openai
cloudflare/vercel
nextjs/google-genai
nextjs/langchain
nextjs/openai
nextjs/vercel
node/google-genai
node/langchain
node/openai
node/vercel
python/google-genai a, blks, blk
python/langchain a, blks, blk
python/litellm a, blks, blk
python/manual a, blks, blk
python/openai a, blks, blk

LLM Tests

SDK Basic Error LLM Test Basic LLM Test Conversation ID LLM Test Long Input LLM Test Multi-Turn LLM Test Vision LLM Test
browser/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
browser/google-genai blkstr ✅🔧blkstr blkstr blkstr blkstr blkstr
browser/langchain blkstr blkstr blkstr blkstr blkstr blkstr
browser/openai blkstr blkstr blkstr blkstr ❌📉blkstr blkstr
cloudflare/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
cloudflare/google-genai blkstr blkstr blkstr blkstr ✅🔧blkstr blkstr
cloudflare/langchain blkstr blkstr blkstr blkstr blk ✅🔧str blkstr
cloudflare/openai blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/google-genai blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/langchain blkstr blkstr blkstr blkstr blkstr blkstr
nextjs/openai blkstr blkstr blkstr blkstr blkstr blkstr
node/anthropic blkstr blkstr blkstr blkstr blkstr blkstr
node/google-genai blkstr blkstr ❌📉blkstr blkstr blkstr blkstr
node/langchain blkstr blkstr blkstr blkstr blkstr blkstr
node/manual
node/openai blkstr blkstr blkstr blkstr blkstr blkstr
python/anthropic a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str
python/google-genai a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str
python/langchain a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str
python/litellm a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str
python/manual a, blks, blk a, blks, blk a, blks, blk a, blks, blk a, blks, blk
python/openai a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str a, blka, strs, blks, str

MCP Tests

SDK Basic MCP Tool Call Test MCP Multiple Tool Calls Test MCP Prompt Get Test MCP Resource Read Test MCP Tool Error Test
node/mcp sseio sseio sseio sseio sseio
python/fastmcp a, blk, ssea, blk, io a, blk, ssea, blk, io a, blk, ssea, blk, io a, blk, ssea, blk, io a, blk, ssea, blk, io
python/mcp a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo a, blk, sse, hia, blk, sse, loa, blk, io, hia, blk, io, lo

Legend: ✅ Pass | ❌ Fail | ✅🔧 Fixed | ❌📉 Regressed | ✅🆕 New (pass) | ❌🆕 New (fail) | 🗑️ Removed | str=streaming blk=blocking a=async s=sync io=stdio sse=sse hi=highlevel lo=lowlevel


Generated by AI SDK Integration Tests

Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

@andreiborza andreiborza force-pushed the ab/add-langgraph-tool-calls branch from d84fd14 to 390aa97 Compare April 2, 2026 09:59
let actualArgs: Record<string, unknown>;
const argsRaw =
foundCall.arguments ||
foundCall.args ||
Copy link
Copy Markdown
Collaborator

@constantinius constantinius Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we should do that. This comes from the gen_ai.output.messages (or the old tool_call one) which includes the tool calls as ToolCallRequestPart, which has "arguments", not "args".

Here's the relevant schema:

        "ToolCallRequestPart": {
            "additionalProperties": true,
            "description": "Represents a tool call requested by the model.",
            "properties": {
                "type": {
                    "const": "tool_call",
                    "description": "The type of the content captured in this part.",
                    "title": "Type",
                    "type": "string"
                },
                "id": {
                    "anyOf": [
                        {
                            "type": "string"
                        },
                        {
                            "type": "null"
                        }
                    ],
                    "default": null,
                    "description": "Unique identifier for the tool call.",
                    "title": "Id"
                },
                "name": {
                    "description": "Name of the tool.",
                    "title": "Name",
                    "type": "string"
                },
                "arguments": {
                    "default": null,
                    "description": "Arguments for the tool call.",
                    "title": "Arguments"
                }
            },

I would opt to fix that in the integration, tbh.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, ok. But doesn't foundCall.function also go against this schema? As far as I can tell that is also there for OpenAI's format.

Not saying we should add yet another exception, I can remap it in the SDK but remapping logic becomes brittle quick.

Let me know what you think @constantinius.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went ahead and reverted in 8976eee and will normalize on the SDK side for the time being.

Comment on lines +44 to +49
name: "{{ toolDef.name }}",
description: "{{ toolDef.description }}",
schema: z.object({
{% if toolDef.parameters and toolDef.parameters.properties %}
{% for paramName, paramDef in toolDef.parameters.properties %}
{{ paramName }}: {{ zodType(paramDef.type) }}{% if paramDef.description %}.describe("{{ paramDef.description }}"){% endif %},
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Tool definition properties like name and description are rendered into a JavaScript template without escaping, which can lead to syntax errors if they contain special characters.
Severity: HIGH

Suggested Fix

Apply an appropriate escaping mechanism to the toolDef.name, toolDef.description, and toolDef.error variables within the Nunjucks template. Using a filter that JSON-encodes the string, similar to the | dump filter used for toolDef.result, would ensure the output is a valid JavaScript string literal.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: src/runner/templates/agents/node/langgraph/template.njk#L44-L49

Potential issue: The Nunjucks template at
`src/runner/templates/agents/node/langgraph/template.njk` directly interpolates tool
definition properties like `toolDef.name`, `toolDef.description`, and `toolDef.error`
into JavaScript string literals. Since Nunjucks is configured with `autoescape: false`
and no specific JavaScript escaping filter is applied, any special characters (e.g.,
double quotes, backslashes, newlines) within these properties will break the syntax of
the generated JavaScript. This will cause the generated test script to fail with a
`SyntaxError` at runtime when a tool's metadata contains such characters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants