feat: add Streamable HTTP transport support for remote MCP providers

[nabinchha]

📋 Summary

Adds streamable_http as a supported transport type for MCPProvider, resolving #357. Previously, remote MCP providers only supported SSE transport, causing silent 5-minute timeouts when connecting to Streamable HTTP-only endpoints (e.g. Tavily). Also adds DataDesigner.list_mcp_tool_names() for discovering available tools on a configured provider, and updates documentation across the board.

🔄 Changes

✨ Added

• streamable_http transport option on MCPProvider.provider_type (default remains "sse" for backwards compatibility)
• DataDesigner.list_mcp_tool_names() — convenience method to discover tools exposed by a configured MCP provider
• list_tool_names() helper in mcp.io module
• stub_streamable_http_provider test fixture
• Tests for streamable_http config validation, session creation, and CLI form builder

🔧 Changed

• MCPIOService._get_or_create_session() — routes streamable_http providers through streamablehttp_client, handles variable-length context manager results (3-tuple from streamablehttp_client vs 2-tuple from sse_client)
• MCPProviderFormBuilder — renamed _run_sse_form → _run_remote_form to handle both SSE and Streamable HTTP with appropriate labels
• MCPProviderController._select_provider — shows "Streamable HTTP" vs "SSE" in provider display

📚 Docs

• mcp-providers.md — updated overview table, fields table, Python examples, and YAML examples for both transports
• configure-mcp-cli.md — added Streamable HTTP to provider type selection and updated endpoint examples
• code_reference/mcp.md — updated MCPProvider description to mention both transports

🔍 Attention Areas

⚠️ Reviewers: Please pay special attention to the following:

• io.py#L222-L223 — Context manager result unpacking changed from read, write = ... to ctx_result[0], ctx_result[1] to handle both 2-tuple (SSE) and 3-tuple (Streamable HTTP) returns from the MCP SDK
• mcp.py#L47 — provider_type expanded from Literal["sse"] to Literal["sse", "streamable_http"] — this changes the Pydantic discriminator union; verify deserialization of existing configs still works

---

🤖 Generated with AI

[greptile-apps]

Greptile Summary

This PR successfully adds Streamable HTTP transport support for remote MCP providers, resolving the 5-minute timeout issue when connecting to Streamable HTTP-only endpoints. The implementation properly handles the different tuple return types from the MCP SDK (2-tuple for SSE/stdio vs 3-tuple for streamable_http) through indexed unpacking.

Key Changes:

• Expanded MCPProvider.provider_type from Literal["sse"] to Literal["sse", "streamable_http"] with backward-compatible default of "sse"
• Updated _get_or_create_session() to route streamable_http providers through streamablehttp_client and handle variable-length context manager returns
• Refactored CLI form builder from _run_sse_form to _run_remote_form with proper Literal type annotations
• Added DataDesigner.list_mcp_tool_names() convenience method for tool discovery
• Comprehensive test coverage for the new transport type
• Thorough documentation updates across all relevant guides

Previous Review Comments Addressed:

• ✅ provider_type parameter now uses Literal["sse", "streamable_http"] instead of str
• ✅ Test mock correctly returns 3-tuple matching MCP SDK behavior for streamable_http

The implementation is clean, well-tested, and maintains backward compatibility.

Confidence Score: 5/5

• This PR is safe to merge with high confidence
• The implementation is well-architected, properly tested, and maintains backward compatibility. The critical tuple unpacking logic correctly handles both 2-tuple and 3-tuple returns from different MCP SDK clients. All previous review comments have been addressed. Type safety is enforced through Literal types, and comprehensive tests validate the new functionality. Documentation is thorough and complete.
• No files require special attention

Important Files Changed

Filename	Overview
packages/data-designer-config/src/data_designer/config/mcp.py	Adds streamable_http to MCPProvider.provider_type Literal type with proper defaults and documentation
packages/data-designer-engine/src/data_designer/engine/mcp/io.py	Adds streamable_http transport support with proper branching logic and tuple unpacking to handle both 2-tuple (SSE/stdio) and 3-tuple (streamable_http) context manager returns
packages/data-designer/src/data_designer/cli/forms/mcp_provider_builder.py	Refactored _run_sse_form to _run_remote_form with proper type annotations and support for both SSE and streamable_http providers
packages/data-designer/src/data_designer/interface/data_designer.py	Adds list_mcp_tool_names() convenience method with proper error handling for unknown providers
packages/data-designer-engine/tests/engine/mcp/test_mcp_io.py	Adds test for streamable_http session creation with mock that properly returns 3-tuple as per MCP SDK

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    Start[_get_or_create_session] --> CheckSession{Session exists<br/>in pool?}
    CheckSession -->|Yes| ReturnSession[Return cached session]
    CheckSession -->|No| CheckProvider{Provider type?}
    
    CheckProvider -->|LocalStdioMCPProvider| StdioClient[Create stdio_client<br/>with command/args/env]
    CheckProvider -->|MCPProvider +<br/>streamable_http| StreamableClient[Create streamablehttp_client<br/>with endpoint/headers]
    CheckProvider -->|MCPProvider +<br/>sse default| SSEClient[Create sse_client<br/>with endpoint/headers]
    
    StdioClient --> ContextEnter[ctx.__aenter__]
    StreamableClient --> ContextEnter
    SSEClient --> ContextEnter
    
    ContextEnter --> UnpackTuple["Unpack tuple:<br/>read = ctx_result[0]<br/>write = ctx_result[1]"]
    
    UnpackTuple --> CreateSession[Create ClientSession<br/>with read, write]
    CreateSession --> InitSession[Initialize session]
    InitSession --> CacheSession[Store in session pool]
    CacheSession --> ReturnNewSession[Return new session]

Loading

_{Last reviewed commit: f0b0e6a}

[greptile-apps]

_{9 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Additional Comments (1)

packages/data-designer/src/data_designer/cli/forms/mcp_provider_builder.py
Update docstring to mention Streamable HTTP support

    """Builds interactive forms for MCP provider configuration.

    Supports both MCPProvider (remote SSE and Streamable HTTP) and LocalStdioMCPProvider (subprocess).
    """

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/data-designer/src/data_designer/cli/forms/mcp_provider_builder.py
Line: 22-24

Comment:
Update docstring to mention Streamable HTTP support

```suggestion
    """Builds interactive forms for MCP provider configuration.

    Supports both MCPProvider (remote SSE and Streamable HTTP) and LocalStdioMCPProvider (subprocess).
    """
```

<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>

How can I resolve this? If you propose a fix, please make it concise.

[eric-tramel]

LGTM. Food for thought -- an Option C is to use a auto-detect/fallback on health check. This would mean that, unless it was specifically set by the user, we could "just" find which one of these two modes works and then run with it accordingly.

[eric-tramel]

Nit: Do we opt for StrEnum?