Skip to content

feat: add experimental mcp destination type for MCP Events webhooks - #1181

Open
alexluong wants to merge 12 commits into
mainfrom
feat/mcp-destination
Open

alexluong wants to merge 12 commits into
mainfrom
feat/mcp-destination

Conversation

@alexluong

@alexluong alexluong commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Add an mcp destination type that delivers events to MCP Events (draft SEP-3415) webhook subscriptions. Experimental and off by default (EXPERIMENTAL_MCP_ENABLED=false); its config keys carry the EXPERIMENTAL_ prefix (experimental.mcp in YAML).

Stacked on #1173. Until it merges, the diff here includes its changes; this PR is the MCP commits on top of it.

In MCP Events, an agent subscribes through an MCP server to one event type and gives a callback URL; the server then POSTs signed events there until the subscription expires. Here each subscription is one Outpost destination of type mcp, created and refreshed by the operator's MCP server with the Admin API key. Outpost covers the webhook mode only: no replay, cursor is always null.

Changes

  1. Signing and envelope. Each event is wrapped in the MCP Events envelope (eventId, name, timestamp, data, cursor, at most 256 KiB) and signed with Standard Webhooks headers plus X-MCP-Subscription-Id. webhook-id is msg_ plus 22 base64url characters (128 random bits), minted when the event is matched to the subscription and kept across automatic retries; it is never derived from eventId, as the draft requires. A manual retry gets a new one. Timestamp and signature are computed again on every attempt. Rotating credentials.secret keeps the old secret signing alongside the new one for 24 hours. Receivers should dedupe retries on webhook-id; a receiver that dedupes by eventId must scope it to the subscription (X-MCP-Subscription-Id), because one event is delivered once per matching subscription.
  2. Callback verification. Before a destination is stored, and when an update changes its config or credentials after the cached check has expired, Outpost sends a signed challenge to the callback URL and the endpoint must answer 2xx with the nonce echoed. A passed check is cached per tenant, principal and URL (EXPERIMENTAL_MCP_VERIFICATION_TTL_SECONDS) and cleared when the tenant is deleted. Challenges are rate limited per tenant and principal, and failed challenges per tenant and callback host.
  3. Callback rules. The URL must be https in normalized form, as the draft requires. Connections go through the address guard from feat: block private addresses for webhook destinations #1176 and no redirect is followed. For development only, EXPERIMENTAL_MCP_ALLOW_INSECURE_CALLBACKS also accepts http when every address the host resolves to is in DESTINATIONS_PRIVATE_ADDRESS_ALLOWLIST.
  4. Own retry schedule. mcp retries on EXPERIMENTAL_MCP_RETRY_SCHEDULE (default 30 s, 2 min, 10 min) through the retry hook from feat: let destination types classify failures and set their own retry policy #1174. The delivery task carries the message ID, so retries reuse it.
  5. API shape. mcp destinations are created and updated through the destinations API with the Admin API key only; feat: add MCP subscribe and unsubscribe endpoints #1183 later limits creating and updating them to the subscribe route. One topic, no *. config.url, config.principal, config.arguments and topics are immutable. expires_at is accepted and returned for mcp destinations, using the expiry from feat: add destination expiry #1173. Each subscription counts toward MAX_DESTINATIONS_PER_TENANT; at the limit, creating one first removes the tenant's expired ones. Create checks for a duplicate ID and a full tenant before sending the challenge; the store still refuses both atomically (fix: make concurrent tenant and destination writes safe #1172). The mcp type is left out of destination-type metadata for tenant JWT callers, so the portal shows mcp destinations as a generic entry.
  6. Verification errors. Callback and address failures return 422 on config.url with a category (connection_refused, timeout, tls_error, challenge_failed, https_required, address_not_allowed), never the endpoint's response. A verification rate limit returns 429 (rate_limited) with Retry-After. An unavailable verification store returns 503 (verification_unavailable). All three apply to create and to updates that re-verify.
  7. OpenAPI and SDKs. mcp is in the OpenAPI spec as an experimental destination type, so the SDKs model it; they pick it up in the usual SDK release PRs. EXPERIMENTAL_MCP_ENABLED and EXPERIMENTAL_MCP_RETRY_SCHEDULE are listed in the ManagedConfig schema.

Deviations from the draft

The one list for this chain; #1182, #1183 and the later PRs point here.

  • EXPERIMENTAL_MCP_ALLOW_INSECURE_CALLBACKS is an http exception the draft doesn't have (it requires https). Development only, documented as outside the draft.
  • Event data isn't checked against a payloadSchema; the draft says data conforms to the schema. Event definitions come in feat: add MCP event definitions in the events/list format and a catalog endpoint #1182; validating at publish is a follow-up.
  • Failed challenges are limited per tenant and callback host, not per host across tenants.
  • Auto-disable suspends a failing destination and drops its events while suspended. Refreshing the subscription (feat: add MCP subscribe and unsubscribe endpoints #1183) re-enables it, but the draft asks to resume the dropped events.
  • Draft SHOULDs not implemented: terminating subscriptions when an event type is removed or changed incompatibly; re-checking a principal's permission at delivery time (a subscription lives at most EXPERIMENTAL_MCP_SUBSCRIPTION_TTL_MAX_SECONDS); terminated and gap envelopes; delivery status reporting.
  • Replay: cursor, maxAgeMs and truncated are accepted and ignored.

Behavior changes

For destination types other than mcp:

  • Creating or updating a destination with a non-null expires_at returns 422 (expires_at is not supported for this destination type); null is accepted.
  • A stored destination whose type isn't registered is listed with its credentials removed, and the read doesn't fail. This applies to any type.
  • A destination whose type isn't registered gets one failed, non-retryable attempt per event; it isn't redelivered from the queue and doesn't count toward auto-disable.

For mcp destinations:

  • mcp always blocks private addresses, whatever DESTINATIONS_BLOCK_PRIVATE_ADDRESSES is set to; DESTINATIONS_PRIVATE_ADDRESS_ALLOWLIST applies to it too.
  • EXPERIMENTAL_MCP_RETRY_SCHEDULE is separate from RETRY_SCHEDULE, RETRY_INTERVAL_SECONDS and MAX_RETRY_LIMIT, which don't apply to mcp.
  • A 410 or 413 response, and an event over 256 KiB (attempt code payload_too_large, nothing sent), is not retried and does not count toward auto-disable. Webhooks still retry and count them.
  • With EXPERIMENTAL_MCP_ENABLED=false, mcp is rejected like any unknown type. Existing mcp destinations stay listed (credentials hidden), match no events and receive no deliveries or attempts; they lapse at their expiry as usual.
  • While MCP Events is off, POST /retry for a stored mcp destination answers 400 destination_type_disabled, and a publish with that destination_id matches nothing (202, empty destination_ids), the same as for a disabled destination.

Smaller: a destination type can opt out of the publisher cache (PublisherCacheBypasser), since mcp publishers hold no resources and one cache entry per subscription would evict costly ones.

Notes

  • Open question: whether mcp destinations should be returned by the destinations API at all, or stay internal to the subscribe endpoints. This affects the SDK Destination union and whether subscriptions count toward MAX_DESTINATIONS_PER_TENANT.
  • Clients built against an earlier draft may expect webhook-id to equal eventId; with this PR they don't match.
  • DESTINATIONS_PROXY_URL, when set, carries mcp deliveries and verification challenges (through the whole proxy chain, as for webhooks); connections to the proxy aren't address-checked, so the proxy must refuse private addresses. Unset, connections are direct and the guard checks each dialed address. HTTP_PROXY/HTTPS_PROXY are never used.

Not in this PR

🤖 Generated with Claude Code

@alexbouchardd

Copy link
Copy Markdown
Contributor

Own retry schedule. mcp retries on EXPERIMENTAL_MCP_RETRY_SCHEDULE (default 30 s, 2 min, 10 min) through the retry hook from #1174. The delivery task carries the message ID, so retries reuse it.

Why would we want to have this supported as a config?

I also wonder how this might interact with the max destination limit, I would imagine this could create a lot of destinations. Have we considered this?

@alexluong

Copy link
Copy Markdown
Collaborator Author

Why would we want to have this supported as a config?

I think allow the retry schedule to be configurable is helpful in general. Outpost OSS itself is less opinionated. We can avoid exposing that for cloud unless explicitly requested for simplicity sake?

I also wonder how this might interact with the max destination limit, I would imagine this could create a lot of destinations. Have we considered this?

Yes, this is the main thing I tried to highlight in our internal Notion page. There's a decision tree we need to decide regarding how we perceive mcp destination, whether it's a real destination that counts for limit, shows up on dashboard, API calls, or whether it's treated differently and we filter it out. I personally think we should ignore it for the time being. It would be less visible but make things a bit simpler from the general UX POV. It's not implemented that way tho.

@alexluong
alexluong force-pushed the feat/mcp-destination branch from 72669c9 to 0b7d1a2 Compare November 4, 2026 15:53
alexluong and others added 12 commits November 4, 2026 22:57
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tries

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alexbouchardd

alexbouchardd commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

I think allow the retry schedule to be configurable is helpful in general. Outpost OSS itself is less opinionated. We can avoid exposing that for cloud unless explicitly requested for simplicity sake?

Seems odd to me that MCP events would have a different retry schedule unless it's spec-mandated, in which case the different schedule should probably be hardcoded

@alexluong
alexluong force-pushed the feat/mcp-destination branch from 0b7d1a2 to 72cd77a Compare November 4, 2026 16:05

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants