Add pluggable tool-search for large MCP toolsets

[krish-jain]

The MCP aggregator returns every tool from every connected MCP server, dot-namespaced as server.tool_name. That full list flows into each provider's _prepare_tools() and is sent on every request. For a multi-server setup this can consume tens of thousands of tokens in tool definitions before the model and degrade tool-selection accuracy past 30–50 tools.

Anthropic shipped the tool search tool with defer_loading: true on individual tools https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search. OpenAI's Responses API has similar patterns, and fast-agent should grow a single, abstraction for this and bridge to each provider's mechanism where one exists.

[JassonG-xt]

I spent some time tracing how tools actually reach the wire before commenting, so here's what I
found and a proposed shape. Line references are against 64eec1b6 (v0.10.4).

There is no shared seam to hook into

The issue mentions "each provider's _prepare_tools()". That method turns out to be
Anthropic-only; each provider assembles its tool payload through its own code path:

Provider	Entry point
Anthropic	_prepare_tools(), llm/provider/anthropic/llm_anthropic.py:1123
OpenAI Responses	_tools_payload(), llm/provider/openai/responses.py:844
Google	_google_available_tools(), llm/provider/google/llm_google_native.py:803

There is also no cross-provider capability protocol to hang this off. supports_web_tools()
exists only on the Anthropic classes (llm_anthropic.py:771, overridden in
llm_anthropic_vertex.py:80); FastAgentLLM (llm/fastagent_llm.py:110), its protocol, and
interfaces.py declare no capability methods at all. So "a single abstraction that bridges to
each provider's mechanism" means introducing the seam rather than plugging into one.

Current state, for the record. OpenAI's server-side tool_search (responses.py:93) is already
wired, but it only triggers for provider-managed MCP attachments carrying defer_loading
(responses.py:866), not for tools coming through MCPAggregator. On Anthropic there is
nothing yet:

$ grep -ri 'tool_search\|defer_loading' src/fast_agent/llm/provider/anthropic/
$

The display vocabulary, on the other hand, already exists: the remote_tool_search family,
the "Deferred tool search" label, and the status text in tool_activity_presentation.py:33,
plus streaming and replay handling in the Responses pipeline. The socket is there; the plug
isn't.

The constraint I think should drive the design

llm_anthropic.py:1263 applies cache_control to the system prompt, and the log line notes it
"caches tools+system in one block".

That makes the obvious approach, re-selecting a task-relevant tool subset each turn, actively
harmful. Any change to the tool list invalidates the tools+system prefix, and rebuilding that
cache can cost more than the tool schemas you saved. It may also be why both vendors landed on
"send everything, mark it deferred" instead of "send a subset".

So whatever we build has to keep a stable resident set across turns within a session and
push only the tail behind a lookup. Deferral selection needs to be deterministic per session,
or it quietly loses money.

Proposed shape

Introduce the capability seam on the LLM base class, following the supports_web_tools()
precedent, and dispatch to one of two backends before each provider serialises its payload:

FastAgentLLM
  supports_deferred_tools() -> bool        # default False
  tool_deferral_backend()   -> Backend | None

policy.split(tools, ctx) -> (resident, deferred)   # pure, deterministic per session
  |- NativeBackend : mark deferred tools, let the provider's own tool search serve them
  |                  (Anthropic tool search; OpenAI reuses the existing responses.py path)
  |- LocalBackend  : drop deferred tools from the payload, inject one `search_tools` meta-tool
                     backed by a dependency-free lexical index over name/description/params

McpAgent.list_tools() keeps its current signature and semantics. The split happens below it, at
the provider boundary, because that is the only layer that knows both what the model supports and
where the cache breakpoint falls.

LocalBackend is what keeps this useful when the provider in front of you has no native
mechanism to bridge to. I'd keep it lexical rather than embedding-based: tool descriptions
already share vocabulary with user requests, and pulling in a model download to route tool calls
seems like a bad trade. If semantic matching later proves necessary, the privacy module
establishes the pattern for an optional extra (find_spec probing in privacy/dependencies.py,
pinned repo_id/revision in privacy/model_resolver.py).

Relationship to #589

@iqdoctor's discover / learn / execute proposal covers much of this ground. I'd rather treat
LocalBackend as one implementation of that Phase 2 than as a competing design; the meta-tool
surface is the same idea, scoped to the aggregator's own tools instead of a standalone proxy.
Happy to align naming with #589 if that's preferred.

Suggested sequencing

1. Capability seam, policy, and measurement only, with no behaviour change. Report
   tool-schema token cost per request; commands/status_summaries.py:312 already shows the
   tiktoken pattern to follow. Without a baseline number none of the later phases can be argued
   for.
2. LocalBackend, which benefits every provider.
3. NativeBackend, Anthropic first, since OpenAI mostly reuses what's already there.

Questions before I write any code

1. Is adding capability methods to the LLM base class the shape you want, or would you rather
   keep this provider-local and accept the duplication?
2. Should the resident/deferred split be configurable per agent (in the spirit of tools: in
   fastagent.config.yaml), or inferred purely from tool count and usage?
3. Is a dependency-free lexical index acceptable in core, or should even that sit behind an
   extra?

Happy to start with step 1 alone if that's the useful piece.