feat(mcp): add lexical capability ranking

[reachjalil]

Draft Proposal

This proposes replacing Den /mcp/agent capability search's per-source additive keyword scoring with a shared lexical ranker. The goal is to improve capability discoverability without adding embeddings, vectors, LLM calls, or npm dependencies.

The search/execute surface stays the same:

Surface	Status
/mcp/agent tools	Still exactly search_capabilities and execute_capability.
/mcp rich endpoint	Keeps searchCapabilities(catalog, query, limit) signature.
Execute paths	Unchanged for REST, External MCP, and marketplace capabilities.
Permissions/credentials	Unchanged; External MCP grants and connection states still gate visibility/execution.

Why This Change

Problem	Proposal
REST, External MCP, and marketplace capabilities scored with separate incomparable formulas.	Sources now emit candidates; /mcp/agent ranks the merged corpus once.
Queries like store a note miss the intended Memory capability because wording differs from operation names.	Add conservative normalization and a small curated synonym map.
Partial/noisy hits could rank alongside actionable setup rows.	Add coverage scoring and a noise floor while keeping needs_connection rows floor-exempt.
Results were hard to interpret.	Add source on matches and document 0-100 per-response score semantics.

What Changed

Area	Change
Shared ranker	Adds ee/apps/den-api/src/mcp/ranking.ts with tokenization, stemming, stopwords, synonyms, rarity, coverage, adjacency, noise floor, and suggestions.
REST catalog	Reworks search.ts into buildRestCandidates() plus the existing searchCapabilities() wrapper.
External MCP	Converts External MCP search into listExternalCapabilityCandidates(), preserving grants, credential checks, best-effort connection handling, and setup/error rows.
Marketplace	Converts marketplace search into listMarketplaceCapabilityCandidates() and ranks it with the same shared scorer.
Agent route	/mcp/agent now gathers REST + External MCP + marketplace candidates, ranks once, short-circuits all-stopword queries, and returns suggestions on zero-result responses.
Tests	Adds pure ranker tests and real-catalog tests; updates one External MCP assertion to expect noisy summary-only hits to be suppressed.

Benchmarking

Benchmark methodology:

Benchmark	How measured
Quality	Compared old additive scorer vs this branch on checked-in packages/docs/openapi.json plus the synthetic Memory/external corpus used by the new unit tests.
Pass rule	top1 or top3 depending on the query's intended ambiguity.
Latency	Ran 35,000 searchCapabilities() calls locally over the checked-in catalog.

Quality Summary

Corpus	Old scorer	New ranker	Delta
Real checked-in catalog	4/7	5/7	+1
Synthetic vocabulary/external cases	6/6	6/6	0
Combined	10/13, 77%	11/13, 85%	+8 percentage points

Notable Query-Level Changes

Query	Expected	Old rank	New rank	Notes
invite a teammate	postInvitations	not top 5	2	Synonym bridge from teammate/person/member plus invitation terms improves discoverability.
store a note	postMemory	2	1	Synonym bridge from store/save and note/memory fixes the vocabulary mismatch case.
list workers	getWorkers	1	1	Existing strong match preserved.
create plugin	postPlugins	1	1	Existing strong match preserved.
linear	mcp:linear:*	1	1	needs_connection row remains discoverable and actionable.
notion page	mcp:notion:createPage	1	1	External MCP direct match preserved.

Latency

Catalog size	Calls	Total time	Average per search
127 capabilities	35,000	21,367.89 ms	0.6105 ms

This is still comfortably sub-millisecond per REST catalog search on this machine. External MCP network calls are unchanged; this only changes how returned candidates are ranked.

Important Scope Notes

Scope	Decision
Skills	This branch is based on upstream/dev, which does not yet include the separate skill-capabilities.ts branch. The ranker type includes skills, so that source can plug into the same candidate interface after the skill PR lands.
Embeddings/vectors/LLMs	Not included. This is intentionally pure lexical ranking.
New dependencies	None.
Execute behavior	Not changed.
Auth/credential behavior	Not changed. Credential ownership and External MCP access checks remain in their existing modules.
Fraimz	Not run in this pass; this is backend ranking logic with focused unit/regression coverage.

Validation

Command	Result
pnpm --filter @openwork-ee/den-api exec bun test test/mcp-capability-ranking.test.ts test/mcp-capability-search.test.ts test/external-capabilities-search-divergence.test.ts test/marketplace-capabilities.test.ts test/mcp-agent-config-policy.test.ts test/mcp-tool-name-length.test.ts test/mcp-invoke-body.test.ts	Passed, 48 tests
pnpm --filter @openwork-ee/den-api exec tsc --noEmit	Passed
pnpm --filter @openwork-ee/den-api build	Passed

DB-backed tests required local MySQL databases openwork_test_pr3 and openwork_test_searchdiv; I created them locally and ran pnpm --filter @openwork-ee/den-db db:push for each before the test sweep.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
openwork-landing	Ready	Preview, Comment, Open in v0	Jul 8, 2026 5:33am

[vercel]

@reachjalil is attempting to deploy a commit to the Different AI Team on Vercel.

A member of the Team first needs to authorize it.