[rdimitrov]

Summary

Anonymous usage metrics today report one number per proxy per day: the tool-call count. That cannot answer the product questions we keep asking: which transports and runtimes people use, how much of the built-in registry is actually run, which MCP clients drive traffic, and whether anyone runs vMCP at all (vMCP sends nothing today). This PR adds a small, fixed-vocabulary profile to each report so those questions become answerable, while keeping cardinality low and never sending names, URLs or images of private servers.

• Proxy profile. Each /api/v1/toolcount report now carries transport, server_kind (container/remote), registry_source (builtin, custom_remote, custom_api, custom_local, none), catalog_server, runtime, auth_mode and authz. Every value is checked against an allowed list; anything else becomes other.
• Built-in catalog name only. catalog_server is the entry name only when the server came from the registry embedded in thv; every other server reports custom. The check lives in cmd/thv, so thv-proxyrunner does not embed the 2.8 MB catalog. In Kubernetes it is therefore always custom or absent. Each local proxy is its own thv process, so the check decodes only the catalog's server names (about 7 KB of allocations instead of about 20 MB for a full parse); a test pins the result to the full parser's names.
• Calls per client. calls_by_client splits the existing count by MCP client, normalized to a fixed set (claude-code, cursor, vscode, …, other, unknown). Legacy sessions are attributed through initialize and the session ID (header or SSE query parameter, an LRU cache of 4096 sessions, so a burst of new sessions forgets only the least recently used one). Modern stateless requests use their per-request clientInfo. The raw client name never leaves the process.
• vMCP reporting. vmcp serve now runs the same collector under component vmcp with backend_count, tool_count (backend tools in the most recent aggregation; excludes composite and optimizer tools, not filtered per user) and features (optimizer, code_mode, composite_tools, outgoing_auth), and sends the version check. In Kubernetes the pod has a read-only root filesystem, so the version ping is sent directly instead of through the file-backed checker, with the operator's TOOLHIVE_ANONYMOUS_ID when set (else a per-process ID). Run locally, vMCP does not print the new-version notice. For the opt-out, vMCP only reads the thv config file: it never creates or rewrites it, so a vMCP newer than the thv sharing the file cannot migrate that thv's config. A missing file means no opt-out; a file it cannot read or parse means it sends nothing instead of exiting. Only thv vmcp serve and the vmcp binary report; a vMCP embedded as a Go library (server.New/server.Serve, as brood-box does) sends no usage metrics and no update checks, and the library docs tell embedders to leave UsageMetricsMiddleware unset.
• Operator instance ID. The operator resolves its telemetry ConfigMap ID before the manager starts and injects it as TOOLHIVE_ANONYMOUS_ID into MCPServer, MCPRemoteProxy and VirtualMCPServer workloads, so reports from one installation can be grouped. Usage metrics prefer that variable over the local ID. An operator opted out with TOOLHIVE_SKIP_UPDATE_CHECK=true passes that variable to its workloads instead, so their proxies and vMCP send no update checks or usage reports, as the chart already documents; a value set on the workload wins. Outside Kubernetes, a client whose anonymous ID did not exist yet at startup (a fresh machine, or a vMCP container without persistent data) reads it again on later flushes instead of never reporting.

The update-service decodes without DisallowUnknownFields, so the new fields are accepted and dropped until it stores them; that follows in update-service and infra.

Type of change

• New feature

Test plan

• Unit tests (task test)
• Linting (golangci-lint v2.15.0 on touched packages; local task lint-fix is broken on Go 1.27)
• Manual testing (describe below)

Ran the real update-service locally behind a logging proxy and pointed patched builds at it. The only patch was the endpoint URL and a 20-second flush interval.

First run, during development:

• Local thv: built-in time server with three clients → registry_source: builtin, catalog_server: io.github.stacklok/time, calls split claude-code 3 / cursor 2 / other 1. SSE proxy attribution through the session query parameter. Remote URL → remote, runtime none. Direct image → registry none. Authz config → authz: on.
• Private registry entry acme/private-time from a local registry file → custom_local / custom; zero occurrences of acme in requests or stored rows.
• Opt-outs: TOOLHIVE_USAGE_METRICS_ENABLED=false and thv config usage-metrics disable both send nothing.
• thv vmcp serve: with optimizer → 8 backends, 17 tools, ["optimizer"]; per-client split including a hand-built Modern (2026-07-28) request.
• kind cluster: operator + two MCPServers + a VirtualMCPServer. All workloads carry the operator's ID, runtime: kubernetes; vMCP reports code_mode and outgoing_auth, 2 backends, 4 tools, and its version ping. Restarting the operator caused no workload rollout (generation stayed at 1).
• Backward compatibility: old and new payloads both return 200 from the current update-service.

Live re-run on 1b2582b33, a fresh build, against update-service with stacklok/update-service#476. Every report went through a logging proxy, and each stored record was compared with its request:

Instance	Stored profile and calls
Built-in time, streamable HTTP	builtin, io.github.stacklok/time, docker; claude-code 3, cursor 2, other 1, toolhive 3
Built-in time, SSE proxy mode	vscode 2, goose 1
Built-in time with authz	authz: on; codex 2
Private registry entry	custom_local, custom; windsurf 2
Direct image	registry none, custom; zed 1
Remote URL	remote, runtime none; claude-desktop 1
thv vmcp serve --optimizer	origin vmcp, 6 backends, 13 tools, ["optimizer"]; claude-code 3, gemini-cli 2, cline 1, jetbrains 2
thv vmcp serve	6 backends, 13 tools; roo-code 2, jetbrains 1 (the jetbrains call was a stateless 2026-07-28 request)

• 27 reports, all 200, 0 mismatches between sent and stored fields. Version checks from the CLI and from vMCP both returned 200.
• Never sent: acme, private-time, the free-text client names, the remote host, tool names and the image name appear 0 times in requests and stored records.
• toolhive 3 on the built-in server is the backend side of calls routed through vMCP.
• A tools/call counts when it arrives, before the server answers, as the proxy middleware on main already does. vMCP rejected the two jetbrains requests on the optimizer instance, and they still count.
• The later commits were covered by the re-run below.

Downstream: the 486 records stored across the replay and the live run went through Fluent Bit 4.0.3 with the production config into a local S3 server, unchanged except the date field Fluent Bit already adds. Trino's OPENX_JSON reader read all 486 with the column types from stacklok/infra#6286. Athena itself was not tested. The infra PR has the read-only check to run after release.

Live re-run on 066155a5d, the startup hardening and version-check changes. The build was stamped v0.1.0, so update-service reported a newer release. e9753869c, 1e99f5098 and 2de5a3010 came after this run. The first changes only comments and docs. The second is the fix this run found. The third makes vMCP read the thv config without creating or migrating it; a table test against real files checks each file is left byte-for-byte unchanged, including a legacy config the old loader rewrote.

Check	Result
Same six workloads and calls as above	100 reports, 0 mismatches between sent and stored fields, no private names sent
Standalone vmcp, unparseable ToolHive config.yaml	Started and served 13 tools, sent nothing, left the file untouched
Standalone vmcp, disable_usage_metrics: true	Sent nothing
Standalone vmcp, working config	Sent usage reports and a version check, printed no new-version notice
thv itself	Still prints the notice
thv vmcp serve restarted within 30 minutes of the last check	Printed the notice from vMCP's cached-check path. Fixed in 1e99f5098; after the fix only thv's own notice prints.

kind cluster on 066155a5d (operator, proxy runner and vMCP images built from the branch):

Check	Result
Operator ID	Created at startup; fetch, osv and the VirtualMCPServer carry it at generation 1
vMCP version check in Kubernetes	Sends the operator's ID, not a per-pod ID
Operator restarted twice	No rollout
ConfigMap access removed for about 3 seconds at operator startup	Retry succeeded; no rollout, ID kept
ConfigMap access removed for longer than the retry window	Workloads rolled out without the ID and reported operator-proxy. The next healthy start rolled them back with the same ID, which is the documented double rollout.
Stored ID corrupted	Operator stored a new ID; one rollout, then stable across another restart
Calls through vMCP	vMCP reported claude-code: 2 with code_mode, outgoing_auth, 2 backends and 4 tools; the fetch backend counted the same calls under toolhive

Scenario run on 7522e44b9, against update-service #476 and a v0.65.0 baseline built from the tag with the same endpoint patch. All eight scenarios passed:

Scenario	Result
Fresh install	Profile and per-client counts stored; a final report is sent on stop
Upgrade from v0.65.0	Same anonymous ID; config.yaml byte-identical; updates file valid
v0.65.0 and this build sharing one home	Neither corrupts the shared files; each can stop and restart the other's workloads
Both opt-outs	No reports from proxies, thv vmcp serve or standalone vmcp
vMCP	Origin vmcp; no new-version notice on fresh or cached checks; config.yaml never created or touched
Update service down or hanging	Same single WARN as v0.65.0, no added startup delay, reports resume afterwards
CI=true	Nothing sent
Operator in kind, upgraded from the v0.65.0 chart and installed fresh	One rollout on upgrade, ID kept, none on operator restart

311834661 and the three commits before it came after this run. They change the opt-out passed by an opted-out operator, the catalog name lookup, session eviction and late anonymous IDs. All four are covered by unit tests run with the race detector, but not by a live run.

API Compatibility

• This PR does not break the v1beta1 API, OR the api-break-allowed label is applied and the migration guidance is described above.

Changes

Area	Change
pkg/usagemetrics	Profile type with allowed values, per-client counts, client normalization, catalog lookup hook, collector options, TOOLHIVE_ANONYMOUS_ID override
pkg/runner/usage_metrics.go	RunConfig.UsageMetricsProfile()
pkg/registry/catalog_names.go, cmd/thv/main.go	Embedded catalog name lookup, registered only by thv
pkg/container/docker/client.go	RuntimeType() accessor for the runtime field
pkg/vmcp/cli/usage_metrics.go, serve.go	vMCP collector, tool-count observer, version check
pkg/vmcp/server	UsageMetricsMiddleware on Config/ServerConfig, applied after MCP parsing
pkg/operator/telemetry, cmd/thv-operator	Resolve instance ID at startup (with retry and repair), inject into workload env
docs/middleware.md, docs/arch/02-core-concepts.md, docs/arch/vmcp-library.md	Field table, allowed values, what is never sent

Does this introduce a user-facing change?

Yes, for anyone with usage metrics enabled (the default; opt-out unchanged).

• Before: each proxy sent {count, timestamp} once per 15 minutes and at midnight; vMCP sent nothing.
• After: the same reports also carry the profile fields above, including catalog_server (the built-in registry entry name, otherwise custom) and calls_by_client (normalized client names with counts). vMCP sends version checks and usage reports under component vmcp. Workloads deployed by the operator get a TOOLHIVE_ANONYMOUS_ID env var.

Never sent: tool names, private server names, URLs, images, namespaces, raw client names, error messages. Documented in docs/middleware.md.

Special notes for reviewers

• PR size. This is above the 400-line / 10-file guideline (about 980 non-test lines across 18 files). Splitting was considered; it was kept as one PR so the reported schema lands and is reviewed as a whole.
• No remote-proxy origin. Remote proxies run both locally and under the operator, so a new User-Agent component would break the existing local/operator split. server_kind: remote carries that instead.
• Why operator workloads get the operator's ID. Operator-deployed proxies run with a read-only root filesystem, so they cannot read the local anonymous ID file. Today every one of them, in every cluster, reports the same placeholder anonymous_id = "operator-proxy". Reusing the operator's existing ID (stored in its telemetry ConfigMap and already sent with its version check) splits that single global bucket into one per cluster. That makes it possible to count Kubernetes installations, see how many servers each runs, and join usage to the operator's version.
• Per-proxy counts are unaffected by the shared ID. The Athena view v_toolcount_daily groups the cumulative count by proxy_id, which is the per-process X-Instance-ID and is unchanged here, so sharing anonymous_id across a cluster's workloads cannot merge their counts. A dashboard panel that filters on anonymous_id = 'operator-proxy' would stop matching; none exists in the infra repo, but panels created only in the Grafana UI should be checked.
• vMCP rows must not be added to proxy rows. vMCP reports land in the same tool-count table with origin = 'vmcp' (confirmed against update-service). Proxy reports stay the measure of tool calls that actually run on a server. A call vMCP forwards is counted by vMCP under the real client and again by the backend proxy under client toolhive, and the two do not map one to one (a composite tool is several backend calls, the optimizer's find_tool is none). The main usage dashboard filters origin IN ('local','operator'), so its totals are unchanged. The raw views v_toolcount_daily and v_proxy_lifetime* do not filter on origin: summing them double counts vMCP traffic and lists each vMCP process as a proxy. The infra follow-up filters those views by origin.
• Operator ID and rollouts. The ID is resolved through the uncached API reader before reconcilers start, so no workload is created without it and then rolled again (operator restarts in testing caused no rollout). The first upgrade adds one env var to every workload pod template, which causes one rollout (normally coincides with the image change). The lookup retries for about 15 seconds. If it still fails, the operator starts without the ID: on a first install workloads keep reporting operator-proxy, while workloads that already carry the ID are rolled out without it and rolled again on the next successful start. A ConfigMap without a usable ID gets one stored at startup, so the leader's update check never replaces it with a different ID later. Turning TOOLHIVE_SKIP_UPDATE_CHECK on or off on the operator also rolls those workloads once. For operators already opted out, the first upgrade adds TOOLHIVE_SKIP_UPDATE_CHECK=true to their workloads, which is one rollout and stops the usage reports their proxies sent before despite the opt-out.
• vMCP config mapping. server.New rebuilds Config through ServerConfig, so the new middleware field is mapped in both buildServeConfig and deriveServerConfig; without that it was silently dropped (caught in manual testing).
• Dashboards. Panels that filter on origin will not see vMCP rows until vMCP panels are added. update-service stores the new fields in stacklok/update-service#476; the Athena columns, the origin filter on the proxy views and the dashboards follow in infra.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 83.41709% with 66 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.30%. Comparing base (87beabf) to head (3118346).

Files with missing lines	Patch %	Lines
pkg/vmcp/cli/usage_metrics.go	72.32%	31 Missing ⚠️
pkg/vmcp/cli/serve.go	0.00%	11 Missing ⚠️
cmd/thv-operator/app/app.go	0.00%	7 Missing ⚠️
pkg/operator/telemetry/telemetry.go	91.48%	4 Missing ⚠️
pkg/updates/checker.go	63.63%	4 Missing ⚠️
pkg/registry/catalog_names.go	88.88%	2 Missing ⚠️
pkg/usagemetrics/client.go	90.00%	2 Missing ⚠️
pkg/container/docker/client.go	0.00%	1 Missing ⚠️
pkg/runner/usage_metrics.go	96.55%	1 Missing ⚠️
pkg/usagemetrics/clients.go	97.56%	1 Missing ⚠️
... and 2 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7145      +/-   ##
==========================================
+ Coverage   81.26%   81.30%   +0.03%     
==========================================
  Files         881      886       +5     
  Lines       89995    90354     +359     
==========================================
+ Hits        73136    73458     +322     
- Misses      16854    16891      +37     
  Partials        5        5              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.