test: restore the metrics collector global after each test

[Andreymi]

The problem

hindsight_api.metrics._metrics_collector is a module-level global that defaults to NoOpMetricsCollector. The API lifespan swaps it for the real collector (create_metrics_collector() in api/http.py) and nothing puts it back, so under xdist every test scheduled after any app-starting test in the same worker runs against the real collector.

That is not a neutral difference. The no-op swallows its arguments; the real collector compares them. A test that mocks an LLM response leaves response.usage.prompt_tokens_details.cached_tokens a MagicMock, and record_llm_call's if cached_input_tokens > 0 raises:

TypeError: '>' not supported between instances of 'MagicMock' and 'int'

Which tests fail depends on the xdist schedule, so the set moves between runs and reads as random flakiness. Every one of them passes in isolation.

Measured on v0.9.2

pytest tests/ -q -m "not hs_llm_mat and not hs_llm_core" on a clean v0.9.2 checkout produced 36 of these TypeErrors. A second run of the same tree produced a partly different set — same code, different victims.

Hit hardest: test_lmstudio_tool_choice.py, test_tool_path_reasoning_effort.py, test_tool_choice_required_downgrade.py, test_reasoning_effort_explicit_config.py.

The fix

Snapshot the global before each test, restore it after — the same guard the neighbouring _cleanup_leaked_span_recorders fixture already applies to the span-recorder registry. test_metrics.py::test_returns_noop_by_default also already does this by hand for itself; this generalises it to every test.

Test-only: no shipped behaviour changes.

Reproducer

# tests/test_aaa_probe.py — stands in for any app-starting test
from hindsight_api.metrics import create_metrics_collector, initialize_metrics

def test_probe():
    initialize_metrics(service_name="probe", service_version="0")
    create_metrics_collector()

pytest tests/test_aaa_probe.py tests/test_lmstudio_tool_choice.py \
       tests/test_tool_path_reasoning_effort.py -p no:randomly

Before: 2 failed, 16 passed. After: 18 passed.

Notes

I checked that nothing depends on the real collector surviving a test: the only test that reads the live global is test_llm_token_metrics.py::test_noop_collector_when_metrics_disabled, which asserts it is the no-op — it is a latent victim of the same leak, so this makes it more reliable, not less. test_base_path.py::test_base_path_metrics reads the Prometheus registry via generate_latest(), not this global, and is unaffected.

[strix-security]

Strix Security Review

Warning

This pull request has 13 commits after the last Strix review (44ecb81). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

No security issues found.

_{Updated for 44ecb81.}

---

Reviewed by Strix
_{Re-run review · Configure security review settings}