Skip to content

test(tracing): isolate span emission into per-test tmp dirs - #371

Merged
forrestjgq merged 1 commit into
mainfrom
test/isolate_tracing_state_dir
Aug 28, 2026
Merged

test(tracing): isolate span emission into per-test tmp dirs#371
forrestjgq merged 1 commit into
mainfrom
test/isolate_tracing_state_dir

Conversation

@forrestjgq

Copy link
Copy Markdown
Contributor

Summary

Unit tests were leaking tracing spans into the real ~/.raven/traces store. Tracing is on by default and trace.span() calls are embedded in library code (agent loop, subagents, TUI RPC), so any test exercising those paths emitted spans with fabricated session keys (session-a, weixin:c, tui:default, ...). Those spans then surfaced as phantom sessions in raven trajectory, while the TUI /sessions list (which reads the real session store) stayed clean.

This PR adds an autouse fixture in tests/conftest.py that:

  • redirects RAVEN_TRACING_DIR to a per-test tmp directory, and
  • resets the lazy module-level TraceStore cache (raven.tracing.spans._store) around each test, since it pins the directory resolved at first emit.

Key decisions:

  • Only the directory is redirected, not the enabled switch, so the suite keeps exercising the real emission path with zero behavior change; leaks land in tmp instead.
  • This centralizes the setenv-plus-cache-reset pattern that 8 tracing/trajectory test files already perform manually. Those tests keep working: their own monkeypatch.setenv runs after the fixture and wins, and they already reset the cache themselves.
  • Existing phantom sessions come from archived span logs written before this fix; the fixture only prevents new leaks. One archive file mixes a real session with test keys, so cleanup needs line-level filtering and is left out of scope.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Test-suite hygiene (test commit type).

Verification

  • uv run pytest tests/test_tracing_api.py tests/test_trajectory_store.py tests/test_trajectory_bundle.py tests/test_trajectory_replay.py tests/test_trajectory_redact.py tests/test_cli_tracing_commands.py tests/test_cli_trajectory_commands.py tests/test_cli_trajectory_browse.py -q: 308 passed, 1 failed (test_viewer_health_true_for_our_viewer, fails identically on unmodified main, environment-specific)

  • uv run pytest tests/test_agent_loop_run_emit.py tests/test_subagent_manager.py tests/test_tui_rpc_turn_send.py tests/test_deep_research_tool.py tests/test_agent_loop_approval.py -q: 109 passed; the real ~/.raven/traces log did not grow during the run

  • Positive check: running tests/test_agent_loop_run_emit.py with --basetemp shows the previously leaked session keys (cli:c, tg:c, weixin:c) now land in per-test tmp dirs

  • uv run pytest tests/ --ignore=tests/integration -q: 6837 passed, 11 failed, 20 errors; the same failure set reproduces on unmodified main (environment-specific: cron/tz, everos server, theme, config loader, viewer health), zero regressions from this change

  • uv run ruff check tests/conftest.py and uv run ruff format --check tests/conftest.py: both pass

  • Relevant tests pass locally

  • Relevant lint / type checks pass locally

  • User-facing docs or screenshots are updated when needed

Docs box left blank: test-only change, no user-facing docs affected.

Risk

  • Security impact considered
  • Backward compatibility considered
  • Rollback path is clear for risky changes

Test-only change; no production code touched. One known interaction: RAVEN_TRACING_DIR takes precedence over RAVEN_HOME, so a future test that fakes RAVEN_HOME and expects traces under it must also set RAVEN_TRACING_DIR itself. Rollback: revert the single commit.

Related Issues

N/A

Tracing is on by default and trace.span() calls are embedded in library
code, so any test exercising the agent loop emitted spans with fabricated
session keys into the real ~/.raven/traces store; they surfaced as phantom
sessions in raven trajectory while the TUI /sessions list stayed clean.

Add an autouse fixture that redirects RAVEN_TRACING_DIR to a per-test tmp
directory and resets the module-level TraceStore cache around each test.
Only the directory is redirected, not the enabled switch, so the suite
keeps exercising the real emission path with zero behavior change. Tests
that pin their own directory keep working: their setenv runs after the
fixture and wins, and they already reset the cache themselves.

Co-authored-by: Claude (claude-fable-5) <noreply@anthropic.com>

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers; this can merge as far as I am concerned.

I reviewed the full github/main...HEAD diff, the tracing configuration/store lifecycle and emission callers, every test override of RAVEN_TRACING_DIR, relevant fixture history, backward compatibility, and the repository rules/domain glossary. The fixture keeps tracing enabled, redirects each test to its own tmp_path, and resets the lazy store on both sides of the test; existing tracing tests still override the directory after the autouse fixture and reset the singleton where they emit, so no assertions were weakened to obtain isolation.

Verification:

  • git diff --check github/main...HEAD passed.
  • uv run pytest tests/test_tracing_api.py tests/test_cli_tracing_commands.py tests/test_trajectory_store.py -q: 119 passed.
  • A controlled agent-loop emission test passed and wrote nothing to an externally supplied tracing directory, confirming the fixture's per-test override.
  • uv run pytest: 6,867 passed, 43 skipped, 13 deselected, with 4 failures and 2 errors before the process exited 139. One untouched theme assertion reproduces in isolation; the other five reports came after the suite exhausted file descriptors and all seven involved late-stage tests passed in a fresh-process rerun. These failures are not caused by the tracing fixture.

@forrestjgq
forrestjgq merged commit dd38d2a into main Aug 28, 2026
11 checks passed
@forrestjgq
forrestjgq deleted the test/isolate_tracing_state_dir branch August 28, 2026 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants