Skip to content

Docs: corpus README says 5-case smoke per PR (CI runs full 10); cassette.py docstring path is stale #75

Description

@JumpTechCode

Summary

Two stale doc strings that no longer match the code.

Evidence

  • sentinel/evals/corpus/README.md:38-39 — claims "the deterministic 5-case smoke subset CI runs on every PR." CI actually runs the full 10-case corpus per PR (.github/workflows/ci.yml:108-109, make evals with no --smoke); the corpus has 10 cases.
  • sentinel/evals/cassette.py:11-12 — docstring says cassettes live under cassettes/<prompt_version>/<model_id>/<key>.json, but the actual layout is flat (cassettes/<key>.json); prompt_version/model_id are folded into the key hash (compute_cassette_key, cassette.py:52-70; path at :153).

Fix

Update both doc strings to match reality (full 10-case per-PR run; flat cassette layout).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions