chore: derive cassette suites from the tree and sort-check manifest lists - #2437
Open
gold-silver-copper wants to merge 1 commit into
Open
chore: derive cassette suites from the tree and sort-check manifest lists#2437gold-silver-copper wants to merge 1 commit into
gold-silver-copper wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Two kinds of hand-maintained list made unrelated PRs collide on the same hunk: the
PROVIDER_CASSETTE_SUITEStable intests/common/cassette_safety.rs(one entry per provider, every wrapper name spelled out, 28 entries) and the per-crate lists inCargo.toml, rig-core's providers module list and the README integration table. This PR removes the first and sort-checks the rest.Cassette suites are discovered from the tree. A provider is a
tests/providers/<provider>/directory with atests/<provider>.rsbinary; a cassette wrapper is any call to a function namedwith_*cassette*inside that directory, first argument the scenario. Discovery is call-site based because some providers define wrappers through macros. The table and thewrapper_namesfield are gone; the module doc states the convention as the invariant.Every existing failure mode is kept: secrets scan per provider directory, loose files under
tests/cassettes/, a cassette directory no binary scans, a non-identifier provider name, a binary that omits the safety module, missing and orphaned cassettes. Two are new:with_openai_cassette) called from a different provider's directory fails with an explicit message instead of a confusing missing/orphaned pair;tests/providers/<x>/directory with notests/<x>.rsbinary that calls cassette wrappers fails, since nothing would check its scenarios.The table was also unsound: a wrapper defined but not listed was silently ignored.
with_openai_boxed_cassetteandwith_anthropic_boxed_cassettewere in that state; both are now discovered (their scenarios already had cassettes, so nothing became missing or orphaned).cargo xtask check-sorted-blocksverifies that every list betweensorted: start/sorted: endmarkers is in case-insensitive byte order (a textual check, since line order is the point). Marked: the companion-crate block and features block inCargo.toml,[dev-dependencies], both parts of[workspace.dependencies],crates/rig-core/src/providers/mod.rs, and the README integration table. It runs in thestable / fmtjob.AGENTS.mdandCONTRIBUTING.mdstate the rule.Decisions:
lancedbin[dependencies]is not a companion crate, so it sits just below the sortedrig-*block with a comment, next to thelancedbfeature that needs it.rig-derivemoved up with the other core deps for the same reason.ScyllaDB, SQLiterather thanSQLite, ScyllaDB.common/cassettes.rsis 3,900 lines they have no use for. The momenttests/cassettes/<provider>appears, every binary fails until theirs includes it, which is the same guarantee the table gave.Changelog
Migration
None.
Type of change
Testing
cargo nextest run --features bedrock -p rig -E 'test(cassette_safety)': 36 tests across 18 provider binaries pass.support.rsfiles where a derived wrapper passes its scenario variable to the base wrapper. The first is handled as described; the second by not treating a wrapper's own body as a call site.tests/providers/anthropic/cassette/callingwith_openai_cassette("agent/completion_smoke", …)andwith_anthropic_cassette("does/not/exist", …)fails with the foreign-provider message and a missing cassette; the probe file was removed.cargo test -p xtask sorted_blocks: 6 unit tests (multi-line entries, table header, unclosed block, duplicates, Rust and Markdown entries).cargo xtask check-sorted-blocks: 7 blocks in 3 files ok.check-test-layoutandgenerate-provider-aliases --checkok.cargo fmt -- --check,cargo clippy -p xtask --all-targets,cargo clippy -p rig --tests --features bedrockclean;git diff --stat -- Cargo.lock tests/cassettesempty;.github/scripts/check-frozen-release-docs.sh origin/mainpasses.Checklist:
CHANGELOG.mdorMIGRATING.md(they are generated at release)