🧪 test(peryx): close the current mutation survivors - #2253
Merged
Conversation
Merging this PR will not alter performance
Comparing 
Footnotes
|
The open flags carry the guarantees: a member is created fresh rather than reused, is never reached through a symlink, and no descriptor the backup holds survives into a child process. Almost none of that was held to anything. Existing tests cover five of the ways the set can be broken, all through `CREATE` and `EXCL`, because dropping either makes a backup fail outright. The rest were free. `EXCL` already refuses an existing path, symlink or not, so the tests that plant a symlink pass whether or not `NOFOLLOW` is set, and nothing looked at `CLOEXEC` at all. `open_dir` was unheld in every respect: it would open a regular file as the backup root, follow a symlink to somewhere the operator never named, and hand the descriptor to any child it spawned. The descriptor assertions compare the whole flag set rather than testing for membership, since a descriptor carrying `CLOEXEC` alongside something else is not the guarantee being claimed.
Three call sites spelled out flag lists, two of them the identical four flags. Naming each set says what it is once, and puts the reason beside it rather than leaving a reader to infer it from four constants. Spelling a fixed set as a bit operation also invites the wrong operator. `^` produces the same value as `|` here, and only because every flag is a disjoint bit; a flag that later overlapped an existing one would make the two diverge with nothing to notice it. `union` names the operation the set actually wants.
The trash record example and the inspect query in the OpenAPI document named the resource `name` and the artifact `reference`, while the handler writes `resource` and `artifact` and reads the same names from the query. A client built from the document sent the wrong parameter and could not find any record. The docs site repeated the same names in its schema table. The document now uses the handler's names, and a test pins both the record example keys and the inspect query parameters to that set, so the two cannot drift apart again without the suite saying so.
A file-scoped mutation run over app/cache.rs, app/config.rs and config/raw.rs left four mutants alive, each because the suite never asked the question the mutated line answers. The fsck and repair fixture driver ignored the indexes it was handed, so a command that resolved no indexes at all produced identical output. It now echoes their names, which makes the resolved index set part of every fsck and repair assertion. The cache listing fixture only filtered by an age of one second, which a placeholder clock of one satisfies; a threshold of decades passes only against the real clock, because the gadget page was fetched at the epoch. The config check only ever printed a plural index count, and the prefetch table was checked on the raw index but never on the resolved one. Each gap has a case now, confirmed by rerunning the same mutants to zero missed.
A file-scoped mutation run over the OpenAPI builders showed that an operation could lose its summary, tag, responses or request body, and a shared example record could turn into null, without any test noticing. The existing checks read specific routes for specific facts, so a builder emptied wholesale slipped between them. One document-wide test now requires every operation to carry a tag, a summary and at least one described response, every JSON body to carry an example or a schema, and every request body to carry content. Binary bodies are exempt because an octet stream has no example worth printing. The schema endpoint gained the example this rule asks of it. The shared records that nest inside larger examples, the analytics window, the quota meters and the retention candidate, are pinned by their field sets, since a document-wide rule cannot tell a real nested record from a null.
Four mutants from the 2026-08-30 inventory were still alive in config and operator code, each on a boundary the suite only ever drove from one side. A secret file was tested one byte over its limit and never at it, so the inclusive comparison could become exclusive unnoticed. The TLS debug form was never printed with only one half of a client identity, which is the case where "either half counts" differs from "both halves". A restore was tested rolling back and moving forward but never onto a target at exactly the backup's serial, where a rollback warning would be wrong. And no backup round trip carried index settings, so a snapshot that dropped them restored to an equal config. Each has its case now, and each was confirmed by applying the mutation and watching the new case fail. The archive copy buffer is spelled as a literal rather than as an arithmetic expression. No test can observe the buffer's size, so an operator there carries no meaning a mutation could contradict.
The 2026-08-30 inventory listed the `||` chains in index, upstream, OIDC, LDAP and webhook classification as surviving mutation, and reading the tables showed why: each chain was driven through one operand. Routing options were refused only for `fallback`, refresh controls only for `credential_refresh_secs`, direct-bind LDAP only for `bind_dn`, and the issuer check only for user info and case. Turning any `||` into `&&` left the tested operand's verdict unchanged. Each table now carries one case per operand, so any single operand refuses on its own. The webhook guards gained the empty-string cases they exist for: an empty `secret` or `secret_env` is neither a literal nor a variable name, and the guard that says so was never exercised. Every mutant was applied by column and watched fail against the new case before the case was kept.
A file-scoped mutation run over config/model.rs, operator/restore.rs and operator/snapshot.rs (164 mutants) left two alive. The local-member lookup in the scheduled-job check could match any node other than this one and no test would notice, because every roster in the suite contained the identity. A roster that names other nodes but not this one now expects the schedule's own refusal, which the mutated lookup turns into a later topology error instead. The metadata and config copy check on restore could accept a member that changed between verification and copy. The blob variant of that check had its own cases; the member variant now has the same two, one per field.
A file-scoped mutation run over config/merge.rs (159 mutants) left one alive: the replica page-size bound was driven from one side only. A page one past the primary limit was refused, but no test asked for exactly the limit, so the inclusive comparison could turn exclusive unnoticed. The runtime-bounds case now runs at the limit as well as below it, with the bound named through the primary's own constant rather than a copied number.
The coverage gate rejected the document-wide OpenAPI check: it recorded each violation inside an `if`, so on a document that satisfies every rule the six recording lines never ran and the gate counted them uncovered. Each rule now produces a label and a verdict on every operation, and the violations are the labels whose verdict says so. Every line runs whatever the document holds, and the assertion is unchanged: the list of operations with missing pieces must be empty.
A file-scoped mutation run over operator/verify.rs (111 mutants) left five alive, all for the same reason: the tests asserted that a report contained the line they cared about and never what else it held. Under a mutated problem counter the expected line still appears, so a `contains` check agrees with a broken count. Each report is now pinned by its problem total, which makes the count part of every assertion: a tampered blob is two problems and a missing one is one, a duplicate index row is three, and a metadata mismatch is two lines and nothing from the scans an opened store would have run. The placement count needed a case rather than an assertion, because no test put a placement row in a single-node store, where the mode check makes the backup and its verification agree on counting none.
The site serves a committed copy of the document since #2252, and `just docs` fails when it differs from what the binary produces. Renaming the trash record's `name` and `reference` to the `resource` and `artifact` the handler actually reads, and giving the schema endpoint the example the document-wide check requires, both changed the generated output. Rebuilt through `just openapi`. Those two edits are the whole diff.
gaborbernat
force-pushed
the
test/mutants-peryx-1893
branch
from
September 9, 2026 14:51
ece28ca to
3d2bc79
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The nightly mutation inventory from #1893 listed 96 surviving mutants in the
peryxcrate as of 2026-08-30. Rerunningcargo mutantsfile by file on currentmaingave the real picture for ten of the crate's files, about 540 of its 1034 mutants: most of the listed rows had died to work merged since, and the survivors that remained clustered around the same shape, a predicate or bound that the suite only ever drove from one side. 🔍One survivor was a defect rather than a gap. The OpenAPI document described a trash record with
nameandreference, and told a client to look up/+trash/recordby those names, while the handler writes and readsresourceandartifact. A client built from the document sent the wrong query and could not find any record. The document and the docs site now use the handler's names, and the document is checked for the wire field set so the two cannot drift again.The rest are one-sided fixtures: a secret file tested one byte over its limit and never at it, a replica page size tested past the primary's cap and never at it, a restore tested rolling back and forward and never onto a target at the backup's own serial,
||chains in index, upstream, OIDC, LDAP and webhook classification each driven through a single operand, and a fsck fixture that ignored the indexes it was handed so a command resolving none produced identical output. Each of those now has the case that decides it, and every change was confirmed by applying the mutation at its exact column and watching the new case fail before it was kept. 🧪 The OpenAPI builders also gained one document-wide check so an operation emptied of its summary, responses or request body fails regardless of which route it serves. The archive copy buffer inoperator/mod.rsis written as a literal because no behaviour can observe its arithmetic.Files not yet measured on current
mainareoperator/verify.rs,server.rs,process.rs,operator/mod.rs,app/retention.rs,operator/backup.rsand a tail of small files; they follow in later batches on the same issue.Refs #1893