Skip to content

Add YAML workflow loader validation tests (#543) - #631

Merged
leynos merged 1 commit into
mainfrom
issue-543-add-yaml-loader-validation-tests-for-workflow-contracts
Sep 1, 2026
Merged

Add YAML workflow loader validation tests (#543)#631
leynos merged 1 commit into
mainfrom
issue-543-add-yaml-loader-validation-tests-for-workflow-contracts

Conversation

@leynos

@leynos leynos commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Summary

This branch adds focused YAML workflow-loader tests so GitHub Actions trigger
keys and malformed workflow roots retain their intended parsing and validation
contracts.

Closes #543.

Review walkthrough

  • Start with the loader tests to verify YAML 1.2 boolean resolution, the string preservation of GitHub Actions trigger words, and the root-document rejection paths.

Validation

  • make check-fmt: passed
  • make lint: passed
  • make doc-coverage: passed (99.07%)
  • make typecheck: passed
  • make test: passed (2,679 passed, 3 skipped; doctests passed)
  • make markdownlint: passed
  • make nixie: passed
  • Focused loader selection: 11 passed

Notes

make test-workflow-contracts has a pre-existing, unrelated failure in the
check-fmt pipeline assertion. The focused loader selection and all required
commit gates pass; this branch does not alter that unrelated contract.

Summary by Sourcery

Strengthen YAML workflow-loader contract coverage for scalar parsing and malformed document validation.

Enhancements:

  • Add focused workflow-loader contract coverage for YAML 1.2 boolean resolution and preservation of GitHub Actions trigger words as strings.
  • Add validation coverage ensuring malformed or non-string-keyed workflow document roots are rejected.

Tests:

  • Expand workflow contract tests to cover scalar resolution and invalid workflow document roots.

Enhancements:

  • Add focused workflow-loader contract coverage for YAML 1.2 boolean
    resolution and preservation of GitHub Actions trigger words as strings.
  • Add validation coverage for rejecting empty, scalar, sequence, and
    non-string-keyed workflow roots.

Tests:

  • Expand workflow contract tests to cover loader scalar resolution and
    malformed document-root validation.

References

@coderabbitai

coderabbitai Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Team

Run ID: 8a3a220e-0492-4de8-91e5-968951cada43

📥 Commits

Reviewing files that changed from the base of the PR and between 53c6f8a and 5007c4f.

📒 Files selected for processing (1)
  • tests/workflow_contracts/ci_lint_test.py
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • leynos/monotony (auto-detected)
  • leynos/whitaker (auto-detected)
  • leynos/rstest-bdd (auto-detected)
  • leynos/shared-actions (auto-detected)
  • leynos/mdtablefix (auto-detected)

Included review availability: 2 reviews are currently available. Your included PR review attempts over the past 7 days set your current allowance at 4 reviews per hour.


Summary

  • Add focused _WorkflowLoader tests for GitHub Actions on keys and YAML 1.2 boolean values.
  • Add _load tests for empty, scalar, sequence, and non-string-keyed mapping roots.
  • Link coverage to issue #543.
  • Keep validation independent of unrelated workflow contract tests.

Walkthrough

The pull request adds contract tests for the workflow YAML loader. The tests cover GitHub Actions trigger words, YAML 1.2 boolean variants, and invalid workflow document roots.

Changes

Workflow loader contracts

Layer / File(s) Summary
Workflow loader contract tests
tests/workflow_contracts/ci_lint_test.py
Add tests for string trigger words, lowercase, title-case and uppercase boolean literals, and invalid empty, scalar, list and non-string-keyed workflow documents.

Suggested labels: Issue

Poem

on stays text in the YAML stream
Booleans resolve as the tests dream
Empty roots meet a guarded gate
Lists and scalars cannot pass their fate
Workflow contracts now stand straight

Merge Risk: ⚪ Minimal · up to 5007c

This PR adds focused validation coverage without changing production behavior, and no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the YAML workflow-loader validation tests and references linked issue #543.
Description check ✅ Passed The description directly explains the added YAML loader tests, acceptance coverage, validation results, and the unrelated pre-existing failure.
Linked Issues check ✅ Passed The changes satisfy issue #543 by testing preservation of the on key, YAML boolean resolution, and rejection of empty, scalar, sequence, and non-string-keyed mapping documents.
Out of Scope Changes check ✅ Passed Keep the changes within scope: the pull request only adds workflow-loader contract tests required by issue #543.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 1 files.
Testing (Overall) ✅ Passed Pass this check. The pull request changes only tests/workflow_contracts/ci_lint_test.py; it adds no untested product behaviour. The tests directly exercise _WorkflowLoader and load_workflow with…
User-Facing Documentation ✅ Passed Pass the user-facing documentation check. The pull request changes only tests/workflow_contracts/ci_lint_test.py (+80 lines) and adds contract tests. It does not change production behaviour, the `_W…
Developer Documentation ✅ Passed PASS — The pull request changes only tests/workflow_contracts/ci_lint_test.py. It adds contract tests and imports the existing _WorkflowLoader and load_workflow helpers. It does not add or chang…
Module-Level Documentation ✅ Passed The changed module has a module-level docstring at line 1. It clearly states the module purpose, explains the CI workflow contracts it protects, identifies related contract modules and the shared `wor…
Testing (Unit And Behavioural) ✅ Passed Pass the testing check. The diff adds focused unit coverage for _WorkflowLoader trigger-word and YAML 1.2 boolean resolution, including all case variants. It adds boundary coverage through `load_wor…
Testing (Property / Proof) ✅ Passed Pass the property-testing check. The pull request changes only tests/workflow_contracts/ci_lint_test.py; it introduces no production invariant, lemma, or proof assumption. `tests/workflow_contracts/…
Testing (Compile-Time / Ui) ✅ Passed Accept the testing change. The pull request changes only tests/workflow_contracts/ci_lint_test.py and adds Python tests for YAML parsing and validation. No Rust or TypeScript compile-time behaviour …
Unit Architecture ✅ Passed Pass. The committed diff changes only tests/workflow_contracts/ci_lint_test.py and adds 80 test lines. It does not change production query or command paths, dependencies, global state, network acces…
Domain Architecture ✅ Passed The pull request only adds contract tests in tests/workflow_contracts/ci_lint_test.py. It does not change domain logic, domain entities, commands, repositories, or adapter boundaries. The tests exer…
Observability ✅ Passed Pass the observability check. The pull request changes only tests/workflow_contracts/ci_lint_test.py and adds loader contract tests. It does not introduce or alter production operational behaviour, …
Full details: Testing (Overall)

Explanation

Pass this check. The pull request changes only tests/workflow_contracts/ci_lint_test.py; it adds no untested product behaviour. The tests directly exercise _WorkflowLoader and load_workflow without mocks. They assert preservation of on, yes, off, and no as strings, all six supported true/false case variants, and rejection of empty, scalar, list, and non-string-keyed mapping roots. The assertions check both returned values and relevant validation messages, so plausible resolver or validation regressions fail.

Full details: User-Facing Documentation

Explanation

Pass the user-facing documentation check. The pull request changes only tests/workflow_contracts/ci_lint_test.py (+80 lines) and adds contract tests. It does not change production behaviour, the _WorkflowLoader implementation, or any user-facing interface. No new documentation requirement is triggered.

Full details: Developer Documentation

Explanation

PASS — The pull request changes only tests/workflow_contracts/ci_lint_test.py. It adds contract tests and imports the existing _WorkflowLoader and load_workflow helpers. It does not add or change an internal API, architectural boundary, abstraction, tooling requirement, build requirement, design decision, roadmap item, or execplan. Therefore, this documentation check is not applicable.

Full details: Module-Level Documentation

Explanation

The changed module has a module-level docstring at line 1. It clearly states the module purpose, explains the CI workflow contracts it protects, identifies related contract modules and the shared workflow_loading.py helper, and gives the test command. The pull-request diff adds only tests and imports; it does not remove or weaken this documentation.

Full details: Testing (Unit And Behavioural)

Explanation

Pass the testing check. The diff adds focused unit coverage for _WorkflowLoader trigger-word and YAML 1.2 boolean resolution, including all case variants. It adds boundary coverage through load_workflow for empty, scalar, sequence, and non-string-keyed roots. The tests use real YAML input and temporary files, with no mocks. The changed file is the test module only, so no additional end-to-end workflow test is required for this pull request.

Full details: Testing (Property / Proof)

Explanation

Pass the property-testing check. The pull request changes only tests/workflow_contracts/ci_lint_test.py; it introduces no production invariant, lemma, or proof assumption. tests/workflow_contracts/workflow_loading.py is identical to the parent revision. The additions are finite parameterized examples for six YAML boolean spellings and four invalid root forms, so this check does not require Hypothesis or a formal proof.

Full details: Testing (Compile-Time / Ui)

Explanation

Accept the testing change. The pull request changes only tests/workflow_contracts/ci_lint_test.py and adds Python tests for YAML parsing and validation. No Rust or TypeScript compile-time behaviour is changed, so a trybuild or equivalent compile-time test is not applicable. The tests assert stable semantic values and error fragments from a structured loader; they do not test generated text, UI output, or a snapshot-worthy rendering. Direct assertions are therefore appropriate, and no snapshot test is required by this check.

Full details: Unit Architecture

Explanation

Pass. The committed diff changes only tests/workflow_contracts/ci_lint_test.py and adds 80 test lines. It does not change production query or command paths, dependencies, global state, network access, clocks, or persistence. The tests call the existing fallible loader APIs explicitly, dispose _WorkflowLoader instances in finally blocks, and use tmp_path.write_text only to create isolated test fixtures. No explicit Unit Architecture failure condition is introduced.

Full details: Domain Architecture

Explanation

The pull request only adds contract tests in tests/workflow_contracts/ci_lint_test.py. It does not change domain logic, domain entities, commands, repositories, or adapter boundaries. The tests exercise the existing YAML workflow loader at its workflow-loading boundary and do not introduce HTTP, SQL, ORM, filesystem, queue, UI, CLI, or vendor concerns into domain code.

Full details: Observability

Explanation

Pass the observability check. The pull request changes only tests/workflow_contracts/ci_lint_test.py and adds loader contract tests. It does not introduce or alter production operational behaviour, service boundaries, metrics, tracing, alerts, or logging. No additional observability is required for these test-only changes.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issue-543-add-yaml-loader-validation-tests-for-workflow-contracts

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.


Comment @coderabbitai help to get the list of available commands.

@sourcery-ai

sourcery-ai Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Adds focused workflow-loader contract tests covering YAML 1.2 boolean behavior, preservation of GitHub Actions trigger words as strings, and rejection of invalid root document shapes with exact validation messages.

File-Level Changes

Change Details Files
Add focused tests for YAML scalar resolution and GitHub Actions trigger-key preservation.
  • Verify on, yes, off, and no remain strings in both keys and values.
  • Verify lowercase, titlecase, and uppercase YAML 1.2 true/false literals resolve to booleans.
  • Instantiate and dispose the private workflow loader directly for parser-level assertions.
tests/workflow_contracts/ci_lint_test.py
Add validation coverage for malformed workflow document roots.
  • Parameterize empty, scalar, sequence, and non-string-keyed mapping documents.
  • Write temporary workflow files and assert the exact user-facing validation errors from load_workflow.
tests/workflow_contracts/ci_lint_test.py

Assessment against linked issues

Issue Objective Addressed Explanation
#543 Add focused tests verifying that _WorkflowLoader preserves the GitHub Actions "on" key and other YAML 1.2 non-boolean trigger words as strings while resolving true and false literals as booleans.
#543 Add focused tests verifying that _load rejects empty, scalar, and list workflow documents, as well as mappings containing non-string keys.
#543 Keep the validation tests isolated from unrelated workflow contract tests and assert the intended user-facing error behavior.

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

codescene-access[bot]

This comment was marked as outdated.

Cover YAML 1.2 booleans and malformed workflow roots so contract failures
remain explicit and close to their parsing boundary.
@leynos
leynos force-pushed the issue-543-add-yaml-loader-validation-tests-for-workflow-contracts branch from 29921cb to 5007c4f Compare August 31, 2026 23:49
codescene-access[bot]

This comment was marked as outdated.

@leynos
leynos marked this pull request as ready for review August 31, 2026 23:50

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @leynos, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 1 day and 21 hours by commenting @sourcery-ai review. Upgrade to get a review now.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 31, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-31T23:52:09.226918Z 5007c4f Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai coderabbitai Bot added the Issue label Aug 31, 2026

@codescene-access codescene-access Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No quality gates enabled for this code.

@leynos
leynos merged commit fec16a3 into main Sep 1, 2026
25 of 28 checks passed
@leynos
leynos deleted the issue-543-add-yaml-loader-validation-tests-for-workflow-contracts branch September 1, 2026 00:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add YAML loader validation tests for workflow contracts

1 participant