feat(plan-feature): add convention applicability rules

[ruromero]

Summary

• Creates shared/convention-applicability-rules.md defining how to validate convention file-type applicability before applying conventions to tasks or upgrading review suggestions
• Adds a new step (step 2) in plan-feature/SKILL.md convention-aware enrichment "How to apply" list requiring applicability validation per the shared rules
• Updates eval assertions across all 4 plan-feature evals to verify convention applicability filtering behavior

Implements TC-4285

Test plan

• Verify shared/convention-applicability-rules.md defines file-type extraction, comparison logic, rationale format, ambiguous-case handling, and filtering example
• Verify plan-feature/SKILL.md convention-aware enrichment references the shared rules at step 2
• Verify eval assertions include applicability filtering coverage in all 4 evals
• Run claude plugin validate plugins/sdlc-workflow to confirm plugin validity

🤖 Generated with Claude Code

Summary by Sourcery

Define shared rules for validating convention file-type applicability and integrate them into the plan-feature skill and shared conventions documentation.

Enhancements:

• Update the plan-feature skill to require validating convention file-type applicability and recording rationales before applying conventions to tasks.
• Introduce a shared convention-applicability-rules module for consistent applicability checking across skills.

Documentation:

• Add shared documentation describing convention file-type applicability rules, rationale formatting, ambiguous-case handling, and examples.
• Document shared convention applicability rules in CONVENTIONS.md for reuse by plan-feature and verify-pr skills.

Tests:

• Extend plan-feature evals to assert correct filtering behavior based on convention applicability.

[sourcery-ai]

Reviewer's Guide

Introduces shared convention applicability rules for file-type-aware use of CONVENTIONS.md, wires plan-feature to validate conventions against target files before enrichment, documents the shared module in CONVENTIONS.md, and updates plan-feature evals to assert correct applicability filtering behavior.

Sequence diagram for convention applicability validation in plan-feature

sequenceDiagram
    participant PlanFeatureSkill
    participant Task
    participant ConventionsMd
    participant ConventionApplicabilityRules
    participant ImplementationNotes

    PlanFeatureSkill->>Task: Read FilesToModify and FilesToCreate
    PlanFeatureSkill->>ConventionsMd: Collect candidate conventions

    loop For each candidate convention
        PlanFeatureSkill->>ConventionApplicabilityRules: Determine scope signals from convention section
        ConventionApplicabilityRules-->>PlanFeatureSkill: ConventionScope (file types, directories, or broad)

        PlanFeatureSkill->>ConventionApplicabilityRules: Compare ConventionScope with task target files
        ConventionApplicabilityRules-->>PlanFeatureSkill: ApplicabilityResult (applies or excluded) with rationale text

        alt Convention is applicable
            PlanFeatureSkill->>ImplementationNotes: Append Per_CONVENTIONS_md_line with action
            PlanFeatureSkill->>ImplementationNotes: Append applicability rationale
        else Convention is not applicable
            PlanFeatureSkill-->>ImplementationNotes: Skip convention (no entry)
        end
    end

Loading

File-Level Changes

Change	Details	Files
Add shared convention applicability rules document and reference it from shared modules index.	Create a shared markdown spec describing how to derive convention target file types, compare scope against task/PR files, handle ambiguous cases, and require human-readable applicability rationales. Document the new shared rules module in the shared modules section so other skills (plan-feature, verify-pr) are expected consumers.	plugins/sdlc-workflow/shared/convention-applicability-rules.md CONVENTIONS.md
Update plan-feature skill to enforce applicability validation before applying conventions during task enrichment.	Insert a new step in the convention-aware enrichment instructions to validate convention file-type applicability using the shared rules before including conventions. Require exclusion of conventions with non-overlapping file-type scope and inclusion of an applicability rationale for those that pass. Renumber subsequent steps in the "How to apply" instructions to account for the new validation step.	plugins/sdlc-workflow/skills/plan-feature/SKILL.md
Extend plan-feature evals to assert convention applicability filtering behavior.	Update eval definitions so they cover scenarios where conventions should be filtered out or treated as broadly applicable based on file-type scope. Align eval expectations with the new applicability rationale and filtering rules for all plan-feature-related evals.	evals/plan-feature/evals.json

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've reviewed your changes and they look great!

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[ruromero]

Verification Report for TC-4285 (commit b361dfd)

Check	Result	Details
Review Feedback	PASS	1 review (sourcery-ai[bot], positive — "looks great!"); no code change requests
Root-Cause Investigation	N/A	No sub-tasks created
Scope Containment	PASS	All 4 PR files match task scope (1 created + 3 modified including doc update)
Diff Size	PASS	+120 -6 across 4 files — proportionate
Commit Traceability	PASS	Commit references TC-4285 via Implements TC-4285 in body
Sensitive Patterns	PASS	1 false positive: .env in adversarial eval assertion text (expected test fixture content)
CI Status	WARN	Eval PR Changes failed (Google Cloud auth unavailable for fork PRs — infrastructure, not code); Plugin Validation passed
Acceptance Criteria	PASS	4/4 criteria met (see details below)
Test Quality	N/A	No test files in PR
Test Change Classification	N/A	No test files in PR
Verification Commands	N/A	No commands in task

Acceptance Criteria Details

#	Criterion	Result
1	shared/convention-applicability-rules.md defines file-type extraction, comparison logic, rationale format, ambiguous-case handling, filtering example	PASS
2	plan-feature/SKILL.md convention-aware enrichment references shared rules with applicability validation step	PASS
3	Applicability rationale format defined for downstream skill auditing	PASS
4	Conventions without explicit scope treated as broadly applicable (no false rejections)	PASS

Overall: WARN

CI infrastructure limitation only — no code defects found. All acceptance criteria met, scope contained, commit traceable.

---

This comment was AI-generated by sdlc-workflow/verify-pr v0.7.2.

[github-actions]

Eval Results

Eval Results: plan-feature

Eval	Passed	Failed	Pass Rate
eval-1	13/13	0	100%
eval-2	11/11	0	100%
eval-3	12/12	0	100%
eval-4	8/8	0	100%
eval-5	12/12	0	100%

Pass rate: 100% · Tokens: 52,962 · Duration: 202s

Baseline (3146a86): 82% · 44,434 tokens · 175s

---

Generated by sdlc-workflow/run-evals v0.9.1

[ruromero]

Verification Report for TC-4285 (commit 8258abe)

Check	Result	Details
Review Feedback	N/A	No inline review comments; bot reviews only
Root-Cause Investigation	N/A	No sub-tasks created
Scope Containment	PASS	All 4 task-specified files present, exact match
Diff Size	PASS	+122/-7 across 4 files — proportionate to task scope
Commit Traceability	PASS	Single commit references TC-4285 via "Implements TC-4285"
Sensitive Patterns	PASS	No secrets or sensitive data detected
CI Status	PASS	All 4 CI checks passed
Acceptance Criteria	PASS	4 of 4 criteria met
Test Quality	N/A	No traditional test files; project uses eval-based validation
Test Change Classification	ADDITIVE	5 eval assertions added across all eval blocks, none removed
Verification Commands	N/A	No verification commands specified

Overall: PASS

All checks pass. The PR correctly implements the convention applicability rules document, updates the plan-feature SKILL.md with an applicability validation step, adds eval assertions for the new behavior, and updates CONVENTIONS.md with the shared module entry.

---

This comment was AI-generated by sdlc-workflow/verify-pr v0.9.1.

[ruromero]

Verification Report for TC-4285 (commit 8258abe) — Revised

Check	Result	Details
Review Feedback	WARN	Eval 4 assertion 8 (convention applicability format) fails — sub-task TC-4554 created
Root-Cause Investigation	N/A	—
Scope Containment	PASS	All 4 task-specified files present, exact match
Diff Size	PASS	+122/-7 across 4 files — proportionate to task scope
Commit Traceability	PASS	Single commit references TC-4285 via "Implements TC-4285"
Sensitive Patterns	PASS	No secrets or sensitive data detected
CI Status	PASS	All 4 CI checks passed
Acceptance Criteria	WARN	4/4 criteria met structurally, but eval 4 shows the prescribed format is not consistently followed under adversarial conditions
Test Quality	N/A	No traditional test files; project uses eval-based validation
Test Change Classification	ADDITIVE	5 eval assertions added across all eval blocks, none removed
Verification Commands	N/A	No verification commands specified

Overall: WARN

The PR correctly implements the convention applicability rules document and all acceptance criteria are structurally satisfied. However, the eval results review (#pullrequestreview-4363925347) reports that eval 4 assertion 8 (convention applicability format) fails:

• The skill produces prose-format assessments ("Applicable — this task creates the model layer") instead of the prescribed format ("Applies: task modifies <file> matching the convention's <scope>")
• Inapplicable conventions are listed with "Not applicable" annotations instead of being excluded entirely

Sub-task created: TC-4554 — Strengthen convention applicability format to pass adversarial eval

Revises the previous PASS report which did not account for the eval results review.

---

This comment was AI-generated by sdlc-workflow/verify-pr v0.9.1.

[ruromero]

Verification Report for TC-4285 (commit 8258abe) — Revised v2

Check	Result	Details
Review Feedback	WARN	Eval failures identified — 2 sub-tasks created (see below)
Root-Cause Investigation	N/A	—
Scope Containment	PASS	All 4 task-specified files present, exact match
Diff Size	PASS	+122/-7 across 4 files — proportionate to task scope
Commit Traceability	PASS	Single commit references TC-4285 via "Implements TC-4285"
Sensitive Patterns	PASS	No secrets or sensitive data detected
CI Status	PASS	All 4 CI checks passed
Acceptance Criteria	WARN	4/4 criteria met structurally, but evals show two behavioral gaps (see below)
Test Quality	N/A	No traditional test files; project uses eval-based validation
Test Change Classification	ADDITIVE	5 eval assertions added across all eval blocks, none removed
Verification Commands	N/A	No verification commands specified

Overall: WARN

The PR correctly implements the convention applicability rules document and all acceptance criteria are structurally satisfied. However, the eval results review reports two categories of eval failures:

1. Convention applicability format (eval 4 assertion 8):
The skill produces prose-format assessments ("Applicable — this task creates the model layer") instead of the prescribed format ("Applies: task modifies <file> matching the convention's <scope>"), and lists inapplicable conventions with "Not applicable" annotations instead of excluding them entirely.

→ Sub-task: TC-4554 — Strengthen convention applicability format to pass adversarial eval

2. Description digest protocol (evals 1, 4, 5 — assertions 12, 7, 11):
The skill uses placeholder values (<hex>, sha256:a1b2c3d4e5f6, sha256:placeholder) instead of computing real 64-character SHA-256 hex digests. Eval 4 also includes extraneous text in the marker line.

→ Sub-task: TC-4555 — Fix description digest protocol to compute real SHA-256 hashes in evals

Revises the previous reports which did not fully account for the eval results review.

---

This comment was AI-generated by sdlc-workflow/verify-pr v0.9.1.

[ruromero]

Verification Report for TC-4285 (commit 1f7c477)

Check	Result	Details
Review Feedback	N/A	No inline review comments on the PR
Root-Cause Investigation	N/A	No sub-tasks created
Scope Containment	WARN	2 out-of-scope files (CONVENTIONS.md, description-digest-protocol.md) from sub-task commits (TC-4554, TC-4555)
Diff Size	PASS	177 insertions across 5 files — proportionate to task scope
Commit Traceability	WARN	2/4 commits reference TC-4285; 2 reference sub-task IDs (TC-4554, TC-4555)
Sensitive Patterns	PASS	No secrets or credentials detected
CI Status	PASS	All 4 checks pass (Eval PR Run, Plugin Validation, Sourcery, Trigger Eval Dispatch)
Acceptance Criteria	PASS	4/4 criteria met
Test Quality	N/A	No test files in PR (documentation repo)
Test Change Classification	N/A	No test files in PR
Verification Commands	N/A	None specified

Overall: WARN

Scope Containment and Commit Traceability are WARN because sub-tasks TC-4554 and TC-4555 added commits to the same PR branch via Target PR flow. The out-of-scope files (CONVENTIONS.md — 1 line added to shared modules list; description-digest-protocol.md — 18 lines added for "Common Mistakes" section) and their commit references are expected artifacts of the sub-task workflow, not unintended scope creep.

---

This comment was AI-generated by sdlc-workflow/verify-pr v0.9.1.

[mrizzi]

@ruromero thanks for the contribution 👍