Repository navigation
ci: check agent skills against the Agent Skills specification - #7
Merged
Merged
Conversation
The Skills workflow validates every skill in .claude/skills with coolapso/agent-skills-validator in one job, on pull requests that change them. The action is pinned to v0.2.0's commit, and names v0.2.0 as the validator to install, since an action pinned to a commit installs the newest one otherwise. Dependabot moves the commit but not the version, so its pull requests for this action need the version moved by hand.
This was referenced Oct 18, 2026
kalaspuff
pushed a commit
to hyperstructure/design
that referenced
this pull request
Oct 19, 2026
The skills in `.claude/skills` had nothing checking them against the [Agent Skills specification](https://agentskills.io/specification). A new **Skills** workflow (`.github/workflows/skills.yml`) validates them with [coolapso/agent-skills-validator](https://github.com/coolapso/agent-skills-validator), on pull requests that change the skills or the workflow. - **One job for every skill.** It sets `matrix: false` and `path: .claude/skills/*/`, so the action installs the validator once and validates every skill in one call, rather than a job per skill that each installs it. With an exact version to install, it makes no GitHub API calls either. - **Pinned to the release's commit**: `coolapso/agent-skills-validator@92cb02f7838e7d76e9f79f0558802d72927db6a9 # v0.2.0`, the commit that `v0.2.0` tags. Pinned to a commit, the action can't tell which release it is and installs the *latest* validator, so the workflow also sets `version: v0.2.0`. A comment above the step says to change the two together. - **Strict.** The specification's recommendations count as errors too; today that is only AS010, a `SKILL.md` over 500 lines. - **Pull requests only, read-only.** Skills reach `main` only through pull requests, so a run after the merge would add nothing. The job has `contents: read` and no secrets. - **Not a required check.** It doesn't run on pull requests that leave the skills alone, which a required check would wait on forever. The workflow's header says so. It also fixes the one skill it rejects. The release skill's frontmatter didn't parse: `.claude/skills/release/SKILL.md:3: error AS002: frontmatter is not valid YAML: mapping values are not allowed in this context`. Its unquoted description holds `pin): by`, and YAML doesn't allow a colon followed by a space in an unquoted value. The description is quoted now, with the same text. That is how Anthropic's own skills (`docx`, `pptx` and `xlsx` in anthropics/skills) write a description that contains a colon and a space. The rest of the skill already follows the specification's and Anthropic's authoring guidance: its description says what it does and when to use it, and it is 71 lines long. The pull-requests skill mentions the new check. The title is `ci:`, so merging this doesn't release the package. ## Verified - The v0.2.0 validator (its linux/arm64 build, checked against the release's `checksums.txt`) in strict mode: `4 skills validated, 3 valid, 1 invalid (1 error, 0 warnings)` on `main`, and `4 skills validated, 4 valid, 0 invalid (0 errors, 0 warnings)` with the fix. The quoted description is the same string as before: it holds no `"` or `\`. The action's own validation step, run locally with these inputs, validates all four skills in one call and fails on `main`. - actionlint 1.7.12 reports nothing for any workflow. - `npm run check` and `npm test` pass, and `node scripts/lint-commit.mjs` accepts the title. - This pull request runs the check itself, as **Skills / Agent Skills specification** ([its run](https://github.com/hyperstructure/design/actions/runs/39177636859)): one job installs v0.2.0 (`Resolved CLI version request: v0.2.0 (action ref: 92cb02f7838e7d76e9f79f0558802d72927db6a9)`, checksum verified) and validates all four skills in one call: `4 skills validated, 4 valid, 0 invalid (0 errors, 0 warnings)`. The v0.2.0 release isn't an immutable GitHub release. The action checks the binary against the release's own `checksums.txt`, so the commit pin fixes the action's code, but the binary is only as fixed as the release's files. With no secrets and a read-only token, there is little a replaced binary could reach. The same workflow, in each repository with skills: - hyperstructure/fctl#102 - #15 - hyperstructure/fctl-site#21 - hyperstructure/sandbox#7 Session: https://app.fctl.sh/s/ses_01M58NR0YMZ8D5F6JMNPHGF1J1 Co-authored-by: fctl-sh[bot] <343971779+fctl-sh[bot]@users.noreply.github.com>
kalaspuff
approved these changes
Oct 19, 2026
kalaspuff
deleted the
fctl/50-make-sure-that-this-gha-is-added-to-fctl
branch
October 19, 2026 00:13
kalaspuff
pushed a commit
to hyperstructure/fctl
that referenced
this pull request
Oct 19, 2026
The skills in `.claude/skills` had nothing checking them against the [Agent Skills specification](https://agentskills.io/specification). A new **Skills** workflow (`.github/workflows/skills.yml`) validates them with [coolapso/agent-skills-validator](https://github.com/coolapso/agent-skills-validator), on pull requests that change the skills or the workflow. - **One job for every skill.** It sets `matrix: false` and `path: .claude/skills/*/`, so the action installs the validator once and validates every skill in one call, rather than a job per skill that each installs it. With an exact version to install, it makes no GitHub API calls either. - **Pinned to the release's commit**: `coolapso/agent-skills-validator@92cb02f7838e7d76e9f79f0558802d72927db6a9 # v0.2.0`, the commit that `v0.2.0` tags. Pinned to a commit, the action can't tell which release it is and installs the *latest* validator, so the workflow also sets `version: v0.2.0`. A comment above the step says to change the two together. - **Strict.** The specification's recommendations count as errors too; today that is only AS010, a `SKILL.md` over 500 lines. - **Pull requests only, read-only.** Skills reach `main` only through pull requests, so a run after the merge would add nothing. The job has `contents: read` and no secrets. - **Not a required check.** It doesn't run on pull requests that leave the skills alone, which a required check would wait on forever. The workflow's header says so. The pull-requests skill and the Checks section of `docs/development.md` mention the new check. ## Verified - The v0.2.0 validator (its linux/arm64 build, checked against the release's `checksums.txt`) passes all five skills here in strict mode: `5 skills validated, 5 valid, 0 invalid (0 errors, 0 warnings)`. The action's own validation step, run locally with these inputs, expands the glob to all five and validates them in one call: `5 skills validated, 5 valid, 0 invalid (0 errors, 0 warnings)`. - actionlint 1.7.12 reports nothing for any workflow. - `cargo xtask check --quick` passes, and `cargo xtask lint-commit` accepts the title. - This pull request runs the check itself, as **Skills / Agent Skills specification** ([its run](https://github.com/hyperstructure/fctl/actions/runs/39177634519)): one job installs v0.2.0 (`Resolved CLI version request: v0.2.0 (action ref: 92cb02f7838e7d76e9f79f0558802d72927db6a9)`, checksum verified) and validates all five skills in one call: `5 skills validated, 5 valid, 0 invalid (0 errors, 0 warnings)`. The v0.2.0 release isn't an immutable GitHub release. The action checks the binary against the release's own `checksums.txt`, so the commit pin fixes the action's code, but the binary is only as fixed as the release's files. With no secrets and a read-only token, there is little a replaced binary could reach. The same workflow, in each repository with skills: - #102 - hyperstructure/design#15 - hyperstructure/fctl-site#21 - hyperstructure/sandbox#7 Session: https://app.fctl.sh/s/ses_01M58NR0YMZ8D5F6JMNPHGF1J1 Co-authored-by: fctl-sh[bot] <343971779+fctl-sh[bot]@users.noreply.github.com>
kalaspuff
pushed a commit
to hyperstructure/fctl-site
that referenced
this pull request
Oct 19, 2026
The skills in `.claude/skills` had nothing checking them against the [Agent Skills specification](https://agentskills.io/specification). A new **Skills** workflow (`.github/workflows/skills.yml`) validates them with [coolapso/agent-skills-validator](https://github.com/coolapso/agent-skills-validator), on pull requests that change the skills or the workflow. - **One job for every skill.** It sets `matrix: false` and `path: .claude/skills/*/`, so the action installs the validator once and validates every skill in one call, rather than a job per skill that each installs it. With an exact version to install, it makes no GitHub API calls either. - **Pinned to the release's commit**: `coolapso/agent-skills-validator@92cb02f7838e7d76e9f79f0558802d72927db6a9 # v0.2.0`, the commit that `v0.2.0` tags. Pinned to a commit, the action can't tell which release it is and installs the *latest* validator, so the workflow also sets `version: v0.2.0`. A comment above the step says to change the two together. - **Strict.** The specification's recommendations count as errors too; today that is only AS010, a `SKILL.md` over 500 lines. - **Pull requests only, read-only.** Skills reach `main` only through pull requests, so a run after the merge would add nothing. The job has `contents: read` and no secrets. - **Not a required check.** It doesn't run on pull requests that leave the skills alone, which a required check would wait on forever. The workflow's header says so. The pull-requests skill mentions the new check. ## Verified - The v0.2.0 validator (its linux/arm64 build, checked against the release's `checksums.txt`) passes all four skills here in strict mode: `4 skills validated, 4 valid, 0 invalid (0 errors, 0 warnings)`. The action's own validation step, run locally with these inputs, validates all four in one call: `4 skills validated, 4 valid, 0 invalid (0 errors, 0 warnings)`. - actionlint 1.7.12 reports nothing for any workflow. - `npm run check` passes (with the two `noImportantStyles` warnings in `404.html` and `500.html` that `main` has too) and so does `npm test`. `node scripts/lint-commit.mjs` accepts the title. - This pull request runs the check itself, as **Skills / Agent Skills specification** ([its run](https://github.com/hyperstructure/fctl-site/actions/runs/39177638451)): one job installs v0.2.0 (`Resolved CLI version request: v0.2.0 (action ref: 92cb02f7838e7d76e9f79f0558802d72927db6a9)`, checksum verified) and validates all four skills in one call: `4 skills validated, 4 valid, 0 invalid (0 errors, 0 warnings)`. The v0.2.0 release isn't an immutable GitHub release. The action checks the binary against the release's own `checksums.txt`, so the commit pin fixes the action's code, but the binary is only as fixed as the release's files. With no secrets and a read-only token, there is little a replaced binary could reach. The same workflow, in each repository with skills: - hyperstructure/fctl#102 - hyperstructure/design#15 - #21 - hyperstructure/sandbox#7 Session: https://app.fctl.sh/s/ses_01M58NR0YMZ8D5F6JMNPHGF1J1 Co-authored-by: fctl-sh[bot] <343971779+fctl-sh[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The skills in
.claude/skillshad nothing checking them against the Agent Skills specification. A new Skills workflow (.github/workflows/skills.yml) validates them with coolapso/agent-skills-validator, on pull requests that change the skills or the workflow.matrix: falseandpath: .claude/skills/*/, so the action installs the validator once and validates every skill in one call, rather than a job per skill that each installs it. With an exact version to install, it makes no GitHub API calls either.coolapso/agent-skills-validator@92cb02f7838e7d76e9f79f0558802d72927db6a9 # v0.2.0, the commit thatv0.2.0tags. Pinned to a commit, the action can't tell which release it is and installs the latest validator, so the workflow also setsversion: v0.2.0. A comment above the step says to change the two together.SKILL.mdover 500 lines.mainonly through pull requests, so a run after the merge would add nothing. The job hascontents: readand no secrets.AGENTS.md's table of files and the commit skill's list of checks mention it.
Dependabot's monthly pull request for the actions moves the commit and its
# v…comment, but notversion:. When it moves this action, moveversionin the same pull request.Verified
checksums.txt) passes all three skills here in strict mode:3 skills validated, 3 valid, 0 invalid (0 errors, 0 warnings). The action's own validation step, run locally with these inputs, validates all three in one call:3 skills validated, 3 valid, 0 invalid (0 errors, 0 warnings).provision.shandcheck.sh. The Commits workflow's title script accepts the title.Resolved CLI version request: v0.2.0 (action ref: 92cb02f7838e7d76e9f79f0558802d72927db6a9), checksum verified) and validates all three skills in one call:3 skills validated, 3 valid, 0 invalid (0 errors, 0 warnings). It doesn't touch the image, so the Image workflow doesn't run.The v0.2.0 release isn't an immutable GitHub release. The action checks the binary against the release's own
checksums.txt, so the commit pin fixes the action's code, but the binary is only as fixed as the release's files. With no secrets and a read-only token, there is little a replaced binary could reach.The same workflow, in each repository with skills:
Session: https://app.fctl.sh/s/ses_01M58NR0YMZ8D5F6JMNPHGF1J1