Repository navigation
ci: check agent skills against the Agent Skills specification - #15
Merged
Merged
Conversation
The Skills workflow validates every skill in .claude/skills with coolapso/agent-skills-validator in one job, on pull requests that change them. The action is pinned to v0.2.0's commit, and names v0.2.0 as the validator to install, since an action pinned to a commit installs the newest one otherwise. It finds the release skill's frontmatter invalid: the description is unquoted and holds a colon and a space, which YAML doesn't allow there. It is quoted now, with the same text.
This was referenced Oct 18, 2026
kalaspuff
approved these changes
Oct 19, 2026
kalaspuff
deleted the
fctl/50-make-sure-that-this-gha-is-added-to-fctl
branch
October 19, 2026 00:12
kalaspuff
pushed a commit
to hyperstructure/sandbox
that referenced
this pull request
Oct 19, 2026
The skills in `.claude/skills` had nothing checking them against the [Agent Skills specification](https://agentskills.io/specification). A new **Skills** workflow (`.github/workflows/skills.yml`) validates them with [coolapso/agent-skills-validator](https://github.com/coolapso/agent-skills-validator), on pull requests that change the skills or the workflow. - **One job for every skill.** It sets `matrix: false` and `path: .claude/skills/*/`, so the action installs the validator once and validates every skill in one call, rather than a job per skill that each installs it. With an exact version to install, it makes no GitHub API calls either. - **Pinned to the release's commit**: `coolapso/agent-skills-validator@92cb02f7838e7d76e9f79f0558802d72927db6a9 # v0.2.0`, the commit that `v0.2.0` tags. Pinned to a commit, the action can't tell which release it is and installs the *latest* validator, so the workflow also sets `version: v0.2.0`. A comment above the step says to change the two together. - **Strict.** The specification's recommendations count as errors too; today that is only AS010, a `SKILL.md` over 500 lines. - **Pull requests only, read-only.** Skills reach `main` only through pull requests, so a run after the merge would add nothing. The job has `contents: read` and no secrets. - **Not a required check.** It doesn't run on pull requests that leave the skills alone, which a required check would wait on forever. The workflow's header says so. AGENTS.md's table of files and the commit skill's list of checks mention it. Dependabot's monthly pull request for the actions moves the commit and its `# v…` comment, but not `version:`. When it moves this action, move `version` in the same pull request. ## Verified - The v0.2.0 validator (its linux/arm64 build, checked against the release's `checksums.txt`) passes all three skills here in strict mode: `3 skills validated, 3 valid, 0 invalid (0 errors, 0 warnings)`. The action's own validation step, run locally with these inputs, validates all three in one call: `3 skills validated, 3 valid, 0 invalid (0 errors, 0 warnings)`. - actionlint 1.7.12 (the Lint workflow's) reports nothing for any workflow, and ShellCheck passes `provision.sh` and `check.sh`. The Commits workflow's title script accepts the title. - This pull request runs the check itself, as **Skills / Agent Skills specification** ([its run](https://github.com/hyperstructure/sandbox/actions/runs/39177640636)): one job installs v0.2.0 (`Resolved CLI version request: v0.2.0 (action ref: 92cb02f7838e7d76e9f79f0558802d72927db6a9)`, checksum verified) and validates all three skills in one call: `3 skills validated, 3 valid, 0 invalid (0 errors, 0 warnings)`. It doesn't touch the image, so the Image workflow doesn't run. The v0.2.0 release isn't an immutable GitHub release. The action checks the binary against the release's own `checksums.txt`, so the commit pin fixes the action's code, but the binary is only as fixed as the release's files. With no secrets and a read-only token, there is little a replaced binary could reach. The same workflow, in each repository with skills: - hyperstructure/fctl#102 - hyperstructure/design#15 - hyperstructure/fctl-site#21 - #7 Session: https://app.fctl.sh/s/ses_01M58NR0YMZ8D5F6JMNPHGF1J1 Co-authored-by: fctl-sh[bot] <343971779+fctl-sh[bot]@users.noreply.github.com>
kalaspuff
pushed a commit
to hyperstructure/fctl
that referenced
this pull request
Oct 19, 2026
The skills in `.claude/skills` had nothing checking them against the [Agent Skills specification](https://agentskills.io/specification). A new **Skills** workflow (`.github/workflows/skills.yml`) validates them with [coolapso/agent-skills-validator](https://github.com/coolapso/agent-skills-validator), on pull requests that change the skills or the workflow. - **One job for every skill.** It sets `matrix: false` and `path: .claude/skills/*/`, so the action installs the validator once and validates every skill in one call, rather than a job per skill that each installs it. With an exact version to install, it makes no GitHub API calls either. - **Pinned to the release's commit**: `coolapso/agent-skills-validator@92cb02f7838e7d76e9f79f0558802d72927db6a9 # v0.2.0`, the commit that `v0.2.0` tags. Pinned to a commit, the action can't tell which release it is and installs the *latest* validator, so the workflow also sets `version: v0.2.0`. A comment above the step says to change the two together. - **Strict.** The specification's recommendations count as errors too; today that is only AS010, a `SKILL.md` over 500 lines. - **Pull requests only, read-only.** Skills reach `main` only through pull requests, so a run after the merge would add nothing. The job has `contents: read` and no secrets. - **Not a required check.** It doesn't run on pull requests that leave the skills alone, which a required check would wait on forever. The workflow's header says so. The pull-requests skill and the Checks section of `docs/development.md` mention the new check. ## Verified - The v0.2.0 validator (its linux/arm64 build, checked against the release's `checksums.txt`) passes all five skills here in strict mode: `5 skills validated, 5 valid, 0 invalid (0 errors, 0 warnings)`. The action's own validation step, run locally with these inputs, expands the glob to all five and validates them in one call: `5 skills validated, 5 valid, 0 invalid (0 errors, 0 warnings)`. - actionlint 1.7.12 reports nothing for any workflow. - `cargo xtask check --quick` passes, and `cargo xtask lint-commit` accepts the title. - This pull request runs the check itself, as **Skills / Agent Skills specification** ([its run](https://github.com/hyperstructure/fctl/actions/runs/39177634519)): one job installs v0.2.0 (`Resolved CLI version request: v0.2.0 (action ref: 92cb02f7838e7d76e9f79f0558802d72927db6a9)`, checksum verified) and validates all five skills in one call: `5 skills validated, 5 valid, 0 invalid (0 errors, 0 warnings)`. The v0.2.0 release isn't an immutable GitHub release. The action checks the binary against the release's own `checksums.txt`, so the commit pin fixes the action's code, but the binary is only as fixed as the release's files. With no secrets and a read-only token, there is little a replaced binary could reach. The same workflow, in each repository with skills: - #102 - hyperstructure/design#15 - hyperstructure/fctl-site#21 - hyperstructure/sandbox#7 Session: https://app.fctl.sh/s/ses_01M58NR0YMZ8D5F6JMNPHGF1J1 Co-authored-by: fctl-sh[bot] <343971779+fctl-sh[bot]@users.noreply.github.com>
kalaspuff
pushed a commit
to hyperstructure/fctl-site
that referenced
this pull request
Oct 19, 2026
The skills in `.claude/skills` had nothing checking them against the [Agent Skills specification](https://agentskills.io/specification). A new **Skills** workflow (`.github/workflows/skills.yml`) validates them with [coolapso/agent-skills-validator](https://github.com/coolapso/agent-skills-validator), on pull requests that change the skills or the workflow. - **One job for every skill.** It sets `matrix: false` and `path: .claude/skills/*/`, so the action installs the validator once and validates every skill in one call, rather than a job per skill that each installs it. With an exact version to install, it makes no GitHub API calls either. - **Pinned to the release's commit**: `coolapso/agent-skills-validator@92cb02f7838e7d76e9f79f0558802d72927db6a9 # v0.2.0`, the commit that `v0.2.0` tags. Pinned to a commit, the action can't tell which release it is and installs the *latest* validator, so the workflow also sets `version: v0.2.0`. A comment above the step says to change the two together. - **Strict.** The specification's recommendations count as errors too; today that is only AS010, a `SKILL.md` over 500 lines. - **Pull requests only, read-only.** Skills reach `main` only through pull requests, so a run after the merge would add nothing. The job has `contents: read` and no secrets. - **Not a required check.** It doesn't run on pull requests that leave the skills alone, which a required check would wait on forever. The workflow's header says so. The pull-requests skill mentions the new check. ## Verified - The v0.2.0 validator (its linux/arm64 build, checked against the release's `checksums.txt`) passes all four skills here in strict mode: `4 skills validated, 4 valid, 0 invalid (0 errors, 0 warnings)`. The action's own validation step, run locally with these inputs, validates all four in one call: `4 skills validated, 4 valid, 0 invalid (0 errors, 0 warnings)`. - actionlint 1.7.12 reports nothing for any workflow. - `npm run check` passes (with the two `noImportantStyles` warnings in `404.html` and `500.html` that `main` has too) and so does `npm test`. `node scripts/lint-commit.mjs` accepts the title. - This pull request runs the check itself, as **Skills / Agent Skills specification** ([its run](https://github.com/hyperstructure/fctl-site/actions/runs/39177638451)): one job installs v0.2.0 (`Resolved CLI version request: v0.2.0 (action ref: 92cb02f7838e7d76e9f79f0558802d72927db6a9)`, checksum verified) and validates all four skills in one call: `4 skills validated, 4 valid, 0 invalid (0 errors, 0 warnings)`. The v0.2.0 release isn't an immutable GitHub release. The action checks the binary against the release's own `checksums.txt`, so the commit pin fixes the action's code, but the binary is only as fixed as the release's files. With no secrets and a read-only token, there is little a replaced binary could reach. The same workflow, in each repository with skills: - hyperstructure/fctl#102 - hyperstructure/design#15 - #21 - hyperstructure/sandbox#7 Session: https://app.fctl.sh/s/ses_01M58NR0YMZ8D5F6JMNPHGF1J1 Co-authored-by: fctl-sh[bot] <343971779+fctl-sh[bot]@users.noreply.github.com>
kalaspuff
pushed a commit
to hyperstructure/fctl
that referenced
this pull request
Oct 20, 2026
) Sessions showed one pull request: the one fctl-agent reported last, the primary repository's first, as `#120` with no repository. A session that opened pull requests in several of its project's repositories, or a second one after the first was merged, listed only one of them and didn't say where it was. ## What changed - **The control plane keeps every pull request a session's agent finds.** After each turn fctl-agent reports the pull request of the branch each repository has checked out, so it only ever reports the latest one per repository. SessionDO now adds the new ones to the session's list, in the order they come, and takes the newer title and state of the ones it has (`fctl_domain::pulls`, at most 20). Each carries the repository it was opened in: the one its URL names, or else the repository whose state it came in. The list is written to D1 in a new `sessions.pull_requests` column (migration 0014) and is part of the account export. - **Protocol.** `Session.pull_requests` and `PullRequestRef.repo` are new and optional; `Session.pull_request` stays for clients a version behind. Agents and hosts don't change. Sessions from before this get their one pull request in the list, with its repository, when they are read, from D1 or from their SessionDO. - **Web app.** The session header links every pull request as `repository#number` (`fctl#120`, `design#14`), with its title on hover; they wrap together as one list. The rail has a **Pull requests** section with each one's title and `owner/name#number`, in place of the card under **Changes**. The floor, a workspace's page and a project's page say `PRs fctl#120, design#14`. - **The floor stays current.** SessionDO told the dashboards about a session when its status changed, and a turn's git report arrives after the turn ends. So a new pull request, and the +/− counts, reached the floor's rows only after a reload or the next turn. A git report that changes what the lists show now goes to the dashboards too. - **CLI.** `fctl show` prints each pull request with its repository (below); `--json` has the list. - **Docs.** `getting-started.md` and `cli.md` say so. The privacy policy's "branch and pull request" becomes "pull requests" in hyperstructure/fctl-site#28. ``` $ fctl show playground#3 playground#3 Name every pull request a session opens … prs: fctl#121 #121 design#15 hyperstructure/design#15 fctl-site#32 hyperstructure/fctl-site#32 ``` ## Verification - Unit tests for `fctl_domain::pulls` (repositories from URLs and from repository states, merging over turns, the limit, sessions from before the list), for `fctl show`'s lines (also from a control plane before the list) and for the web app's helpers. `cargo xtask check` passes. - End to end with `cargo xtask dev`. The dev instance has no GitHub, so the dev agents got a stand-in `gh` that answers `gh pr view` with a pull request in another repository after each turn, and the project a setup script that pushes each commit to a local remote, so fctl-agent sees its branch pushed and looks the pull request up. - On `main`, a session whose agent found fctl#120, design#14 and fctl-site#31 showed only `#31`. - I restarted the stack on this branch with the same data, and migration 0014 applied. The sessions `main` made read with their one pull request in the list, with its repository. Another turn in one of them added sandbox#7 after fctl-site#31. - A new session's agent found fctl#121, design#15 and fctl-site#32: D1, the API, `fctl show` and the web app list all three. With the session page and the floor open, the pull request a further turn found showed up on both without a reload. ## Screenshots At b04cf34, with the dev stack's data. Both columns show the same sessions, read from this branch's control plane: on the left `main`'s web app, on the right this branch's. Playground #3's agent found a pull request in four repositories. | Before | After | |---|---| |  |  | |  |  | |  |  | |  |  | |  |  | Made in fctl session https://app.fctl.sh/s/ses_01M5CBQV867BG6ARZKJHZPGVY0 --------- Co-authored-by: fctl-sh[bot] <fctl-sh[bot]@users.noreply.github.com> Co-authored-by: fctl-sh[bot] <343971779+fctl-sh[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The skills in
.claude/skillshad nothing checking them against the Agent Skills specification. A new Skills workflow (.github/workflows/skills.yml) validates them with coolapso/agent-skills-validator, on pull requests that change the skills or the workflow.matrix: falseandpath: .claude/skills/*/, so the action installs the validator once and validates every skill in one call, rather than a job per skill that each installs it. With an exact version to install, it makes no GitHub API calls either.coolapso/agent-skills-validator@92cb02f7838e7d76e9f79f0558802d72927db6a9 # v0.2.0, the commit thatv0.2.0tags. Pinned to a commit, the action can't tell which release it is and installs the latest validator, so the workflow also setsversion: v0.2.0. A comment above the step says to change the two together.SKILL.mdover 500 lines.mainonly through pull requests, so a run after the merge would add nothing. The job hascontents: readand no secrets.It also fixes the one skill it rejects. The release skill's frontmatter didn't parse:
.claude/skills/release/SKILL.md:3: error AS002: frontmatter is not valid YAML: mapping values are not allowed in this context. Its unquoted description holdspin): by, and YAML doesn't allow a colon followed by a space in an unquoted value. The description is quoted now, with the same text. That is how Anthropic's own skills (docx,pptxandxlsxin anthropics/skills) write a description that contains a colon and a space. The rest of the skill already follows the specification's and Anthropic's authoring guidance: its description says what it does and when to use it, and it is 71 lines long. The pull-requests skill mentions the new check.The title is
ci:, so merging this doesn't release the package.Verified
checksums.txt) in strict mode:4 skills validated, 3 valid, 1 invalid (1 error, 0 warnings)onmain, and4 skills validated, 4 valid, 0 invalid (0 errors, 0 warnings)with the fix. The quoted description is the same string as before: it holds no"or\. The action's own validation step, run locally with these inputs, validates all four skills in one call and fails onmain.npm run checkandnpm testpass, andnode scripts/lint-commit.mjsaccepts the title.Resolved CLI version request: v0.2.0 (action ref: 92cb02f7838e7d76e9f79f0558802d72927db6a9), checksum verified) and validates all four skills in one call:4 skills validated, 4 valid, 0 invalid (0 errors, 0 warnings).The v0.2.0 release isn't an immutable GitHub release. The action checks the binary against the release's own
checksums.txt, so the commit pin fixes the action's code, but the binary is only as fixed as the release's files. With no secrets and a read-only token, there is little a replaced binary could reach.The same workflow, in each repository with skills:
Session: https://app.fctl.sh/s/ses_01M58NR0YMZ8D5F6JMNPHGF1J1