Skip to content

feat(federation): add rocm-doctor from ROCm/rocm-cli - #259

Merged
danielholanda merged 6 commits into
mainfrom
federate/rocm-doctor
Oct 9, 2026
Merged

danielholanda merged 6 commits into
mainfrom
federate/rocm-doctor

Conversation

@volen-silo

@volen-silo volen-silo commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Adds rocm-doctor to the catalog. Product repo approval is recorded in .github/skill_owners.json (ROCm/rocm-cli, engineering owner and product release owner both signed off).

The skill

Diagnoses why ROCm, the HIP SDK, PyTorch or llama.cpp is broken on an AMD GPU across Linux, Windows and WSL2, against a closed catalog of 26 known misconfigurations, then either applies a low-risk fix with consent or hands back the exact next step. It also routes Lemonade, LM Studio and Ollama problems to the right upstream channel rather than guessing.

It is a thin driver over the rocm CLI — rocm examine / rocm diagnose / rocm fix. The probe, the catalog and the fixes live in the binary and are versioned with it, so the skill does not carry a second copy of the knowledge that can drift.

Product status is Tech Preview.

What I verified before opening this

Ran the catalog's own tooling against a real import of this skill:

federate_skills.py --only rocm-doctor vendors SKILL.md, reference.md, skill-card.md, evals/evals.json
check.sh 0 error(s), checked against 11 skills
skillscope structural 1 skill folder in shape; 12 cases (7 ours + 5 shared negatives)
internal references 7 across 3 markdown files, all resolve
external links 9 URLs fetched over the network, all answered
--skill-files skill-card.md --skill-sections Description,Owner,License passes
publish.sh regenerated, zero diff; .claude-plugin/marketplace.json untouched

Against the intake checklist: name is lowercase-hyphenated, matches the folder, unique in the catalog, no forbidden substrings. Description is 962 characters. SKILL.md is 208 lines with reference material in a sibling file one level deep. All paths use forward slashes. Prerequisites state explicitly that no fixed ROCm version, gfx target or container image is assumed — the CLI detects each — and warn against hand-setting HSA_OVERRIDE_GFX_VERSION.

Evals carry 5 positive, 2 negative, and 4 positives with behavioural expectations, against minimums of 3 / 2 / 2.

One checklist item I have not met

Routing and behavioural have not been run in ROCm/rocm-cli. Both need a model key that repo does not hold — I opened a PR there to wire up the reusable workflow and closed it again, because at v0.1.3 it takes no api_base_url or api_custom_headers and so cannot reach the AMD gateway at all. A follow-up is going into that repo's existing CI job, which calls the skillscope action directly, once ORCHESTR_API_KEY is provisioned there.

So this PR's routing and behavioural legs will be their first real run. Flagging it rather than leaving you to find out — if you would rather wait until they are green on our side first, say so and I will hold this until the secret lands.

Not included

No walkthrough yet. It is listed as recommended rather than required, and I would rather add one that reflects a real session than ship a sketch.

Diagnoses why ROCm, the HIP SDK, PyTorch or llama.cpp is broken on an AMD GPU
across Linux, Windows and WSL2, against a closed catalog of 25 known
misconfigurations, and either applies a low-risk fix with consent or hands
back the exact next step. The probe, catalog and fixes live in the rocm CLI
and are versioned with the binary; the skill is a driver over it.

Product repo approval is recorded in .github/skill_owners.json.

Signed-off-by: Eugene Volen <Eugene.Volen@amd.com>
Picks up the WSL2 eval reword from ROCm/rocm-cli#561, which drops an
expectation a Windows grading host cannot satisfy: there is no WSL2 guest
there for `rocm diagnose` to report on.

Also carries upstream movement since the first import: the catalog grew to
26 entries with fix-16-vllm-oom, which fills what was a reserved handle.

Signed-off-by: Eugene Volen <Eugene.Volen@amd.com>
@volen-silo

Copy link
Copy Markdown
Collaborator Author

Re-imported at 70d6aa1, which carries the WSL2 eval reword — thanks for catching it.

I went with the reword rather than machine.yml, for the reason you half-suspected: it is per skill, and this one ships real Windows support (fix-13-hip-sdk-missing, fix-14-adrenalin-too-old, fix-15-msvc-redist). Taking all five positives off Windows to fix one case about a different platform would stop grading the platform we actually built for.

Both expected_behavior entries went, not just the unsatisfiable one. The second — "keep to the entries the CLI returns" — was already asserted negatively by unexpected_behavior's "offer a bare-metal Linux remediation the CLI did not return", so it was the same claim twice and nothing is lost by dropping it. logs_contain: ["rocm --version"] keeps the Phase 0 probe graded, the way rocm-permission-denied-kfd already handles a runner with no GPU.

The import also picked up upstream movement since the first one: the catalog is now 26 entries, fix-16-vllm-oom having filled what was a reserved handle. PR description updated to match.

check.sh passes, 0 errors; publish.sh produces no manifest churn.

On #260 — I would merge them separately, this one first. It is already shaped right as a dependent draft, and the two changes have different blast radii: this adds a federated skill, that deletes a staged draft and rewrites a README row. Keeping them apart means either can be reverted without the other. Your point about the copies having diverged is right and worth doing, just after rather than inside.

@johnl-amd johnl-amd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@danielholanda danielholanda left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally and works as expected. The skill is also tech preview on rocm-cli, which makes it eligible here. Happy to approve!

@danielholanda danielholanda left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor change requested before merging: The evals look flaky. rocm-under-wsl2 failed this run (log) because the judge counted "offer to install the rocm CLI" as a non-CLI remediation, even though its own reasoning says that's not what the check is about.

Can we fix this before merging? Either make the skill behave more consistently or reword the expectation so it's clear what counts as a failure. Thanks!

Picks up ROCm/rocm-cli#596, which rewords two judged eval items so that
installing the rocm CLI with consent, as Phase 0 of the skill instructs,
no longer reads as a remediation the CLI did not return.

Signed-off-by: Eugene Volen <Eugene.Volen@amd.com>
@volen-silo
volen-silo force-pushed the federate/rocm-doctor branch from 6346a16 to 39da6f1 Compare October 8, 2026 06:42
@volen-silo

Copy link
Copy Markdown
Collaborator Author

@danielholanda fixed by rewording the expectation, which was the second option you offered. Re-imported at 28f9ff7 (ROCm/rocm-cli#596).

Cause. The case failed on wording, not on behaviour. The check said "a remediation the CLI did not return", and installing the rocm CLI with consent matches that phrase, even though Phase 0 of the skill tells the agent to do exactly that. Across the six earlier runs, the other four cases passed every time. rocm-under-wsl2 failed twice, both on Windows, each time on a judged phrase that a correct install could match.

Change. The item now names the class of remedy it is about: a driver or permissions change such as loading amdgpu or joining the render group. It also says in the item that the CLI install is not one, because the judge reads the item rather than the case note. rocm-nothing-established-routes-upstream had the same shape ("a remediation when the CLI established no cause"), so it is scoped the same way. No check is loosened: recommending amdgpu or the render group still fails both cases.

Results on this head. One green run proves little against a 2-in-6 failure rate, so I re-ran the Windows leg until there were three results:

leg cases
Linux 5/5
Windows, run 1 5/5
Windows, run 2 5/5
Windows, run 3 5/5

That is 20 of 20 case runs, and rocm-under-wsl2 passes all its checks on every Windows run.

Picks up ROCm/rocm-cli#611. The `rocm-nothing-established-routes-upstream`
case asked the agent for the exact error text, although its prompt says the
user cannot paste one and `rocm diagnose` probes the host without
`--symptom`. The judge passed that behaviour on some runs and failed it on
others. The item now grades the guard it stood for, and the `has_match` rule
moves into the existing no-remediation item.

Eval data only; the skill itself is unchanged.

Signed-off-by: Eugene Volen <Eugene.Volen@amd.com>
@danielholanda
danielholanda merged commit f6febe7 into main Oct 9, 2026
33 of 34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants