Skip to content

Add BLADE analysis skill#1584

Closed
jmabry wants to merge 9 commits into
mainfrom
add-blade-analysis-skill
Closed

Add BLADE analysis skill#1584
jmabry wants to merge 9 commits into
mainfrom
add-blade-analysis-skill

Conversation

@jmabry

@jmabry jmabry commented Jun 11, 2026

Copy link
Copy Markdown

Summary

  • Add a new nemo-gym-blade-analysis skill for both Claude and Codex agents.
  • Document a BLADE-style rollout analysis workflow: artifact inventory, pass@k metrics, task buckets, root-cause taxonomy, evidence rules, and recommendation mapping.
  • Keep original CVDP as an opt-in reference and point to the existing in-repo CVDP example rollout artifacts under resources_servers/cvdp/data/.
  • Include only Nemotron 3 Super golden analysis artifacts as optional reference material; the large rollout JSONL is intentionally excluded.

Issue: #1585

Validation

  • Confirmed existing CVDP example rollouts are already present in resources_servers/cvdp/data/.
  • Confirmed no staged .jsonl rollout files were added.
  • Ran git diff --cached --check.
  • Parsed copied JSON artifacts with python3 -m json.tool.
  • Scanned new skill files for private benchmark names, internal URLs, credential strings, and provider route strings.
  • Ran three sub-agent review passes; fixed findings around CVDP opt-in loading and sanitized extractor model route metadata.

Note

uvx pre-commit run --files ... could not complete because GitHub returned HTTP 503 while fetching pre-commit-hooks. The relevant local checks were run manually.

@copy-pr-bot

copy-pr-bot Bot commented Jun 11, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@jmabry

jmabry commented Jun 13, 2026

Copy link
Copy Markdown
Author

closing due to signed commit issue that requires pushing a new branch

@jmabry jmabry closed this Jun 13, 2026
@jmabry

jmabry commented Jun 13, 2026

Copy link
Copy Markdown
Author

superceded by #1591

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant