Stop outdated CLIs before submissions create; report upgrades over MCP

[arav-agarwal2]

Summary

Companion to mlcommons/mlperf-endpoints-api#34. That PR makes POST /submissions return 426 to any CLI older than the latest PyPI release. This PR makes the CLI handle that well.

• Version check before submissions create. Before downloading any run, the command asks PyPI for the latest release (outdated(fresh=True)).

  • If a newer release exists, it exits 1 with the upgrade command. Without this, an outdated user would sit through the downloads and the checker only to be rejected at the end.
  • --dry-run only warns, so the checker can still be run on an old CLI.
  • If PyPI can't be reached, it falls back to the 24-hour cache. If that's empty too, it carries on and leaves the decision to the API.

• Clean 426 message. _http._raise_status shows the API's detail (the required version and upgrade command) instead of API error 426: {"detail": …}.

• MCP. The server used to strip the CLI's upgrade notice from tool output. Now it reports it:

  • in the server instructions, checked once at startup;
  • in an upgrade field on text-tool results;
  • appended to the CliError message.

  The text tells the model that create_submission will be rejected, and how to upgrade and restart the server (pip, or --refresh for uvx).

• _version_check:

  • new public outdated() and UPGRADE_COMMAND;
  • _is_newer now compares every release component, not just the first three, so 1.4.0.1 counts as newer than 1.4.0.0. That matches the API's packaging.version comparison. Dev builds (1.4.1.dev3+g…) still count as ahead of their base release.

Testing

• New test_version_check.py covers:
  • version comparison;
  • outdated() (cache, fresh, fallback, expiry);
  • the exit notice: an older install is told to upgrade, from PyPI or from the cache; a current install isn't; nothing is shown when PyPI can't be reached.
• Other tests:
  • create command: an outdated CLI exits before downloading; the check asks PyPI fresh; a dry run continues;
  • _raise_status with a 426;
  • the MCP upgrade field, the failure message and the instructions.
• A suite-wide fixture keeps tests off PyPI and out of ~/.cache.
• pytest: 1707 passed, 1 failed. The failure is test_builder.py::TestRunArchiveLayouts::test_a_separate_accuracy_run_still_wins, which fails on main too. mypy src is clean.
• Manual: installed this branch as 1.3.9.0 against the live PyPI (latest 1.4.0.0):
  • submissions create stops before downloading;
  • --dry-run warns and continues;
  • the MCP instructions include the upgrade message.

🤖 Generated with Claude Code

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅