Stop guessing how to run local models.
LLMRig is an evidence-driven Autopilot for local AI. Give it a model and a machine to see what is known, blocked, or unknown before you approve setup and verification.
pipx install llmrig
llmrig plan mlx-community/Qwen3.5-27B-4bitRepresentative output:
LLMRig Autopilot Plan
=====================
Machine: Apple M4 Max · 48 GiB
Model: mlx-community/Qwen3.5-27B-4bit
Runtime Format Quant Local Executable
------------ ------- ------- --------- ------------
omlx MLX 4-bit unknown not_executable
mlx-lm MLX 4-bit unknown not_executable
Setup path: omlx · MLX
Reason: exactly one compatible candidate has a complete, supported
Autopilot setup path. This is a setup selection, not a performance ranking.
Planned actions
1. Acquire the selected artifact
2. Prepare the selected runtime
3. Load/register the artifact
4. Verify with LLMRig's deterministic workload
No action has been taken.
plan is read-only. If the evidence is insufficient, LLMRig stays inconclusive instead of inventing a winner.
Ready to continue? Run the workflow and approve any mutation explicitly:
llmrig run mlx-community/Qwen3.5-27B-4bitRunning models locally still involves too much guesswork:
- Large weights can be downloaded before you discover the intended setup is not viable on your machine.
- Runtime, format, context, and quantization advice is fragmented across model cards, forums, and machine-specific anecdotes.
- oMLX, Ollama, MLX-LM, and llama.cpp expose different artifacts and capabilities, so a model name alone is not an execution plan.
Ollama runs models. LLMRig sits above runtimes and determines what the current evidence supports for this model on this machine — then, with approval, it can apply the supported setup and verify what actually happened.
And when the evidence is not there, LLMRig says unknown instead of converting absence into certainty:
unknown != false
compatible != local
local != executable
executable != measurable
measurable != measured
measured != recommended
measured performance != model quality
discovery metadata != installation trust
pipx install llmrig # or: python -m pip install llmrig
llmrig solve MODEL # inspect evidence; change nothing
llmrig plan MODEL # build a deterministic read-only plan
llmrig run MODEL # approve → apply → verifysolve and plan are read-only. Downloads, runtime starts, model loads, and other mutations require explicit approval. For non-interactive automation, use --yes only when that intent is deliberate.
Planning evidence and measured performance are separate facts. Before verification, unavailable performance stays unknown.
These are real v0.9 Apple Silicon observations recorded by LLMRig's race-v2 verification workload:
| Machine | Model | Runtime | Quant | Generation tok/s | Prompt tok/s | Latency |
|---|---|---|---|---|---|---|
| Apple M4 Max · 48 GiB | mlx-community/Qwen3-0.6B-4bit |
MLX-LM | 4-bit | 560.85 | 1629.77 | 0.2639 s |
| Apple M4 Max · 48 GiB | mlx-community/Qwen3-0.6B-4bit |
MLX-LM | 4-bit | 560.23 | 1687.91 | 0.2630 s |
Both rows are genuine measured observations from the same model/runtime/hardware setup, with two measured runs per observation. The first observation predates context-token persistence, so LLMRig does not backfill the missing context value.
More privacy-safe measurements are available in the Benchmark Passports dataset. These numbers describe those exact measured executions; they are not a leaderboard or a claim about model quality.
Four commands, one principle: explicit intent at every mutation boundary.
llmrig plan mlx-community/Qwen3.5-27B-4bitProduces a deterministic plan ID covering machine identity, viable runtime/artifact candidates, required actions, blockers, and a verification step defined before anything changes. Downloads nothing, starts nothing.
If several setup paths stay valid, LLMRig stays inconclusive rather than inventing a winner.
llmrig apply mlx-community/Qwen3.5-27B-4bit --plan-id plan-... --yesLLMRig recomputes the plan immediately before applying. If the evidence no longer reproduces the approved plan ID, apply fails closed with plan drift detected. Hugging Face downloads are pinned to the exact repository revision resolved by the Hub.
llmrig verify
llmrig verify receipt-...Recomputes evidence from scratch, refuses any mutation, and runs the deterministic measurement workload. Old success records are never trusted; missing metrics stay unknown — never synthesized.
llmrig run MODEL --yesPlan → approval → apply → verify in one shortcut. It does not bypass the approval boundary.
Every applied workflow emits a privacy-safe receipt: plan ID, receipt ID, model/artifact/runtime identity, actions attempted, status, verification measurements, and timestamps. Filesystem paths, API keys, cookies, and secrets are never serialized.
Verified measurements persist locally as graph-shaped evidence across machine × model × artifact × quantization × runtime × context × measurement. Predictions and measurements are stored as separate facts; calibration deltas are computed only when the same metric exists on both sides. No community upload happens by default.
llmrig runtimes
llmrig runtimes --json| Runtime | Readiness detection | LLMRig execution / measurement | Primary artifact evidence |
|---|---|---|---|
| oMLX | Yes | Yes, for provenance-backed local models | MLX |
| Ollama | Yes | Yes | Ollama-managed artifacts |
| MLX-LM | Yes | Yes, for explicit/evidenced local artifacts | MLX |
| llama.cpp | Yes | Yes, for explicit/evidenced local artifacts | GGUF |
Runtime probing is read-only. Unsupported actions surface as explicit blockers, never silent emulation.
LLMRig resolves exact owner/repository identifiers through read-only Hugging Face Hub metadata (no weight downloads during resolution), with strict evidence rules — e.g. an mlx tag alone is not proof of MLX packaging, and conflicting context, quantization, or provenance evidence fails closed.
- LLMRig Autopilot Space — evidence-first walkthrough of the Autopilot flow
- Benchmark Passports dataset — privacy-safe RigGraph measurement samples
- LLMRig collection — Space, dataset, and referenced model grouped together
- Launch discussion — reproducible launch notes and evidence
The Space is intentionally a static explainer: it does not inspect a visitor's machine. Run LLMRig locally for real discovery, planning, execution, and verification.
| Command | Purpose |
|---|---|
solve --verify |
verify a comparable solve candidate set |
race |
compare local configurations |
choose |
explain one measured objective |
optimize |
expose an unranked noise-aware Pareto frontier |
bench |
full Ollama benchmark |
Measurement never establishes model quality, and results inside the configured 5% race threshold stay inconclusive.
import llmrig
result = llmrig.solve("mlx-community/Qwen3.5-27B-4bit")
print(result.plan.recommendation_status)llmrig.solve(
model,
*,
context=None,
local_artifacts=(),
verify=False,
) -> llmrig.SolveResultThe SDK prints nothing and never exits the process. SolveResult and SolveCandidate are the public contracts; _llmrig stays private.
pipx is recommended:
pipx install llmrig
llmrig --versionUpgrade with pipx upgrade llmrig, or use a virtual environment:
python3 -m venv .venv
source .venv/bin/activate
python -m pip install llmrigWindows PowerShell:
python -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install llmrigPython 3.9+ on macOS, Linux, and Windows. Hugging Face Hub support is included for exact, revision-pinned artifact acquisition.
LLMRig
┌───────────┐
│ RigGraph │
└─────┬─────┘
│
┌──────────┴──────────┐
│ Autopilot Engine │
└──────────┬──────────┘
│
┌──────────┬───────┼───────┬──────────┐
↓ ↓ ↓ ↓ ↓
oMLX Ollama MLX-LM llama.cpp future
The north star is Detect → Decide → Configure → Run → Verify, with explicit intent at every mutation boundary. See ROADMAP_V1.md.
CHANGELOG.md— canonical release historyROADMAP_V1.md— product directionRELEASING.md— GitHub Release and PyPI processSECURITY.md— security policy and trust boundariesCONTRIBUTING.md— contribution guide
MIT