[chrishayuk]

A GPU decode step that fails mid-generation is an error

What was wrong

The fused GPU decode loop has two exits for a step that went wrong: the backend returned nothing for the step, or the step produced no candidate tokens (which is how the backend reports a failed command buffer). Both did break, and the loop returned the tokens it had. generate_streaming then built the GenerateResult with error: None. A generation cut short by a GPU fault came back as a complete, successful one, just shorter.

What changes

DecodeLoopOutcome carries error: Option<GenerateError>, set at both exits with the step number and how many tokens the loop had produced. generate_streaming puts it on the result, alongside the tokens generated before the failure. A loop that ends at max_tokens or an end-of-sequence token still has no error.

MockGpuBackend::failing_after(steps) makes the mock's decode_token answer None after a number of successful steps, so this path is testable.

Evidence

a_gpu_decode_step_that_fails_mid_generation_is_reported runs the public generate with the mock: a backend that never fails produces max_tokens tokens and no error (the control); one that fails after two steps produces fewer tokens and must carry an error. With main's decode loop and caller (df20de1e) and the new mock, the test fails at "a generation cut short by a failed step must carry an error". With the fix it passes, as does a second test on the loop directly.

Full larql-inference suite: 1,974 passed, 0 failed. cargo fmt --check and cargo clippy -p larql-inference --lib --tests --benches --no-deps -D warnings clean.

Not covered

• Tested with the mock backend only; I did not induce a real Metal command-buffer failure.
• How the server turns this error into a response mid-stream (after tokens have been sent) was not changed or tested here. Before this change the server could not tell at all.
• The CPU fallback's equivalent is in PR 795.