[Afloat (Afloat16)]

Description

FLOAT8 weight quantization computes the input standard deviation in the weight dtype. Finite FP16/FP32 weights can therefore produce an infinite or zero scale when the intermediate squared deviations overflow or underflow. The resulting codes can collapse or contain NaNs even when the final scale is representable.

Accumulate the standard deviation in float64 for FP16/FP32 input weights, then cast the final scale back to the original weight dtype. Other direct-helper input dtypes retain NumPy's existing default statistics/result dtype. The codebook reference distribution and zero-variance policy are unchanged.

Validation

• Eight new unittest methods cover large and small nonconstant weights, independent math.fsum standard-deviation references, quantized FP8 codes, ordinary values/options, readonly strided inputs, zero variance and six other direct-helper input dtypes.
• Native CPU DequantizeLinear + Add inference consumes the saved/reloaded FP32-weight quantization result, with persisted scales and codes checked independently.
• New and existing quantization utility tests: 17 methods passed, 161 subtests passed.
• Restoring the original implementation produces 19 failing subtests, with 8 completed methods and 11 passing subtests. These include finite large/small inputs as well as FP16 rounding controls.
• Ruff 0.12.12 lint/format and git diff --check passed.

Exact-source native validation: https://github.com/Afloat16/onnxruntime/actions/runs/43406537495

The runtime was built using the repository's official CPU Release build.py/CMake process at 8dea1194e974fa07b1d2f07a190949c3a92cab80. Its C++ sources match base a8246c0d094f1ab0d6bf236228e5e67683321cd7. This Python-only validation installs that built wheel, overlays the complete quantization package from this exact commit, and verifies byte equality and hashes before normal native imports.

Motivation and Context

This repairs the direct FLOAT8 weight-statistics path. Histogram calibration, the codebook distribution, integer quantization and constant-weight policy keep their existing behavior. The tests establish numerical correctness for the covered FP16/FP32 scales; they do not measure end-to-end model accuracy or throughput.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟢 Approval recommended

No actionable findings were identified in the FLOAT8 weight-statistics stability change.

No open issues 🎉

What changed in this PR

This PR improves FLOAT8 weight quantization stability by preventing intermediate overflow and underflow when computing FP16/FP32 weight statistics.

Changes:

• Compute standard deviation in float64, then cast the final scale back to the original weight dtype.
• Preserve existing behavior for other input dtypes, the FLOAT8 reference distribution, and zero-variance inputs.
• Add numerical regression tests and a saved-model native CPU dequantization roundtrip.

No actionable findings were identified.

File	Description
onnxruntime/​test/​python/​quantization/​test_float8_weight_statistics.py	Adds coverage for extreme weights, scale and code accuracy, dtype compatibility, and native inference.
onnxruntime/​python/​tools/​quantization/​quant_utils.py	Uses float64 statistics for FP16/FP32 FLOAT8 weight quantization while preserving scale dtype.

_{Review effort: Balanced}