Skip to content

Handle scalar and empty tensors in integer quantization helpers - #34185

Open
Afloat (Afloat16) wants to merge 2 commits into
microsoft:mainfrom
Afloat16:fix/quantization-scalar-empty-tensors-20261113
Open

Afloat (Afloat16) wants to merge 2 commits into
microsoft:mainfrom
Afloat16:fix/quantization-scalar-empty-tensors-20261113

Conversation

@Afloat16

Copy link
Copy Markdown

Description

Integer compute_data_quant_params uses len(data) to decide whether an array contains values. This raises on a zero-dimensional tensor and misclassifies empty shapes such as (2, 0) as nonempty, leading to invalid reductions or inconsistent override handling.

Use data.size for the three element-count checks. Scalars now follow the same quantization calculation as a one-element vector. Empty integer tensors use the existing identity scale/zero-point defaults regardless of which dimension is empty, while preserving their shape.

Validation

  • Seven new unittest methods cover FP16/FP32, INT4/UINT4/INT8/UINT8/INT16/UINT16, symmetric/asymmetric settings, explicit hard-coded scale/code expectations, range overrides, minimum range, readonly inputs and invalid-argument controls.
  • A native CPU DequantizeLinear + Add model consumes the scalar quantized initializer after ONNX save/reload.
  • New and existing quantization utility tests: 16 methods passed, 279 subtests passed.
  • Restoring the original helper produces 1 failing method and 123 failing subtests (pytest reports 124 failures), with 6 completed methods and 25 passing subtests.
  • Ruff 0.12.12 lint/format and git diff --check passed.

Exact-source native validation: https://github.com/Afloat16/onnxruntime/actions/runs/43402852485

The runtime was built using the repository's official CPU Release build.py/CMake process at 8dea1194e974fa07b1d2f07a190949c3a92cab80. Its C++ sources match base a8246c0d094f1ab0d6bf236228e5e67683321cd7. This Python-only validation installs that built wheel, overlays the complete quantization package from this exact commit, and verifies byte equality and hashes before normal native imports. It does not claim a separate C++ rebuild for this Python-only change.

Motivation and Context

The public array helpers should determine emptiness by total element count. This change is limited to integer quantization; FLOAT8's separate statistics path is unchanged. Some higher-level initializer callers already flatten their inputs, so this is not a claim that every static quantization entry point previously rejected scalars. No model-quality or throughput improvement is claimed.

Copilot AI balanced review requested due to automatic review settings November 13, 2026 09:54
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the scalar and empty-array quantization changes.

No open issues 🎉

What changed in this PR

This PR fixes scalar and empty-array handling in ONNX Runtime’s integer quantization helpers without changing the FLOAT8 path.

Changes:

  • Replace three len(data) checks with data.size, allowing scalars to quantize normally and empty tensors to retain identity parameters and their original shape.
  • Add regression tests for quantization types, overrides, readonly inputs, invalid arguments, and scalar serialization followed by native CPU inference.
File Description
onnxruntime/​test/​python/​quantization/​test_quantize_data_shapes.py Adds seven test methods covering scalar and empty tensors, controls, and native inference.
onnxruntime/​python/​tools/​quantization/​quant_utils.py Uses total element count for integer quantization’s emptiness checks.

Review effort: Balanced

@mirounga
mirounga self-requested a review November 13, 2026 16:06
from pathlib import Path

import numpy as np
import onnx
Remove the mixed import forms flagged by CodeQL without changing the test behavior.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants