Skip to content

Preserve histogram probability mass in entropy calibration - #34180

Open
Afloat (Afloat16) wants to merge 2 commits into
microsoft:mainfrom
Afloat16:fix/entropy-histogram-reconstruction-20261112
Open

Afloat (Afloat16) wants to merge 2 commits into
microsoft:mainfrom
Afloat16:fix/entropy-histogram-reconstruction-20261112

Conversation

@Afloat16

Copy link
Copy Markdown

Description

Correct reconstruction of a quantized histogram when evaluating entropy-calibration thresholds:

  • Store reconstructed bucket averages in floating point instead of truncating them into the integer histogram dtype.
  • Expand the final quantized bucket over its complete interval, including the remainder bins already accumulated into that bucket.
  • Distribute each bucket's mass only over the originally occupied histogram bins.

These changes make the reconstructed distribution preserve the quantized bucket mass and original support. Candidate threshold enumeration, outlier handling, smoothing, KL selection, and threshold dtypes retain their existing behavior.

Motivation and Context

Integer truncation and incomplete expansion can change the distribution compared by KL divergence, selecting an incorrect clipping interval. The regression cases use independent Python bucket reconstruction to check mass conservation and support, then exercise real CPU calibration, persisted calibration data, static quantization, and quantized inference.

For the fixed integration fixture, correcting histogram reconstruction changes the stored UINT8 activation scale from 16/255 to the independently expected 13/255. This is a correctness result for that fixture, not a claim of a general model-accuracy gain.

Validation

  • Built the Release CPU runtime from this exact PR commit using the official tools/ci_build/build.py on Ubuntu 24.04 / Python 3.12. The source-build job completed compilation and produced the wheel.
  • The successful native validation consumes that exact source-built wheel and checks its source commit and installed calibration source bytes: 27 passed, 17 subtests passed, including the existing calibration module.
  • Restoring the original calibration implementation in the same runtime reproduces 1 failing test method and 10 failing subtests; the other controls pass. The follow-up validation corrects source-package shadowing during the first job's pytest collection.
  • Ruff lint, Ruff formatting, and git diff --check pass.

Validation covers the CPU calibration/quantization paths above; the full C++ and GPU suites were not run.

Copilot AI balanced review requested due to automatic review settings November 13, 2026 09:06
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified in the entropy-calibration histogram reconstruction changes.

No open issues 🎉

What changed in this PR

This PR corrects histogram reconstruction in entropy calibration so KL-divergence comparisons preserve bucket probability mass and histogram support.

Changes:

  • Use floating-point reconstructed counts to avoid truncating fractional averages.
  • Include remainder bins in the final bucket and distribute mass only over occupied bins.
  • Add regression tests covering threshold selection, CPU calibration, persisted ranges, static quantization, and inference.

No actionable findings were identified.

File Description
onnxruntime/​test/​python/​quantization/​test_entropy_reconstruction.py Adds reference-based regression tests and CPU calibration-to-inference coverage.
onnxruntime/​python/​tools/​quantization/​calibrate.py Fixes reconstruction precision, final-bucket coverage, and support masking.

Review effort: Balanced

@mirounga
mirounga self-requested a review November 13, 2026 16:12
from pathlib import Path

import numpy as np
import onnx
Remove the mixed import forms flagged by CodeQL without changing the test behavior.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants