Skip to content

Add INT_W8A8_CONVROT: W8A8 quantization with Hadamard rotation#1632

Draft
dxqb wants to merge 1 commit into
Nerogar:masterfrom
dxqb:int8-convrot-squashed
Draft

Add INT_W8A8_CONVROT: W8A8 quantization with Hadamard rotation#1632
dxqb wants to merge 1 commit into
Nerogar:masterfrom
dxqb:int8-convrot-squashed

Conversation

@dxqb

@dxqb dxqb commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

# Prototype knobs - env-driven, no UI/config plumbing (see PLAN in-an-new-branch-reflective-cocke).
# Block size for the group-wise Hadamard rotation. Not empirically validated; 128 is a common
# group size in block quantization literature, picked as a starting point to sweep from.
CONVROT_BLOCK_SIZE = int(os.environ.get("CONVROT_BLOCK_SIZE", "128"))
# Isolates the untested part of this experiment: whether quantizing the backward grad-output
# to int8 (matching plain int8w8's backward) hurts LoRA gradient quality vs keeping it in
# bf16. Default matches int8w8 so the two paths are directly comparable.
CONVROT_BF16_DY = os.environ.get("CONVROT_BF16_DY", "0") == "1"

Test plan

  • pre-commit run --all-files passes
  • Launched the affected UI or script and exercised the change
  • Tested with at least one real preset / config when relevant (note which: ____)

AI assistance

  • Early AI prototype — opened for discussion, not ready for review

Adds LinearInt8ConvRot and a hadamard rotation helper, wired up as a
new DataType option alongside the existing W8A8 variants. The Hadamard
matrix is precomputed into a non-persistent buffer so forward/backward
run cleanly inside torch.compile'd + gradient-checkpointed blocks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant