Skip to content

[preconditioner] thread-safe TF32 context manager, generalize rank_deficient_stability_config to MatrixFunctionConfig, refactor _precondition_grad - #273

Open
Vishal-sys-code wants to merge 2 commits into
facebookresearch:mainfrom
Vishal-sys-code:fix/preconditioner-todos-and-optimizations
Open

[preconditioner] thread-safe TF32 context manager, generalize rank_deficient_stability_config to MatrixFunctionConfig, refactor _precondition_grad#273
Vishal-sys-code wants to merge 2 commits into
facebookresearch:mainfrom
Vishal-sys-code:fix/preconditioner-todos-and-optimizations

Conversation

@Vishal-sys-code

Copy link
Copy Markdown

Resolves three open TODOs in distributed_shampoo/preconditioner.

  1. Thread-safe TF32 context manager: _matrix_inverse_root_coupled_higher_order_newton was toggling torch.backends.cuda.matmul.allow_tf32 directly with try/finally. Since this flag is process-global, concurrent calls from multi-threaded optimizer setups could race on it. Added _TF32_LOCK (RLock) and _scoped_tf32_setting context manager so the read/modify/restore is atomic. The fast path (when disable_tf32 is False) skips lock acquisition entirely. Added a multithreaded regression test in matrix_functions_test.py.

  2. Generalize rank_deficient_stability_config to MatrixFunctionConfig: The field was on EigendecompositionConfig with a TODO to move it up to the base class. Moved it to MatrixFunctionConfig along with the default factory. Cleaned up the conditional TODOs in PerturbationConfig and PseudoInverseConfig docstrings that were contingent on this generalization. Removed the assert isinstance(..., EigendecompositionConfig) guards in _compute_preconditioned_gradient and _compute_outer_product_list that only existed because the field was not on the base class yet.

  3. Refactor _precondition_grad: The implementation used functools.reduce with a lambda that called grad.to(dtype=target_dtype) on every iteration and reconstructed (*range(1, grad.ndim), 0) on every non-preconditioned step. Replaced with a direct loop, a single upfront dtype cast, and a precomputed permutation tuple. Removed the now-unused functools.reduce import.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant