(opt): use foldhash for ordered hash containers - #10347
Conversation
PR SummaryLow Risk Overview Call sites that only needed SipHash for type inference (e.g. Sierra → CASM is the deliberate exception: Reviewed by Cursor Bugbot for commit c4a7f02. Bugbot is set up for automated code reviews on this repo. Configure here. |
fe63179 to
697ea61
Compare
eytan-starkware
left a comment
There was a problem hiding this comment.
@eytan-starkware+AGNT made 4 comments.
Reviewable status: 0 of 5 files reviewed, 3 unresolved discussions (waiting on eytan-starkware, orizi, and TomerStarkware).
a discussion (no related file):
Note: the comments below are from an automatic orizi-review run (Claude agents reviewing in Ori's style, findings adversarially verified before posting). Treat with the usual bot skepticism.
crates/cairo-lang-sierra-to-casm/src/references.rs line 31 at r1 (raw file):
/// them live at once, so this uses the DoS-resistant `RandomState` (SipHash) rather than the /// crate-default fast hasher, whose weaker collision resistance would allow quadratic blowup. pub type StatementRefs = OrderedHashMap<VarId, ReferenceValue, RandomState>;
this undoes the PR in the one place it matters most. StatementRefs is the hottest map in the backend — take_vars/put_vars hash every arg/result of every statement, and it's cloned at every backwards-jump target. pinning SipHash here keeps the slow hasher exactly where the profile shows it.
the threat model also seems inconsistent: VarId hashes only its u64 id, and every other container keyed by the same untrusted sierra ids already runs foldhash — e.g. TypeSizeMap = UnorderedHashMap<ConcreteTypeId, i16> is program-sized and attacker-keyed. so either HashDoS on untrusted sierra needs a project-wide answer (in its own PR), or drop the pin.
pub type StatementRefs = OrderedHashMap<VarId, ReferenceValue>;
and drop the use std::collections::hash_map::RandomState; on line 1.
crates/cairo-lang-sierra/src/edit_state.rs line 41 at r1 (raw file):
} impl<V, BH: core::hash::BuildHasher> EditState<V> for OrderedHashMap<VarId, V, BH> {
this generalization only exists to serve the RandomState pin in StatementRefs. revert it with the pin.
impl<V> EditState<V> for OrderedHashMap<VarId, V> {
crates/cairo-lang-semantic/src/items/imp.rs line 3681 at r1 (raw file):
) -> OrderedHashSet<CrateId<'db>> { let mut crates = [crate_id, db.core_crate()].into_iter().unique().collect_vec(); let mut crates_set = OrderedHashSet::<CrateId<'db>>::from_iter(crates.iter().copied());
turbofish is redundant now - the type flows from the return type.
let mut crates_set = OrderedHashSet::from_iter(crates.iter().copied());
OrderedHashMap/Set iterate in insertion order, so iteration determinism is independent of the hasher - yet they still used std's SipHash, which showed up throughout the lowering optimization passes. Use hashbrown's default hasher (foldhash), matching the unordered containers. The one site naming RandomState to pin type inference now names the default hasher instead. foldhash is only minimally DoS-resistant, so StatementRefs - the one sierra-to-casm map keyed by an untrusted program's VarIds, which an attacker can make thousands of live at once - is pinned back to SipHash; this measured as free on the compilation benchmark. Everything else on the sequencer path is keyed by dense sequential ids (type/libfunc/ function), unaffected. Improves full-compilation CPU by ~9% (additive with the size-estimation memoization). SIERRA_UPDATE_NO_CHANGE_TAG=only generalizes a hasher bound and swaps in-memory hash-map hashers; no change to any compilation output. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
697ea61 to
c4a7f02
Compare
orizi
left a comment
There was a problem hiding this comment.
@orizi+AGNT made 3 comments and resolved 3 discussions.
Reviewable status: 0 of 5 files reviewed, all discussions resolved (waiting on TomerStarkware).
crates/cairo-lang-semantic/src/items/imp.rs line 3681 at r1 (raw file):
Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…
turbofish is redundant now - the type flows from the return type.
let mut crates_set = OrderedHashSet::from_iter(crates.iter().copied());
Done.
crates/cairo-lang-sierra/src/edit_state.rs line 41 at r1 (raw file):
Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…
this generalization only exists to serve the
RandomStatepin inStatementRefs. revert it with the pin.impl<V> EditState<V> for OrderedHashMap<VarId, V> {
The pin stays (see the discussion on references.rs) - it was requested explicitly and measured free - so this generalization stays with it.
crates/cairo-lang-sierra-to-casm/src/references.rs line 31 at r1 (raw file):
Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…
this undoes the PR in the one place it matters most.
StatementRefsis the hottest map in the backend —take_vars/put_varshash every arg/result of every statement, and it's cloned at every backwards-jump target. pinning SipHash here keeps the slow hasher exactly where the profile shows it.the threat model also seems inconsistent:
VarIdhashes only itsu64id, and every other container keyed by the same untrusted sierra ids already runs foldhash — e.g.TypeSizeMap = UnorderedHashMap<ConcreteTypeId, i16>is program-sized and attacker-keyed. so either HashDoS on untrusted sierra needs a project-wide answer (in its own PR), or drop the pin.pub type StatementRefs = OrderedHashMap<VarId, ReferenceValue>;and drop the
use std::collections::hash_map::RandomState;on line 1.
The pin was an explicit requirement (orizi), and it was measured before landing: pinning StatementRefs to SipHash is perf-neutral on the corelib and staking diagnostics benchmarks and on the full sierra-to-casm compilation stopwatch (interleaved A/B). The "hottest map" intuition doesn't survive measurement - the map is small (live vars at a statement), so hashing cost there is noise.
The threat model is consistent: felt252_serde enforces i == id.id for type/libfunc/function declarations at deserialization, so ConcreteTypeId-style keys are dense sequential integers on the sequencer path - collision-proof under any hasher, foldhash included. VarId is the one sierra id an attacker chooses freely (no density validation), and thousands can be live at once, hence the pin exactly here and nowhere else.

OrderedHashMap/Set iterate in insertion order, so iteration determinism
is independent of the hasher - yet they still used std's SipHash, which
showed up throughout the lowering optimization passes. Use hashbrown's
default hasher (foldhash), matching the unordered containers. The one
site naming RandomState to pin type inference now names the default
hasher instead.
Improves full-compilation CPU by ~9% (additive with the size-estimation
memoization).
Co-Authored-By: Claude Fable 5 noreply@anthropic.com