Skip to content

(opt): use foldhash for ordered hash containers - #10347

Open
orizi wants to merge 1 commit into
mainfrom
orizi/08-14-_opt_use_foldhash_for_ordered_hash_containers
Open

(opt): use foldhash for ordered hash containers#10347
orizi wants to merge 1 commit into
mainfrom
orizi/08-14-_opt_use_foldhash_for_ordered_hash_containers

Conversation

@orizi

@orizi orizi commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

OrderedHashMap/Set iterate in insertion order, so iteration determinism
is independent of the hasher - yet they still used std's SipHash, which
showed up throughout the lowering optimization passes. Use hashbrown's
default hasher (foldhash), matching the unordered containers. The one
site naming RandomState to pin type inference now names the default
hasher instead.

Improves full-compilation CPU by ~9% (additive with the size-estimation
memoization).

Co-Authored-By: Claude Fable 5 noreply@anthropic.com

@reviewable-StarkWare

Copy link
Copy Markdown

This change is Reviewable

orizi commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@orizi
orizi marked this pull request as ready for review August 14, 2026 07:40
@cursor

cursor Bot commented Aug 14, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Hasher-only performance change with one security-conscious exception; determinism relies on insertion order, not hash seed.

Overview
Switches the default hasher for OrderedHashMap and OrderedHashSet from std’s SipHash to hashbrown’s foldhash, aligned with unordered containers. Comments note that insertion-order iteration keeps compile output deterministic regardless of hasher.

Call sites that only needed SipHash for type inference (e.g. crate_dependencies in semantic) drop explicit RandomState type args. EditState is implemented for OrderedHashMap<VarId, V, BH> for any BuildHasher, so maps with a custom hasher still work.

Sierra → CASM is the deliberate exception: StatementRefs pins RandomState because VarId keys can come from untrusted Sierra and mass collisions on a fast hasher could cause quadratic blowup; that path keeps SipHash for DoS resistance.

Reviewed by Cursor Bugbot for commit c4a7f02. Bugbot is set up for automated code reviews on this repo. Configure here.

@orizi
orizi force-pushed the orizi/08-14-_opt_use_foldhash_for_ordered_hash_containers branch 2 times, most recently from fe63179 to 697ea61 Compare August 14, 2026 08:29

@eytan-starkware eytan-starkware left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eytan-starkware+AGNT made 4 comments.
Reviewable status: 0 of 5 files reviewed, 3 unresolved discussions (waiting on eytan-starkware, orizi, and TomerStarkware).


a discussion (no related file):
Note: the comments below are from an automatic orizi-review run (Claude agents reviewing in Ori's style, findings adversarially verified before posting). Treat with the usual bot skepticism.


crates/cairo-lang-sierra-to-casm/src/references.rs line 31 at r1 (raw file):

/// them live at once, so this uses the DoS-resistant `RandomState` (SipHash) rather than the
/// crate-default fast hasher, whose weaker collision resistance would allow quadratic blowup.
pub type StatementRefs = OrderedHashMap<VarId, ReferenceValue, RandomState>;

this undoes the PR in the one place it matters most. StatementRefs is the hottest map in the backend — take_vars/put_vars hash every arg/result of every statement, and it's cloned at every backwards-jump target. pinning SipHash here keeps the slow hasher exactly where the profile shows it.

the threat model also seems inconsistent: VarId hashes only its u64 id, and every other container keyed by the same untrusted sierra ids already runs foldhash — e.g. TypeSizeMap = UnorderedHashMap<ConcreteTypeId, i16> is program-sized and attacker-keyed. so either HashDoS on untrusted sierra needs a project-wide answer (in its own PR), or drop the pin.

pub type StatementRefs = OrderedHashMap<VarId, ReferenceValue>;

and drop the use std::collections::hash_map::RandomState; on line 1.


crates/cairo-lang-sierra/src/edit_state.rs line 41 at r1 (raw file):

}

impl<V, BH: core::hash::BuildHasher> EditState<V> for OrderedHashMap<VarId, V, BH> {

this generalization only exists to serve the RandomState pin in StatementRefs. revert it with the pin.

impl<V> EditState<V> for OrderedHashMap<VarId, V> {

crates/cairo-lang-semantic/src/items/imp.rs line 3681 at r1 (raw file):

) -> OrderedHashSet<CrateId<'db>> {
    let mut crates = [crate_id, db.core_crate()].into_iter().unique().collect_vec();
    let mut crates_set = OrderedHashSet::<CrateId<'db>>::from_iter(crates.iter().copied());

turbofish is redundant now - the type flows from the return type.

    let mut crates_set = OrderedHashSet::from_iter(crates.iter().copied());

OrderedHashMap/Set iterate in insertion order, so iteration determinism
is independent of the hasher - yet they still used std's SipHash, which
showed up throughout the lowering optimization passes. Use hashbrown's
default hasher (foldhash), matching the unordered containers. The one
site naming RandomState to pin type inference now names the default
hasher instead.

foldhash is only minimally DoS-resistant, so StatementRefs - the one
sierra-to-casm map keyed by an untrusted program's VarIds, which an
attacker can make thousands of live at once - is pinned back to SipHash;
this measured as free on the compilation benchmark. Everything else on
the sequencer path is keyed by dense sequential ids (type/libfunc/
function), unaffected.

Improves full-compilation CPU by ~9% (additive with the size-estimation
memoization).

SIERRA_UPDATE_NO_CHANGE_TAG=only generalizes a hasher bound and swaps
in-memory hash-map hashers; no change to any compilation output.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@orizi
orizi force-pushed the orizi/08-14-_opt_use_foldhash_for_ordered_hash_containers branch from 697ea61 to c4a7f02 Compare August 16, 2026 10:40

@orizi orizi left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@orizi+AGNT made 3 comments and resolved 3 discussions.
Reviewable status: 0 of 5 files reviewed, all discussions resolved (waiting on TomerStarkware).


crates/cairo-lang-semantic/src/items/imp.rs line 3681 at r1 (raw file):

Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…

turbofish is redundant now - the type flows from the return type.

    let mut crates_set = OrderedHashSet::from_iter(crates.iter().copied());

Done.


crates/cairo-lang-sierra/src/edit_state.rs line 41 at r1 (raw file):

Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…

this generalization only exists to serve the RandomState pin in StatementRefs. revert it with the pin.

impl<V> EditState<V> for OrderedHashMap<VarId, V> {

The pin stays (see the discussion on references.rs) - it was requested explicitly and measured free - so this generalization stays with it.


crates/cairo-lang-sierra-to-casm/src/references.rs line 31 at r1 (raw file):

Previously, eytan-starkware+AGNT (Agent AGNT for eytan-starkware) wrote…

this undoes the PR in the one place it matters most. StatementRefs is the hottest map in the backend — take_vars/put_vars hash every arg/result of every statement, and it's cloned at every backwards-jump target. pinning SipHash here keeps the slow hasher exactly where the profile shows it.

the threat model also seems inconsistent: VarId hashes only its u64 id, and every other container keyed by the same untrusted sierra ids already runs foldhash — e.g. TypeSizeMap = UnorderedHashMap<ConcreteTypeId, i16> is program-sized and attacker-keyed. so either HashDoS on untrusted sierra needs a project-wide answer (in its own PR), or drop the pin.

pub type StatementRefs = OrderedHashMap<VarId, ReferenceValue>;

and drop the use std::collections::hash_map::RandomState; on line 1.

The pin was an explicit requirement (orizi), and it was measured before landing: pinning StatementRefs to SipHash is perf-neutral on the corelib and staking diagnostics benchmarks and on the full sierra-to-casm compilation stopwatch (interleaved A/B). The "hottest map" intuition doesn't survive measurement - the map is small (live vars at a statement), so hashing cost there is noise.

The threat model is consistent: felt252_serde enforces i == id.id for type/libfunc/function declarations at deserialization, so ConcreteTypeId-style keys are dense sequential integers on the sequencer path - collision-proof under any hasher, foldhash included. VarId is the one sierra id an attacker chooses freely (no density validation), and thousands can be live at once, hence the pin exactly here and nowhere else.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants