Skip to content

Optimize DefaultDeterminizeStateTable with move semantics in CompactHashBiTable.#83

Open
copybara-service[bot] wants to merge 1 commit intomainfrom
copybara/900056191
Open

Optimize DefaultDeterminizeStateTable with move semantics in CompactHashBiTable.#83
copybara-service[bot] wants to merge 1 commit intomainfrom
copybara/900056191

Conversation

@copybara-service
Copy link
Copy Markdown

@copybara-service copybara-service bot commented Apr 15, 2026

Optimize DefaultDeterminizeStateTable with move semantics in CompactHashBiTable.

This change adds a perfect-forwarding FindId template (and a const T& overload to support initializer lists) to CompactHashBiTable, allowing entries to be efficiently copied or moved into the table. This is used in DefaultDeterminizeStateTable to avoid manual memory management and support storing std::unique_ptr<StateTuple>.

In particular:

  • forcing a reference to T to allow implicit conversions (e.g. string literals to std::string).
  • addressing template deduction failures for initializer lists in FindId by providing a non-template const T& overload.

Also added a new benchmark BM_DeterminizeTransHighlyNonDeterministic to stress test determinization with highly non-deterministic transducers generating long output strings.

Impact (based on fair comparison with clean config and long strings):

  • Transducer determinization improved by ~9.3% (BM_DeterminizeTrans).
  • On-the-fly transducer determinization improved by ~17.6% (BM_DeterminizeFstTrans).
  • Acceptor determinization showed improvements of 2-3%.
  • The new highly non-deterministic test is performance-neutral.
  • Code cleanup: Removed manual memory management (raw pointers and manual delete) in DefaultDeterminizeStateTable to address an existing TODO.

Also:

  • Fixed a potential use-after-free (null pointer dereference) in FindState where tuple->subset was accessed after tuple was moved into the state table.

@copybara-service copybara-service bot changed the title Internal change Optimize DefaultDeterminizeStateTable with move semantics in CompactHashBiTable. Apr 15, 2026
@copybara-service copybara-service bot force-pushed the copybara/900056191 branch 3 times, most recently from 0ce9d42 to a1094f1 Compare April 15, 2026 11:58
@copybara-service copybara-service bot changed the title Optimize DefaultDeterminizeStateTable with move semantics in CompactHashBiTable. Internal change Apr 15, 2026
@copybara-service copybara-service bot changed the title Internal change Optimize DefaultDeterminizeStateTable with move semantics in CompactHashBiTable. Apr 15, 2026
…ctHashBiTable`.

This change adds a perfect-forwarding `FindId` template (and a `const T&` overload to support initializer lists) to `CompactHashBiTable`, allowing entries to be efficiently copied or moved into the table. This is used in `DefaultDeterminizeStateTable` to avoid manual memory management and support storing `std::unique_ptr<StateTuple>`.

In particular:
- forcing a reference to `T` to allow implicit conversions (e.g. string literals to `std::string`).
- addressing template deduction failures for initializer lists in `FindId` by providing a non-template `const T&` overload.

Also added a new benchmark `BM_DeterminizeTransHighlyNonDeterministic` to stress test determinization with highly non-deterministic transducers generating long output strings.

Impact (based on fair comparison with clean config and long strings):
- Transducer determinization improved by ~9.3% (`BM_DeterminizeTrans`).
- On-the-fly transducer determinization improved by ~17.6% (`BM_DeterminizeFstTrans`).
- Acceptor determinization showed improvements of 2-3%.
- The new highly non-deterministic test is performance-neutral.
- Code cleanup: Removed manual memory management (raw pointers and manual delete) in `DefaultDeterminizeStateTable` to address an existing TODO.

Also:
- Fixed a potential use-after-free (null pointer dereference) in `FindState` where `tuple->subset` was accessed after `tuple` was moved into the state table.
PiperOrigin-RevId: 900056191
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant