Skip to content

Optimize MakeCurrentSnapshotsQuery: single grouped max() pass (SQLite) - #118

Draft
hahn-kev-bot wants to merge 1 commit into
mainfrom
claude/makecurrentsnapshotsquery-perf-2yhcq7
Draft

hahn-kev-bot wants to merge 1 commit into
mainfrom
claude/makecurrentsnapshotsquery-perf-2yhcq7

Conversation

@hahn-kev-bot

Copy link
Copy Markdown
Collaborator

Summary

MakeCurrentSnapshotsQuery selects the newest snapshot per entity. It used a first_value() window function and then removed the duplicate rows it produced with GROUP BY s.EntityId. The plan sorted the snapshot/commit join twice:

USE TEMP B-TREE FOR RIGHT PART OF ORDER BY   <- window sort
USE TEMP B-TREE FOR GROUP BY                  <- second pass

This PR replaces it with a single grouped max() aggregation.

How it works

Two facts make the new query sort-free:

  • Scanning Snapshots via IX_Snapshots_EntityId delivers rows already grouped by EntityId, so GROUP BY needs no sort.
  • max() is a streaming aggregate, so there is no window sort either.

SQLite guarantees that when a query has exactly one min()/max() aggregate, every bare (non-aggregated) column takes its value from the row that produced that max (docs). So s.* is the whole snapshot row whose commit is greatest.

max() needs a single scalar, so the commit order (DateTime, Counter, Id) is encoded as one lexically-ordered text key: DateTime is stored by EF as fixed-width sortable text (the same assumption the old ORDER BY relied on), Counter is a non-negative long zero-padded to 20 digits (covers the full long range) so text order matches numeric order, and Id (a fixed-width GUID) is last.

EXPLAIN QUERY PLAN after the change — both temp B-trees are gone:

SCAN s USING INDEX IX_Snapshots_EntityId
SEARCH c USING INDEX (Id=?)

Provider note

This relies on SQLite's bare-column behaviour and is not valid on other providers (e.g. Postgres). The query is SQLite-only by design; a portable equivalent would be ROW_NUMBER() ... WHERE rn = 1.

Benchmarks

Measured with a standalone Python/SQLite harness (schema and indexes mirroring the EF model), 50k entities, best of several runs:

distribution original new
~127k snapshots (with history) 786 ms 497 ms (~37% faster)
~55k snapshots (after pruning) 502 ms 324 ms (~35% faster)

Correctness

The returned result set was verified identical to the previous query for:

  • the no-filter case, and
  • ignoreChangesAfter filters (including DateTime ties broken by Counter/Id) spanning early, middle and late history.

Testing

Validated at the SQL layer only — the .NET SDK was not available in the environment used to write this, so dotnet test (the RepositoryTests cover CurrentSnapshots) has not been run yet. Please run the suite before merging.

Possible follow-up

Denormalizing the commit sort keys onto Snapshots with a composite index removes the commit join entirely (~3x in the same benchmark), but needs a schema migration — out of scope for this PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01C2nfncTWgKAjER7Ff4PUMS


Generated by Claude Code

The current-snapshots query selected the latest snapshot per entity with a
first_value() window function and then removed the duplicate rows it produced
with GROUP BY s.EntityId. The plan sorted the snapshot/commit join twice: a
temp B-tree for the window ORDER BY, then a second temp B-tree for the GROUP BY.

Replace it with a single grouped aggregation. Scanning Snapshots via
IX_Snapshots_EntityId delivers rows already grouped by EntityId, so GROUP BY
needs no sort, and max() is a streaming aggregate, so there is no window sort
either. SQLite guarantees that with exactly one max() aggregate the bare
columns come from the row that produced the max, so "s".* is the snapshot whose
commit is greatest. The commit order (DateTime, Counter, Id) is encoded as one
lexically-ordered text key (Counter zero-padded to 20 digits, covering the full
non-negative long range) so a single max() reproduces the tuple ordering.

EXPLAIN drops both temp B-trees; the plan is now just a scan of Snapshots by
EntityId plus the commit lookup. Measured on SQLite (50k entities): ~497ms vs
~786ms with history, ~324ms vs ~502ms after pruning (~35-37% faster).

This relies on SQLite's bare-column behaviour and is not valid on other
providers. Result set verified identical to the previous query for the
no-filter case and for ignoreChangesAfter filters (including DateTime ties
broken by Counter/Id) spanning early, middle and late history.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C2nfncTWgKAjER7Ff4PUMS
@coderabbitai

coderabbitai Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants