Repository navigation

Index hierarchy - #728

Open

MarkWolters wants to merge 119 commits into

mainfrom

index_hierarchy

Open

Index hierarchy#728
MarkWolters wants to merge 119 commits into
mainfrom
index_hierarchy

Commits on Dec 23, 2025
- Initial commit of the ASH indexing and vector aspects, plus some basic testing programs. Not executable yet.
  tlwillke
  committed
  727563d
Commits on Dec 24, 2025
- First working version with scorer. Other minor improvements.
  tlwillke
  committed
  7a60422
Commits on Dec 26, 2025
- Enable float encoding and scoring of A, maintaining double for training. Minor dataset updates as well.
  tlwillke
  committed
  0dc6fb2
- Fix for shape of thin QR matrix. Plus scaffolding for default (scalar) ASH backend. Validated ASH error and performance correctness.
  tlwillke
  committed
  f730faa
Commits on Dec 27, 2025
- First implementation with fast scalar and SIMD projection encoding.
  tlwillke
  committed
  e91a842
Commits on Dec 30, 2025
Commits on Dec 31, 2025
- New distance calculation benchmark for PQ.
  tlwillke
  committed
  5c4f3fa
Commits on Jan 1, 2026
Commits on Jan 3, 2026
Commits on Jan 4, 2026
- Added multi-landmark support to the scalar scorer. Updated DistancesASH to acceptt landmark count.
  tlwillke
  committed
  1855f3d
Commits on Jan 5, 2026
- Created a pooled landmark-specific query buffer. Still using the non-pooled version for performance purposes though.
  tlwillke
  committed
  e67f1bc
- Added debug instrumentation to kmeans++ clusterer.
  tlwillke
  committed
  a4ce5e4
Commits on Jan 6, 2026
- Bug fixes in SIMD scoreRange for copying queries into contiguous scratch and mixed landmarks.
  tlwillke
  committed
  97ea704
Commits on Jan 7, 2026
Commits on Jan 8, 2026
Commits on Jan 12, 2026
- Replaced QR with Thin SVD in runWithinTraining and temporarily overriding payload alignment exception for recall benchmarking purposes.
  tlwillke
  committed
  41b6313
Commits on Jan 13, 2026
- Changed computeNativeV(X) to computeNativeVt(X) to fix ITQ. Other small fixes. ITQ and multi-landmarks work.
  tlwillke
  committed
  92a14be
Commits on Jan 14, 2026
- Partial implementation of ASH single vector scoring logic SIMD support. Does not work yet.
  tlwillke
  committed
  19eb873
- Initial ASH integration into graph index. Scalar scoring with separated vectors. Recall and perf look acceptable.
  tlwillke
  committed
  45285b0
Commits on Feb 14, 2026
- Batch kmeans clustering. Renamed various scorers. Introduced new packed layout with header interleaving. Introduced ashMaskedAdd_512.
  tlwillke
  committed
  a843ae3
Commits on Feb 15, 2026
- Merge latest main into branch.
  tlwillke
  committed
  5ae0229
- Additional cleanup for the merge of latest main into branch.
  tlwillke
  committed
  1d64270
Commits on Feb 23, 2026
- Merge latest main.
  tlwillke
  committed
  d3e8fcc
Commits on Mar 3, 2026
- Merged latest main, including logging support. Hierarchical indexing does not work.
  tlwillke
  committed
  276a5e3
- Merging latest main into branch.
  tlwillke
  committed
  9a3f4b4
Commits on Mar 5, 2026
- Extend csv logging to include build and search compressor information.
  tlwillke
  committed
  67849bc
- Merge remote-tracking branch 'origin/logging-improvements' into ash-dev
  tlwillke
  committed
  664236b
Commits on Mar 24, 2026
- Keep ITQ in flat column-major buffers and compute Xbin in row blocks to reduce conversions and memory traffic without changing training behavior. Iterations are ~2.6X faster.
  tlwillke
  committed
  9ad4f60
Commits on Apr 14, 2026
- Latest work on SIMD masked adds for ASH.
  tlwillke
  committed
  0a7ea66
- Merge latest main into branch.
  tlwillke
  committed
  af69930
- Refactored YAML benchmark configs post-rebase. Added an ASH compressionRatio.
  tlwillke
  committed
  c33f53b
Commits on Jun 10, 2026
- Initial implementation of multibit ASH trainer.
  tlwillke
  committed
  9250a7d
- Removed Landing algo from DistancesASH.
  tlwillke
  committed
  eac630a
- Remove Landing trainer from Asymmetric Hashing.
  tlwillke
  committed
  e0047db
- Removed Landing algo selection from Compression.
  tlwillke
  committed
  1508c6a
Commits on Jun 11, 2026
- Initial implementation of scalar/fallback multibit ASH support.
  tlwillke
  committed
  dad1afe
Commits on Jun 12, 2026
- Initial implementation of Fused ASH.
  tlwillke
  committed
  95909aa
Commits on Aug 28, 2026
- reworking index hierarchy
  MarkWolters
  committed
  ffd4fa9
- adding new builder types
  MarkWolters
  committed
  df46610
Commits on Aug 31, 2026
- going back to old packages to avoid breaking changes
  MarkWolters
  committed
  414f9ca
- cleaning up
  MarkWolters
  committed
  91a1fea
Commits on Sep 2, 2026
- first cut
  MarkWolters
  committed
  88a51bd
- rebase on main
  MarkWolters
  committed
  8a14ec1
Commits on Sep 3, 2026
- adding example program
  MarkWolters
  committed
  3541459
Commits on Sep 4, 2026
- Merge remote-tracking branch 'origin/main' into index_hierarchy
  MarkWolters
  committed
  b79681a
- rebase on main
  MarkWolters
  committed
  e962dc2
Commits on Sep 11, 2026
Commits on Sep 14, 2026
- Merge latest main into dev branch.
  tlwillke
  committed
  894d5ed
Commits on Sep 16, 2026
- Stub off ASH compaction for now.
  tlwillke
  committed
  e9598a1
- Minor dataset config change.
  tlwillke
  committed
  8a73f54
- Initial commit for adaptive termination algo.
  tlwillke
  committed
  a7d5d1f
- Implementation of Distance Adaptive Beam Search.
  tlwillke
  committed
  8de3e70
- Integrate adaptive termination for ASH graph search
  tlwillke
  committed
  97da470
- Enable adaptive termination for ASH construction scores
  tlwillke
  committed
  479a61e
Commits on Sep 17, 2026
- Parallelize ASH ITQ training encoding
  tlwillke
  committed
  00a599b
- Use divide-and-conquer SVD for ASH ITQ polar steps
  tlwillke
  committed
  334185a
- Set ASH adaptive termination default gamma to 0.01
  tlwillke
  committed
  28d9620
- Resize ASH quantization workspace for each vector shape
  tlwillke
  committed
  a551a5d
- Use 20x ASH ITQ training sample with early stopping
  tlwillke
  committed
  b2555f8
- Align ASH benchmark configs at 32x and retain PQ alternatives
  tlwillke
  committed
  21d2d21
- Fix ASH SIMD projection gather for partial blocks
  tlwillke
  committed
  2715f71
- Validate ASH exhaustive recall across scalar and SIMD scorers
  tlwillke
  committed
  9349016
Commits on Sep 18, 2026
- Validate 64-landmark ASH fused graph scoring
  tlwillke
  committed
  f25dacf
- Align ASH and PQ single-thread throughput benchmarks
  tlwillke
  committed
  4d2c48a
- Share ASH query projection in fused graph scorer
  tlwillke
  committed
  b1fe884
- Write fused ASH neighborhoods in bulk
  tlwillke
  committed
  67f532a
Commits on Sep 19, 2026
- Complete symmetric ASH construction scoring for packed codes
  tlwillke
  committed
  baa1027
- Benchmark prepared symmetric ASH queries in DistancesASH
  tlwillke
  committed
  f8e6421
- Compare ASH construction scoring policies and setup costs
  tlwillke
  committed
  992f7c6
- Optimize symmetric ASH scoring from encoded vectors
  tlwillke
  committed
  cc5d0d1
- Measure ASH scoring with correct query setup and isolated modes
  tlwillke
  committed
  62bc6f6
- Prevent boxed SIMD allocations during ASH graph pruning
  tlwillke
  committed
  1338d46
- Measure graph throughput over sustained query batches
  tlwillke
  committed
  f98fcf9
- Expose symmetric ASH scoring through the diversity API
  tlwillke
  committed
  1707d91
- Document symmetric ASH correctness and performance study
  tlwillke
  committed
  922b23e
- Control graph benchmark query concurrency and verify result parity
  tlwillke
  committed
  638edc3
- Document completed ASH versus PQ Figure 9 graph study
  tlwillke
  committed
  d35925e
Commits on Sep 24, 2026
- rebasing on main
  MarkWolters
  committed
  1c4be64
Commits on Sep 29, 2026
- adding MutableHnswIndex type to allow streaming of nodes
  MarkWolters
  committed
  23f4537
- bug fixes
  MarkWolters
  committed
  fa2f88b
- rounding out the examples
  MarkWolters
  committed
  12c9302
- filling holes
  MarkWolters
  committed
  0d0a064
- still cleaning up here
  MarkWolters
  committed
  6aef131
Commits on Oct 2, 2026
- design reworking
  MarkWolters
  committed
  093b927
Commits on Oct 5, 2026
- fixing numerous issues
  MarkWolters
  committed
  7b09ccf
- final updates
  MarkWolters
  committed
  bd000e8
Commits on Oct 6, 2026
- Merge branch 'main' into index_hierarchy
  MarkWolters
  committed
  96df8c1
- release notes
  MarkWolters
  committed
  a827acf
Commits on Oct 8, 2026
- applying api changes to Grid
  MarkWolters
  committed
  3fc069b
Commits on Oct 9, 2026
- initial merge of ash dev with api updates
  MarkWolters
  committed
  ed38c88
- fixing unit tests and licenses
  MarkWolters
  committed
  1d1773c
- make ASH compression type for builder
  MarkWolters
  committed
  61098f9
Commits on Oct 13, 2026
- rebasing on main
  MarkWolters
  committed
  067d2fc
- fixing unit tests and delete unneeded docs
  MarkWolters
  committed
  6eb316c
Commits on Oct 20, 2026
- rebasing on main
  MarkWolters
  committed
  b1b6508
- remove duplicate dependency
  MarkWolters
  committed
  d143d94
- Use scalar ASH kernels on JDK 20; disable thread-leak check in TestNeighbors
  MarkWolters
  committed
  7b3e436
Commits on Oct 21, 2026
- repairing datasets file that had most datasets commented out on this branch
  MarkWolters
  committed
  cee1942

Commits

Commits on Dec 23, 2025

Commits on Dec 24, 2025

Commits on Dec 26, 2025

Commits on Dec 27, 2025

Commits on Dec 30, 2025

Commits on Dec 31, 2025

Commits on Jan 1, 2026

Commits on Jan 3, 2026

Commits on Jan 4, 2026

Commits on Jan 5, 2026

Commits on Jan 6, 2026

Commits on Jan 7, 2026

Commits on Jan 8, 2026

Commits on Jan 12, 2026

Commits on Jan 13, 2026

Commits on Jan 14, 2026

Commits on Feb 14, 2026

Commits on Feb 15, 2026

Commits on Feb 23, 2026

Commits on Mar 3, 2026

Commits on Mar 5, 2026

Commits on Mar 24, 2026

Commits on Apr 14, 2026

Commits on Jun 10, 2026

Commits on Jun 11, 2026

Commits on Jun 12, 2026

Commits on Aug 28, 2026

Commits on Aug 31, 2026

Commits on Sep 2, 2026

Commits on Sep 3, 2026

Commits on Sep 4, 2026

Commits on Sep 11, 2026

Commits on Sep 14, 2026

Commits on Sep 16, 2026

Commits on Sep 17, 2026

Commits on Sep 18, 2026

Commits on Sep 19, 2026

Commits on Sep 24, 2026

Commits on Sep 29, 2026

Commits on Oct 2, 2026