0.61.0

Changes

⚠️ Breaks

• break: zip arg (mask, true, false) (#6766) @joseph-isaacs
• break: remove all compute public functions (#6752) @joseph-isaacs
• clean up numeric compute (#6593) @joseph-isaacs
• Remove deprecated to_arrow compute (#6730) @gatesn
• Remove &dyn Array call sites (#6725) @gatesn
• Make ExecutionArgs a dyn trait (#6715) @gatesn
• Inline visitor vtable to ArrayVTable (#6700) @gatesn
• Inline BaseArrayVTable (#6697) @gatesn

✨ Features

• rwlock instead of mutex around encoding ids (#6720) @danking
• Make the constant metadata a Scalar (#6751) @connortsui20
• feat: remove to canonical p2 (#6742) @joseph-isaacs
• Improve benchmark output formatting (#6744) @AdamGS
• vortex read at for pooled cuda reads (#6675) @onursatici

🚀 Performance

• unsafe Scalar::struct_unchecked in scalar_at vtable (#6741) @palaska
• Implement BetweenReduceRule for ConstantArray (#6695) @robert3005
• Remove unnecessary to_canonical in DictArray::take (#6686) @palaska
• Eliminate redundant decompressions in VarBinArray canonicalization (#6692) @palaska
• Remove unnecessary to_canonical in ChunkedArray filter_indices (#6688) @palaska
• separate in_memory vs file coalescing config (#6690) @onursatici
• Remove to_canonical from MaskedArray::take (#6689) @palaska

🐛 Bug Fixes

• Make sequence metadata in-memory + clean up (#6759) @connortsui20
• fix: CUDA bitpacking bench build (#6750) @0ax1
• fix[util]: add balanced reduce iterator (#6737) @joseph-isaacs
• Fix S3 globbing for duckdbfs with httplib (#6709) @myrrc
• Fix old extension scalar logic (#6701) @connortsui20

📖 Documentation

• VTable Unification Docs (#6735) @gatesn

🧰 Maintenance

22 changes

• chore: replace Buffer::copy_from with Buffer::from_iter for improved performance (#6749) @SYaoJun
• Remove unused BitView (#6763) @gatesn
• Remove sparse metadata prost wrapper (#6761) @connortsui20
• Parallelize slow CUDA tests (#6746) @joseph-isaacs
• Add explain feature to DF benchmarks (#6683) @AdamGS
• chore: build, test and lint all CUDA crates on CI (#6743) @0ax1
• test: Enhance tests for RLE and PCO casting: add assertions for dtype… (#6726) @SYaoJun
• chore: remove to_canonical types p1 (#6738) @joseph-isaacs
• chore(deps): update dependency ray to v2.54.0 [security] (#6633) @renovate[bot]
• fix(deps): update junit-framework monorepo to v6.0.3 (#6733) @renovate[bot]
• chore(deps): update crate-ci/typos action to v1.44.0 (#6734) @renovate[bot]
• fix(deps): update dependency ch.qos.logback:logback-classic to v1.5.32 (#6732) @renovate[bot]
• chore(deps): update rust crate moka to v0.12.14 (#6731) @renovate[bot]
• chore(deps): update all patch updates (#6728) @renovate[bot]
• chore(deps): update rust crate wasm-bindgen-futures to v0.4.64 (#6729) @renovate[bot]
• Some adjustments to methods on ExtDTypeRef (#6716) @connortsui20
• chore: Update .clang-tidy to check all C++ source files (#6711) @SYaoJun
• chore: vortex-cuda cleanups (#6703) @0ax1
• Extension Scalar Cleanup (#6699) @connortsui20
• Update public-api (#6698) @gatesn
• Simplify ScalarFnPlugin (#6687) @gatesn
• Remove code for old benchmarks website (#6684) @AdamGS

0.60.0

0.60.0 largely involves unifying our vtable concepts as per the internal docs page in order to lay the groundwork for future extension points. Most notably: full Extension DType support, as well as AggregateFns.

Changes

⚠️ Breaks

• ScalarFn VTable (#6679) @gatesn
• Extension Scalar VTable (#6680) @connortsui20
• Rename spawn_blocking -> spawn_blocking_io (#6678) @gatesn
• Unify ExtVTable per vtable design doc (#6662) @gatesn
• Add an extension module to vortex (#6624) @connortsui20
• break: clean up compare (#6591) @joseph-isaacs
• Merge vortex-dtype into vortex-array (#6582) @connortsui20
• break: clean up compute (#6572) @joseph-isaacs
• Bump Rust MSRV to 1.90 (#6597) @robert3005
• break[array]: lazy list_contains (#6571) @joseph-isaacs

✨ Features

• Support building duckdb from a hash, including CI (#6658) @myrrc
• DuckDB to use the Scan API (#6652) @gatesn
• Use Scan API (#6391) @gatesn
• feat[gpu]: multi-stage dynamic dispatch with plan builder (#6636) @0ax1
• DuckDB lifetimes (#6620) @joseph-isaacs
• Expose table scan progress to DuckDB (#6596) @myrrc
• expose Executor, AbortHandle, AbortHandleRef, and Handle (#6595) @danking

🚀 Performance

• Avoid redundant chunk execution in pack_struct_chunks (#6682) @palaska
• read future to skip polled and dropped events for completed requests (#6667) @onursatici
• perf[gpu]: speed up dyn dispatch run-end kernel (#6665) @0ax1
• Route default append_to_builder through full execution loop (#6655) @palaska
• Add take-based rebuild strategy for ListView with small lists (#6492) @palaska
• oncelock shared array for zero cost access after materialisation (#6653) @onursatici
• oncelock shared array future on cuda flat layout (#6585) @onursatici
• cow buffers on array decode (#6590) @onursatici
• Extract to buffer once in list_contains (#6642) @palaska
• perf(layout): add empty-range fast paths in chunked reader (#6513) @caicancai
• chore[btrblocks]: more precise FoR expected_compression_ratio (#6301) @joseph-isaacs
• perf: remove double execute in exprs (#6608) @joseph-isaacs
• Use approx footer size for caching instead of walking the layout tree (#6601) @robert3005
• do not double validate the flatbuffer on the cuda layout (#6583) @onursatici
• increase default nvme coalescing (#6586) @onursatici
• decoding stats should lock once (#6587) @onursatici
• perf[array]: chunked take with bounded compute graph (#6589) @joseph-isaacs

🐛 Bug Fixes

14 changes

• cuda ctx bind to thread before unsafe (#6681) @onursatici
• keep DropOnSync alive until after dispatching for cuda slices (#6673) @onursatici
• Convert &mut to & for DuckDB GlobalData (#6634) @gatesn
• resize FSST compressor buffer to be large enough for largest string (#6676) @a10y
• Revert "Support building duckdb from a hash, including CI" (#6670) @myrrc
• Close DuckDB database before re-opening (#6654) @gatesn
• More minimal listview rebuild benches, force canonicalize (#6629) @palaska
• only a read lock for current_array_ref (#6631) @danking
• fix: faster codspeed bench for listview_rebuild (#6628) @joseph-isaacs
• Balance expression trees during binary stat falsification (#6622) @AdamGS
• Fix compression benchmarks build (#6614) @AdamGS
• Fix scalar deserialisation where value was unsigned/signed for signed/unsigned type (#6604) @robert3005
• fix: chunked array take with nulls (#6603) @joseph-isaacs
• fix: fuzzer repeat issue tracking (#6577) @joseph-isaacs

📖 Documentation

4 changes

• Language bindings docs (#6666) @gatesn
• Update docs for vtable unification plan (#6661) @gatesn
• docs: Remove unused CLI argument (code always uses temporary path) (#6637) @SYaoJun
• docs: clarify cuda async execution model (#6611) @0ax1

🧰 Maintenance

24 changes

• DuckDB: test curl impl, don't auto-load extensions (#6677) @myrrc
• chore: exclude cuda gen files from auto-formatting (#6671) @0ax1
• use match_each_canonical macro to reduce some canonical impl boilerplate (#6668) @AdamGS
• chore: trim unused crate deps with cargo-machete (#6663) @0ax1
• cudarc use 12.5 instead of 12.8 (#6660) @a10y
• docs: resurrect lost ddb build.rs comment (#6659) @0ax1
• chore[gpu]: alp + for + bp CUDA benchmark (#6644) @0ax1
• chore: clang-format - braces after if or for (#6643) @0ax1
• chore(deps): update codspeedhq/action digest to 2ac5728 (#6638) @renovate[bot]
• duckdb filesystem: don't use wrapper type (#6641) @myrrc
• use PublicBI fork with R2 hosted dataset (#6640) @a10y
• chore(deps): update all patch updates (#6639) @renovate[bot]
• Remove bad paths (#6626) @connortsui20
• Make the tui an default optional dependecy of vortex-python (#6625) @AdamGS
• refactor: simplify Buffer creation from Vec (#6621) @SYaoJun
• chore[ci]: explicitly use stable or night for all cargo commands (#6553) @joseph-isaacs
• Some nice to have things before extension types work (#6619) @connortsui20
• Use cache-cargo-install action instead of installing manually (#6616) @robert3005
• chore: delete commented out metadata tests (#6615) @vignesh-sankaran
• Add publish dry-run CI task (#6594) @robert3005
• More detailed backcompat comments (#6609) @connortsui20
• chore: rename device allocator to device read file (#6605) @0ax1
• chore: vortex-cuda only depend on vortex (#6602) @0ax1
• lower your expectations (#6579) @a10y

0.59.4

Changes

🐛 Bug Fixes

• Published packages cannot depend on unpublished packages (#6592) @robert3005

0.59.3

Changes

🐛 Bug Fixes

• minor c++ duckdb integration changes (#6566) @myrrc
• Vortex crate doesn't unconditionally depend on vortex-cuda-macros (#6580) @robert3005

0.59.2

Changes

🧰 Maintenance

• Add ci check for java publish (#6578) @robert3005

0.59.1

Changes

🐛 Bug Fixes

• fix: CUDA scalar inplace (#6567) @0ax1

🧰 Maintenance

• chore: drop flaky duckdb http e2e test (#6570) @0ax1
• chore: make auto clang-format consistent (#6568) @0ax1

0.59.0

Changes

• docs: tidy and unify Python quickstart examples (#5972) @caicancai
• Better vx-bench docs in README (#5900) @AdamGS

⚠️ Breaks

• Remove vortex-compute (#6561) @connortsui20
• Introduce VortexError::Other and simplify Display implementation (#6410) @AdamGS
• Move vortex-scalar into vortex-array (#6523) @connortsui20
• break[array]: lazy between (#6528) @joseph-isaacs
• break[array]: mask compute to lazy mask execution (#6485) @joseph-isaacs
• break[array]: lazy compare function (#6495) @joseph-isaacs
• Remove vortex-vector (#6529) @connortsui20
• break[array]: move like compute to lazy compute (#6497) @joseph-isaacs
• Pass buffers to VTable::deserialize (#6473) @connortsui20
• break[array]: move cast traits to exprs (#6487) @joseph-isaacs
• Revert #6363 (#6476) @connortsui20
• Use Scalar directly for constant array metadata inlining (#6439) @connortsui20
• Fix Scalar de/serialization with proto and outline nulls from ScalarValue (#6309) @connortsui20
• chore[array]: inline take compute function (#6376) @joseph-isaacs
• Add Array::deserialize arguments (#6340) @gatesn
• Update FSST to store components of compressed codes (#6325) @gatesn
• Array Execution (#6307) @gatesn
• Execute parent to return ArrayRef (#6257) @gatesn
• Extension DType vtables (#6081) @gatesn
• break[expr]: remove Expression.evaluate use Array.apply(expr) instead (#6167) @joseph-isaacs
• Make metrics more local to prevent unbounded memory usage (#6156) @AdamGS
• fallible scalar_at and validity (#6102) @joseph-isaacs
• Remove the python feature from vortex-error (#6064) @AdamGS
• break[*]: make array methods fallible (#6086) @joseph-isaacs
• chore[array]: replace to_canonical with to execute. (#6061) @joseph-isaacs
• Array VTable (#6000) @gatesn
• Remove duplicate ValidityVTable logic (#5975) @gatesn
• chore[array]: remove CanonicalVTable and move append_to_builder to main VTable (#6045) @joseph-isaacs
• Remove EncodeVTable (#6036) @gatesn
• chore: fallible canonical (#6006) @joseph-isaacs
• feat[array]: use execute to return Canonical array not Vector (#5895) @joseph-isaacs

✨ Features

• Add FileSystem trait and make DuckDB configurable (#6564) @gatesn
• Add scan concurrency config for datafusion source (#5945) @AdamGS
• feat: implement DuckDB filesystem integration for Vortex file handling (#6198) @iceboundrock
• vx convert to support cuda compatible vortex files (#6522) @onursatici
• cuda flat layout (#6521) @onursatici
• feat[xtask]: faster public-api (#6498) @joseph-isaacs
• Improve substrait support to include more types and expressions (#6484) @AdamGS
• feat[array]: refactor zip compute kernel to lazy zip via ScalarFn Array (#6461) @joseph-isaacs
• feat[gpu]: CUDA dynamic dispatch kernel (#6483) @0ax1
• feat[array]: lazy numeric (#6469) @joseph-isaacs
• feat[array]: move over to a expr based cast (#6404) @joseph-isaacs
• refactor[array]: move boolean ops to lazy binary expr (#6447) @joseph-isaacs
• Public api lockfiles (#6348) @gatesn
• feat[array]: refactor invert compute kernel to lazy not (#6446) @joseph-isaacs
• sqllogictest runner for DF and DuckDB on Vortex (#6273) @AdamGS
• feat[array]: move over fill_null to scalar_fn array making it lazy (#6406) @joseph-isaacs
• Upgrade Pco to 1.0.1 (#6354) @mwlon
• Add execution target that recursively canonicalises as well as canonical with canonical validity (#6380) @robert3005
• dict of strings kernel to support outlined views (#6401) @onursatici
• zstd buffers cuda kernel (#6394) @onursatici
• Make VortexSource::new public (#6392) @AdamGS
• Make projection pushdown in DF an opt-in feature (#6373) @AdamGS
• feat[expr]: new fill null expr (#6385) @joseph-isaacs
• zstd buffers array (#6362) @onursatici
• gpu compatible write strategy, move compact strategy to use btrblocks with zstd and pco (#6322) @onursatici
• cuda dictionary of vbv decompression (#6361) @0ax1
• CLI flag to enable tracing for benchmarks (#6350) @gatesn
• chore[array]: move take kernel to execute/reduce rules (#6310) @joseph-isaacs
• chore[cuda]: add date32 exporter test (#6345) @a10y
• feat: display tree with no metadata (#5898) @danking
• feat[cuda]: device exports for varbinview (#6338) @a10y
• vortex-metrics v2 (#6187) @AdamGS
• feat[array]: dict array allows signed codes (#6336) @joseph-isaacs
• feat[cuda]: export arrays to ArrowDeviceArray (#6253) @a10y
• feat[gpu]: CUDA date_time_parts kernel (#6303) @0ax1
• feat: alternate for StructFields (#6276) @blaginin
• Add a canonical matcher (#6292) @gatesn
• Move over to array matchers (#6291) @gatesn
• feat[gpu]: cuda runend decoding (#6275) @0ax1
• feat[btrblocks]: add a dynamic btr blocks compressor (#6207) @joseph-isaacs
• feat[cuda]: constant array for numeric types (#6248) @joseph-isaacs
• feat[array]: validate array encodings on write (#6241) @joseph-isaacs
• feat[cuda]: SequenceArray (#6232) @a10y
• feat[cuda]: patches kernel (#6231) @a10y
• Pass limit through Python Vortex (#6239) @gatesn
• Ship Vortex CLI as part of Python package (#6233) @gatesn
• Feature: Add more Filter<_> implementations (#6173) @connortsui20
• Benchmarks website revamped (#6215) @palaska
• Dataset API (#5949) @gatesn
• feat[cuda]: filter for Decimal/VarBinView (#6196) @a10y
• feat[cuda]: add a cuda stream pool (#6205) @joseph-isaacs
• feat[python]: support full read/write from object storage (#6022) @a10y
• feat[scan]: gpu scan (#6199) @joseph-isaacs
• feat[gpu]: nvidia cub filter kernel (#6188) @0ax1
• feature: Add fastlanes bit unpacking cuda kernels (#6145) @robert3005
• Use built-in DF cache for file metadata and fix metrics wiring (#6164) @AdamGS
• feat: reduce size of segments containing fixed-size-list arrays (#6180) @danking
• Feature: Use array.filter() instead of filter(array) compute (#6146) @connortsui20
• Rewrite benchmark website (#6160) @palaska
• feat: stats pruning for LIKE filters (#6049) @a10y
• feat: Zstd CUDA nvCOMP decompression (#6136) @0ax1
• feat[cuda]: canonical to_host (#6130) @joseph-isaacs
• feat[cuda]: dict decimal take (#6129) @joseph-isaacs
• feat[gpu]: dict take values prim (#6101) @joseph-isaacs
• chore[cuda]: decimal_byte_parts (#6118) @joseph-isaacs
• feat[gpu]: scalar encodings (#6109) @joseph-isaacs
• Add VortexScanReplacement to support reading from '*.vortex' (#6111) @robert3005
• feat: nvcomp wrapper crate (#6108) @0ax1
• Allow configuring different readers for DF integration (#6027) @AdamGS
• feat: introduce CudaDeviceBuffer (#6060) @0ax1
• Array VTable (#6000) @gatesn
• feat[arary]: add a slice array to replace the slice op. (#5958) @joseph-isaacs
• Attempting to simplify the IO read traits (#5935) @AdamGS
• feat: CUDA compilation infra & FoR kernel (#5966) @0ax1
• feat[array]: execute::<PrimitiveArray>() (#5954) @joseph-isaacs
• Spark 4 (#5944) @a10y
• feat: cuda gpu skeleton (#5936) @0ax1
• Add flag to run benchmarks under Samply (#5940) @gatesn
• Datafusion 52 upgrade (#5863) @AdamGS
• Support precision casting for DScalar (#5937) @AdamGS
• feat[array]: use execute in canonical (#5925) @joseph-isaacs
• feat(vx): add JSON output, segments command, and SQL query support (#5642) @palaska
• feat[array]: use execute to return Canonical array not Vector (#5895) @joseph-isaacs
• feature: TableStrategy (#5640) @a10y

🚀 Performance

• Faster primitive take (#6559) @connortsui20
• perf[vortex-btrblocks]: remove listview expansion ratio (#6530) @asubiotto
• VarBinBuilder: eagerly set IsSorted stat on offsets (#6452) @dimitarvdimitrov
• Implement ReduceNode for Array to not require creating new arc when returning children (#6443) @robert3005
• chore: clean cancelled read requests earlier in driver state transitions (#6382) @caicancai
• shared array to avoid duplicate execute (#6453) @onursatici
• vortex-btrblocks: write listviews when elements expand > 50% (#6393) @asubiotto
• Implement ReduceNode for ScalarFnArray to avoid boxing (#6435) @robert3005
• Improve visitor performance by reducing Vec and String allocations (#6337) @AdamGS
• lazy dict layout values eval (#6423) @onursatici
• cuda strategy to use zstd buffers for strings (#6403) @onursatici
• bitunpacking cuda kernels store output into shared memory before copying to main memory (#6384) @robert3005
• chore:optimize next_coalesced() by tracking removed ids in a HashSet (#6353) @caicancai
• constant array metadata for small scalars (#6363) @onursatici
• chore[array]: move over filtering to {reduce/execute}_parent (#6312) @joseph-isaacs
• perf[buf]: inline collect_bool (#6313) @joseph-isaacs
• perf[vortex-array]: avoid expensive list view to list in arrow execution (#6304) @asubiotto
• chore: list contains over scalars (#6242) @blaginin
• perf[arrays]: move over all filter kernels to exec/reduce rules (#6298) @joseph-isaacs
• performance: Use push_unchecked when building indices for element take in fixedsizedlist take (#6222) @robert3005
• perf[vortex-array]: don't materialize const arrays in to_arrow_dictionary (#6265) @asubiotto
• Check read-only cache first (#6249) @gatesn
• Perf: Stream conversion from Parquet to Vortex to prevent OOM (#6228) @connortsui20
• Perf: Use optimized filter implementations is FilterArray::execute (#6189) @connortsui20
• coalesced request align all segments (#6158) @onursatici
• perf: CUDA async to host copy (#6085) @0ax1
• feat: make host to device copy async (#6075) @0ax1
• Remove duplicate ValidityVTable logic (#5975) @gatesn
• performance[vortex-array]: avoid is_valid call in varbinview builder (#6033) @asubiotto
• Only check for ConstantVTable (#6002) @gatesn
• do not validate is_sorted on runend (#6025) @onursatici
• perf: CUDA FoR loop unrolling (#6017) @0ax1
• feat: optimize repeatedScan for limit == 0 (#6015) @caicancai
• Remove unnecessary work from ListArray::validate (#6004) @AdamGS
• scan: make LazyS...

0.59.0-rc.0

Changes

• docs: tidy and unify Python quickstart examples (#5972) @caicancai
• Better vx-bench docs in README (#5900) @AdamGS

⚠️ Breaks

• chore[array]: inline take compute function (#6376) @joseph-isaacs
• Add Array::deserialize arguments (#6340) @gatesn
• Update FSST to store components of compressed codes (#6325) @gatesn
• Array Execution (#6307) @gatesn
• Execute parent to return ArrayRef (#6257) @gatesn
• Extension DType vtables (#6081) @gatesn
• break[expr]: remove Expression.evaluate use Array.apply(expr) instead (#6167) @joseph-isaacs
• Make metrics more local to prevent unbounded memory usage (#6156) @AdamGS
• fallible scalar_at and validity (#6102) @joseph-isaacs
• Remove the python feature from vortex-error (#6064) @AdamGS
• break[*]: make array methods fallible (#6086) @joseph-isaacs
• chore[array]: replace to_canonical with to execute. (#6061) @joseph-isaacs
• Array VTable (#6000) @gatesn
• Remove duplicate ValidityVTable logic (#5975) @gatesn
• chore[array]: remove CanonicalVTable and move append_to_builder to main VTable (#6045) @joseph-isaacs
• Remove EncodeVTable (#6036) @gatesn
• chore: fallible canonical (#6006) @joseph-isaacs
• feat[array]: use execute to return Canonical array not Vector (#5895) @joseph-isaacs

✨ Features

• Make VortexSource::new public (#6392) @AdamGS
• Make projection pushdown in DF an opt-in feature (#6373) @AdamGS
• feat[expr]: new fill null expr (#6385) @joseph-isaacs
• zstd buffers array (#6362) @onursatici
• gpu compatible write strategy, move compact strategy to use btrblocks with zstd and pco (#6322) @onursatici
• cuda dictionary of vbv decompression (#6361) @0ax1
• CLI flag to enable tracing for benchmarks (#6350) @gatesn
• chore[array]: move take kernel to execute/reduce rules (#6310) @joseph-isaacs
• chore[cuda]: add date32 exporter test (#6345) @a10y
• feat: display tree with no metadata (#5898) @danking
• feat[cuda]: device exports for varbinview (#6338) @a10y
• vortex-metrics v2 (#6187) @AdamGS
• feat[array]: dict array allows signed codes (#6336) @joseph-isaacs
• feat[cuda]: export arrays to ArrowDeviceArray (#6253) @a10y
• feat[gpu]: CUDA date_time_parts kernel (#6303) @0ax1
• feat: alternate for StructFields (#6276) @blaginin
• Add a canonical matcher (#6292) @gatesn
• Move over to array matchers (#6291) @gatesn
• feat[gpu]: cuda runend decoding (#6275) @0ax1
• feat[btrblocks]: add a dynamic btr blocks compressor (#6207) @joseph-isaacs
• feat[cuda]: constant array for numeric types (#6248) @joseph-isaacs
• feat[array]: validate array encodings on write (#6241) @joseph-isaacs
• feat[cuda]: SequenceArray (#6232) @a10y
• feat[cuda]: patches kernel (#6231) @a10y
• Pass limit through Python Vortex (#6239) @gatesn
• Ship Vortex CLI as part of Python package (#6233) @gatesn
• Feature: Add more Filter<_> implementations (#6173) @connortsui20
• Benchmarks website revamped (#6215) @palaska
• Dataset API (#5949) @gatesn
• feat[cuda]: filter for Decimal/VarBinView (#6196) @a10y
• feat[cuda]: add a cuda stream pool (#6205) @joseph-isaacs
• feat[python]: support full read/write from object storage (#6022) @a10y
• feat[scan]: gpu scan (#6199) @joseph-isaacs
• feat[gpu]: nvidia cub filter kernel (#6188) @0ax1
• feature: Add fastlanes bit unpacking cuda kernels (#6145) @robert3005
• Use built-in DF cache for file metadata and fix metrics wiring (#6164) @AdamGS
• feat: reduce size of segments containing fixed-size-list arrays (#6180) @danking
• Feature: Use array.filter() instead of filter(array) compute (#6146) @connortsui20
• Rewrite benchmark website (#6160) @palaska
• feat: stats pruning for LIKE filters (#6049) @a10y
• feat: Zstd CUDA nvCOMP decompression (#6136) @0ax1
• feat[cuda]: canonical to_host (#6130) @joseph-isaacs
• feat[cuda]: dict decimal take (#6129) @joseph-isaacs
• feat[gpu]: dict take values prim (#6101) @joseph-isaacs
• chore[cuda]: decimal_byte_parts (#6118) @joseph-isaacs
• feat[gpu]: scalar encodings (#6109) @joseph-isaacs
• Add VortexScanReplacement to support reading from '*.vortex' (#6111) @robert3005
• feat: nvcomp wrapper crate (#6108) @0ax1
• Allow configuring different readers for DF integration (#6027) @AdamGS
• feat: introduce CudaDeviceBuffer (#6060) @0ax1
• Array VTable (#6000) @gatesn
• feat[arary]: add a slice array to replace the slice op. (#5958) @joseph-isaacs
• Attempting to simplify the IO read traits (#5935) @AdamGS
• feat: CUDA compilation infra & FoR kernel (#5966) @0ax1
• feat[array]: execute::<PrimitiveArray>() (#5954) @joseph-isaacs
• Spark 4 (#5944) @a10y
• feat: cuda gpu skeleton (#5936) @0ax1
• Add flag to run benchmarks under Samply (#5940) @gatesn
• Datafusion 52 upgrade (#5863) @AdamGS
• Support precision casting for DScalar (#5937) @AdamGS
• feat[array]: use execute in canonical (#5925) @joseph-isaacs
• feat(vx): add JSON output, segments command, and SQL query support (#5642) @palaska
• feat[array]: use execute to return Canonical array not Vector (#5895) @joseph-isaacs
• feature: TableStrategy (#5640) @a10y

🚀 Performance

• chore:optimize next_coalesced() by tracking removed ids in a HashSet (#6353) @caicancai
• constant array metadata for small scalars (#6363) @onursatici
• chore[array]: move over filtering to {reduce/execute}_parent (#6312) @joseph-isaacs
• perf[buf]: inline collect_bool (#6313) @joseph-isaacs
• perf[vortex-array]: avoid expensive list view to list in arrow execution (#6304) @asubiotto
• chore: list contains over scalars (#6242) @blaginin
• perf[arrays]: move over all filter kernels to exec/reduce rules (#6298) @joseph-isaacs
• performance: Use push_unchecked when building indices for element take in fixedsizedlist take (#6222) @robert3005
• perf[vortex-array]: don't materialize const arrays in to_arrow_dictionary (#6265) @asubiotto
• Check read-only cache first (#6249) @gatesn
• Perf: Stream conversion from Parquet to Vortex to prevent OOM (#6228) @connortsui20
• Perf: Use optimized filter implementations is FilterArray::execute (#6189) @connortsui20
• coalesced request align all segments (#6158) @onursatici
• perf: CUDA async to host copy (#6085) @0ax1
• feat: make host to device copy async (#6075) @0ax1
• Remove duplicate ValidityVTable logic (#5975) @gatesn
• performance[vortex-array]: avoid is_valid call in varbinview builder (#6033) @asubiotto
• Only check for ConstantVTable (#6002) @gatesn
• do not validate is_sorted on runend (#6025) @onursatici
• perf: CUDA FoR loop unrolling (#6017) @0ax1
• feat: optimize repeatedScan for limit == 0 (#6015) @caicancai
• Remove unnecessary work from ListArray::validate (#6004) @AdamGS
• scan: make LazyScanStream initialization non-blocking (#5906) @godnight10061
• perf[sequence]: array constant compare execution kernel (#5795) @joseph-isaacs
• perf: reduce time_elapsed_opening overhead (#5879) @godnight10061

🐛 Bug Fixes

89 changes

• Measure output bytes in cuda bitunpack benchmarks (#6395) @robert3005
• fix[array]: decimal fill_null larger than buffer (#6388) @joseph-isaacs
• fix[array]: dict value nullable values (#6387) @joseph-isaacs
• fix[fuzz]: coverage report (#6375) @joseph-isaacs
• join ops before in format (#6370) @blaginin
• Include config.js in benchmark website docker image (#6379) @palaska
• Exporting struct array to arrow respects vortex nullabilty (#6365) @robert3005
• Fix null take on structs (#6364) @blaginin
• varbinview on device build skips validation (#6358) @onursatici
• fix: vortex-layout doc tests require zstd (#6179) @danking
• DF: Only calculate the size of the metadata cache entry once (#6342) @AdamGS
• bump time version to clear CI (#6335) @onursatici
• All-invalid compressor (#6330) @gatesn
• fix: Empty select (#6314) @gatesn
• Avoid some potential stack overflow cases with deep expression trees (#6321) @AdamGS
• shared array in dict layout (#6305) @onursatici
• skip sparse array validation for device resident arrays (#6315) @onursatici
• varbin to be constructed from buffer handle directly (#6317) @onursatici
• Do not use display_tree in error message (#6251) @gatesn
• Minor memory-related improvements in vortex-array and vortex-duckdb (#6299) @AdamGS
• use is_host to recursively check residency to skip validation (#6297) @onursatici
• Add slice kernel for BitPackedArray execute_parent (#6283) @onursatici
• select preserve validity of fields (#6260) @connortsui20
• fsst array: AllTrue and AllFalse are handled by filter fn (#6272) @blaginin
• fix: separate ptx build profiles (#6252) @0ax1
• fix[cuda]: Fix gpu_scan test (#6208) @a10y
• [benchmark-website] Fix mobile charts, better colors, better tooltip (#6226) @palaska
• fix: exclude flaky benchmarks from codspeed (#6220) @0ax1
• fix: skip buffer validation if on device (#6219) @0ax1
• fix: simd build (#6217) @0ax1
• chore[cuda]: fix windows (#6206) @joseph-isaacs
• Fix: dont do lazy filters on write (#6194) @connortsui20
• fix: evaluate OR for validity (#6184) @blaginin
• fix: cuda ptx folder (#6170) @joseph-isaacs
• Fix: Comment on a failed benchmark (#6181) @connortsui20
• fix[alp]: cast didn't cast patches (#6174) @joseph-isaacs
• fix[fuzz]: fix gpu fuzzer cfg flags (#6165) @joseph-isaacs
• fix[gpu]: synchronous copy to host (#6157) @0ax1
• fix[fuzz]: single sesssion for the fuzzer (#6155) @joseph-isaacs
• fix: sample multiples of 1024 (#6151) @danking
• fix: only link nvcomp for linux (#6144) @0ax1
• fix: fastlanes canonicalize benchmarks (#6141) @0ax1
• fix[fsst]: execute_parent empty validity non empty filter (#6133) @joseph-isaacs
• fix[vortex-array]: fix filter of sliced list (#6113) @asubiotto
• fix[list]: filter list execute parent (#6119) @blaginin
• fix[dtype]: fix typo in conversion from dtype to decimaltype (#6121) @joseph-isaacs
• fix[bitpacking]: filter (#6120) @joseph-isaacs
• fix[vortex-...

0.58.0

Changes

✨ Features

• Feature: Make vortex-tui a library (#5779) @connortsui20

🚀 Performance

• End DataFusion stream early if file can be pruned away by dynamic expr (#5874) @AdamGS

🧰 Maintenance

• chore: add overflow tests for unsigned types in BigCast (#5866) @caicancai

0.57.2

Changes

🐛 Bug Fixes

• fix: Move from macos-13 runners to macos-15-intel runners (#5875) @robert3005