[core] Stream primary-key vector exact fallback one batch at a time

[JunRuiLee]

Purpose

Part of #514, and the first of three PRs split out of the original combined change on the primary-key vector execute_read path. This PR adds streaming exact fallback: it bounds peak memory of the per-file exact rerank to one Arrow batch instead of the whole vector column. Single-query output stays byte-identical.

Stacked PR (base of the stack). Follow-ups build on this branch:

1. this PR — streaming exact fallback
2. batch multi-query — branch pk-vec/2-batch-multi-query (stacked on this)
3. parallel search — [core] Search primary-key vector buckets and files concurrently #556 (stacked on 2)

Brief change log

• perf(table): the per-file exact fallback no longer preloads the whole vector column. It becomes a search closure (Fn + Send + Sync) that streams the column one Arrow batch at a time into per-query bounded top-k heaps, so peak memory is one batch plus the heaps. The streaming loop lives in the table layer; the bucket search only calls the closure and merges its bounded results. Queries are validated before any stream is opened, and the drained row count is checked against the file metadata in both directions.

Tests

• Streaming search matches the whole-column reference (including a NULL row, DV/residual exclusion, and heap eviction); row-count truncation/overrun fail loud.
• cargo test -p paimon green; cargo clippy -p paimon --all-targets -D warnings and cargo fmt --check clean.

API and Format

No on-disk format change, no new result columns, no public API change. Single-query output is byte-identical.

Documentation

Code comments only; no user-facing docs change.

[JingsongLi]

+1