diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3c7b4751..e70b8b4c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -463,7 +463,9 @@ jobs: integration-tests: name: Integration Tests runs-on: ubuntu-latest - timeout-minutes: 20 + # 25 (was 20): the handler package alone may take up to its 15m package + # budget on slow nightly runners — see the -timeout note below. + timeout-minutes: 25 needs: [unit-tests] permissions: contents: read @@ -512,11 +514,17 @@ jobs: - name: Run integration tests working-directory: go run: | + # -timeout is PER PACKAGE. internal/handler legitimately exceeded + # 10m on nightly runners twice (2026-07-26 + 2026-08-02): ~500 + # testcontainer starts across the suite under slow shared-runner + # I/O — the "hanging" test in both panic dumps had run for only + # 2s, it was merely the one on the clock when the package budget + # expired. 15m absorbs runner weather; a REAL hang still fails. go test \ -v \ -tags=integration \ -count=1 \ - -timeout=10m \ + -timeout=15m \ -coverprofile=coverage-integration.out \ -covermode=atomic \ ./... diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 8fbab9e2..9085caf9 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -54,7 +54,14 @@ else's environment — that led to a fix landing in the repository. ""}` string map with no confidence field, losing every link in the cycle; the shipped fix keeps his recover-the-type approach and floor doctrine, hardened with entry discrimination so prose envelopes still - surface as retryable parse errors. + surface as retryable parse errors. Also PR + [#13](https://github.com/GottZ/ctx/pull/13): Voyage AI rerankers answer in + their OpenAPI dialect (`data` container, `usage.total_tokens`), which the + cohere-shaped client decoded to zero results — every Voyage call failed + into the un-reranked fail-open order; the shipped fix keeps his + prefer-results-fall-back-to-data approach, hardened with a probability→ + logit mapping at the wire boundary so Voyage's calibrated [0,1] scores + survive the downstream sigmoid+blend unmangled. ## How to be listed diff --git a/docs/architecture.md b/docs/architecture.md index 968bc29d..ee8457e7 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -247,7 +247,7 @@ On first boot with an empty table, ctxd seeds it from the effective config snaps **Synthesis on the pool chain** (054) — query-path synthesis walks the role chain from `context_backends` (priority-ordered, cooldown-sorted; the chain is the ONLY way to a backend, so the trust gate sits structurally before prompt transmission). Transport-class failures advance to the next backend (e.g. the `llama-cpu` sidecar at priority 10: same GGUF, CPU speed, its own per-role timeout); HTTP-500 and attempt timeouts stop the chain — the server *ran* the request, slow-but-alive is not down. The response heartbeat starts whenever synthesis is on (`synthesize != false`), so a CPU-leg answer survives buffering proxies even with rerank off. Since 055 the WHOLE query path resolves through the chain (translate, temporal, query-embed, rerank dispatch, inline backfill) with a real requirement: `max(query sensitivity, sensitivity of the FINAL prompt set)` — measured after rank filtering, so a credentials block on rank 180 that never enters the prompt cannot lock the failover. An empty dream chain (gaming/disabled/trust) skips the cycle BEFORE the block pick — no claim, no cooldown touch, so a gaming session never smears the back-off statistics. -**Rerank sidecar.** Post-RRF rerank (default-on since Wave 3.5, fail-open) via a local **bge-reranker-v2-m3** cross-encoder sidecar (`http://ctx-rerank:8082`, cohere-style `/v1/rerank`, all-local/$0) or LLM-as-judge on the chat model when `_HOST` is empty. The surface-gold counter-probe showed the cross-encoder earns its keep (nDCG@10 +0.164, MRR +0.169) while blend 0.5 keeps it neutral on latent gold — `graph+ce-bw0.5` is the best arm on both gold sets; the ~80–90s query path stays proxy-safe via the body heartbeat. +**Rerank sidecar.** Post-RRF rerank (default-on since Wave 3.5, fail-open) via a local **bge-reranker-v2-m3** cross-encoder sidecar (`http://ctx-rerank:8082`, cohere-style `/v1/rerank`, all-local/$0) or LLM-as-judge on the chat model when `_HOST` is empty. Voyage AI rerankers (`rerank-2.5`/`-lite`) work as a remote alternative: the client accepts their OpenAPI dialect — results under `data` instead of `results`, `usage.total_tokens` instead of `prompt_tokens` — and maps Voyage's calibrated [0,1] relevance through logit() at the wire boundary, so the raw-logit score contract (and the downstream sigmoid+blend) is backend-invariant. Container name and score domain are coupled backend properties; a `data` score outside [0,1] or an entry missing `relevance_score` errors → fail-open to the pre-rerank order. The surface-gold counter-probe showed the cross-encoder earns its keep (nDCG@10 +0.164, MRR +0.169) while blend 0.5 keeps it neutral on latent gold — `graph+ce-bw0.5` is the best arm on both gold sets; the ~80–90s query path stays proxy-safe via the body heartbeat. ## Supporting mechanisms diff --git a/docs/operations.md b/docs/operations.md index 1887751f..ea4a55d4 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -99,7 +99,7 @@ The **`mut` column** is the mutability class per key: **hot** keys take effect w | `CTX_GRAPH_EXPAND_ENABLED` / `_*` | `true` | hot | Query-time Dream-graph traversal (default-on since Wave 3). Fail-open. Knobs: `_DIRECTED` / `_HOP_DEPTH` / `_SEED_COUNT` / `_SEED_SCORE_FLOOR` / `_PER_SEED_CAP` / `_MAX_INJECTED` / `_MIN_CONFIDENCE`(`_RECURRENT`) / `_BOOST_WEIGHT` / `_HUB_DAMPING` / `_WEIGHT_{TOPICAL,FACTUAL,CAUSAL,RECURRENT}` / `_NEW_PLACEMENT_FRAC` | | `CTX_GRAPH_OVERVIEW_ENABLED` / `_REBUILD_INTERVAL` / `_RESOLUTION` / `_MAX_NODES` / `_REBUILD_TIMEOUT` | `false` / `21600` (s) / `1.0` / `200000` / `900` (s) | hot | Louvain cluster supergraph ("landkarte", `GET /api/graph/overview`): scheduler rebuild gate, cadence and resolution. `_MAX_NODES` is the load-bearing liveness cap (parse-strict): a node set beyond it skips the rebuild fail-safe — Louvain hits a convergence wall past ~200k nodes (`0` = uncapped). `_REBUILD_TIMEOUT` bounds one run. Since wave E-A the rebuild runs in a worker child process (the hidden `ctxd overview-rebuild-worker` subcommand, Options in via stdin / Stats out via stdout, strict JSON both ways): on timeout the child is killed (SIGKILL; the persist tx rolls back cleanly) instead of orphaning a compute goroutine. Since wave E-B the worker self-deprioritizes at entry to nice 19 + idle I/O class (best-effort, WARN-never-fatal) so a minutes-long Louvain run never competes with the daemon or GPU-inference paths. When the worker cannot be *started* (missing binary, exec denied) the rebuild falls back in-process with a WARN — that legacy path still finishes an abandoned goroutine in the background on expiry while the scheduler loop stays live. The rebuild also defers to active queries (demand interruption) | | `CTX_GRAPH_CACHE_ENABLED` / `_REBUILD_INTERVAL` / `_DEBOUNCE_WINDOW` / `_MIN_REBUILD_INTERVAL` / `_MAX_PENDING_AGE` / `_MAX_STALENESS` / `_FAILED_THRESHOLD` / `_DEGREE_WALK_BUDGET` / `_SERVE_EGO` / `_SERVE_EXPAND` | `false` / `21600` (s) / `60` (s) / `300` (s) / `600` (s) / `900` (s) / `3` / `4000` / `false` / `false` | hot | Settings group `graph_cache.*` (Achse 05): the in-process CSR graph cache and its rebuild job. `_ENABLED` is the master gate — while it is off the rebuild goroutine stays inert and every consumer sees an empty snapshot, i.e. its SQL path; a hot enable triggers a boot-build on the next loop. Cadence: `_REBUILD_INTERVAL` is the unconditional hard rebuild measured from the last build *start* (it covers missed NOTIFYs from migration 116), while a dirty signal drives a rebuild once it has been quiet for `_DEBOUNCE_WINDOW`, no more often than `_MIN_REBUILD_INTERVAL` (rebuild-storm brake), and no later than `_MAX_PENDING_AGE` after the oldest unconsumed signal (starvation bound). `_MAX_STALENESS` bounds the maximum cache lie: it is measured as **Dirty-Age** (now − oldest pending signal), never build age, so an idle database stays Fresh indefinitely instead of spending most of a quiet cycle Degraded; past it the automaton degrades and consumers fall back to SQL. `_FAILED_THRESHOLD` consecutive build errors turn the state Failed (status red). `_DEGREE_WALK_BUDGET` caps the hint-filtered degree walk of the Ego arm — past it the reported degree is a lower bound, not a truncation of results. `_SERVE_EGO` / `_SERVE_EXPAND` are the two consumer cut-overs (`store.EgoGraph` hops/edges/degrees resp. `rrf.GraphExpand`): both engines are **built and dark**, activation is the E-05-1 decision after the W05.8 bench, not a deploy. Durations are bare seconds (no suffix) | -| `CTX_DISPATCH_ENABLED` / `_BACKGROUND_QUEUE_MAX` / `_INTERACTIVE_QUEUE_PER_PRINCIPAL` / `_INTERACTIVE_QUEUE_PER_TENANT` / `_INTERACTIVE_QUEUE_MAX` / `_LEASE_REAP_GRACE` / `_LEASE_MAX_AGE` / `_PREEMPT_RELEASE_TIMEOUT` / `_BACKGROUND_AGING_AFTER` | `true` / `32` / `8` / `16` / `64` / `30` (s) / `900` (s) / `2` (s) / `0` (s, off) | hot | Inference dispatch admission layer (`internal/dispatch`, Vorhaben E wave MW1): per-origin lease queue with two classes (interactive > background, FIFO per class, herald term), the cap ladder principal→tenant→aggregate, and the lease reaper (grace past the request deadline; `_LEASE_MAX_AGE` is the fallback for deadline-less contexts). `_ENABLED=false` is the kill switch (pass-through). `_PREEMPT_RELEASE_TIMEOUT` (wave MW18) is the preempt watchdog: a background victim canceled for interactive demand whose release stays out past it is force-released (third legitimate release path, ERROR + divergence counter). Wave MW19 pins the consumer-side preempt semantics in the dream pipeline: a preempted eval takes the plain transient cooldown (~5 min, `dream_eval_count` untouched — a preempt is a scheduling decision, not completed work) and never spills to the next backend or reports health; a preempted recurrence pair is a per-pair non-fatal skip (verdict lost, candidate loop continues, next pair re-acquires) — detected exclusively via `errors.Is` on `dispatch.ErrPreempted` over the returned error chain. All keys global-only. The `Enforcing()` predicate (wave MW14) reports whether the layer actually enforces — enabled AND non-empty policy AND every background-reachable local target slot-capped; a partial policy WARNs per uncovered origin and keeps the legacy yield fallbacks active (waves MW15–MW17 consume this). Wave MW22 adds the fairness usage meter: per HomeScope×origin sliding window charging TOKENS (prompt+completion from backend usage, decision E-F3/C1; missing usage charges 0 and bumps an `uncharged_calls` counter — visibility over estimation). Measurement only; the budget consumer arrives with MW23/MW24. The demand herald (wave MW2) counts interactive demand from HTTP ingress — the query mounts and `/api/synthesize/daily` (third entry) raise it; since wave MW15 (A5-W0) the herald is the ONLY demand signal — the scheduler’s `activeQueries` mirror is gone, the `SchedulerNotifier` seam is the dispatcher’s idempotent `InteractiveArrived() func()` closure, and all five LLM-free consumers (guard, digest, overview, dream, audit) defer on `InteractiveDemand()` (a running daily call now visibly defers a guard batch — the closed inventory gap). Wave MW16 (A5-W2) retires the audit arm’s 2 s self-polling under `Enforcing()`: the classify call waits AT THE TARGET in its background lease instead; with enforcement off the legacy 2 s wait-loop is byte-identical (P-Fallback), and the kill switch stays safe because `UpdateSettings`/`UpdatePolicy` synchronously wake and pass-through every parked waiter (pinned by the drain tests — the design’s §9 wake-on-flip concern is resolved in code). Wave MW17 (A5-W3) does the same for the dream arm and adds the vor-pick demand check: under `Enforcing()` a cycle launch is skipped (loop cadence, non-blocking) while interactive demand stands, so the `PickBlock` transient claim and the error-path `dream_checked_at` stamp never rotate without work under sustained load — wire calls of an already-running cycle keep waiting in their leases, and with enforcement off the legacy 2 s dream yield loop stays byte-identical — and the dispatcher is constructed, policy-derived and hot-reloaded from boot (settings NOTIFY + backend-pool reload). `_BACKGROUND_AGING_AFTER` (wave MW25, default 0 = off) is the starvation escape: a background waiter aged past it may pass the herald term once — never past a waiting interactive, and never on an interactive-role target without `preempt_background` (the E-F5 coupling invariant); aged admits/preempts count per target as the activation gate's inputs. Wave MW7 adds admitted-wait aggregates per target×class (K=512 ring, p95/max/samples in the snapshot) — the empirical inputs of the MW13 activation gate. Wave MW9 (Q-I3) moves the scheduler backfill's lease BEFORE its DB transaction (no admission wait under a held row lock) — follow-up chain links go through a non-blocking TryAcquire and defer to the next cycle when busy. Wave MW11 extends the 091 telemetry columns to every remaining pipeline (stream, embed both roles, rerank, and the five dream sites through ONE funnel) — including the K9 rejection rows for the background embed sites, so backfill starvation under permanent interactive demand is visible instead of producing zero rows. Since wave MW3 every non-stream chain call (translate, temporal, rerank-judge, classify/audit, synthesize, dream router, daily) acquires a per-attempt lease before the wire call — interactive from the auth principal, background for the autonomous loops; since wave MW4 the interactive class is a ctx-bound privilege (the principal derives from the authenticated request context, there is no principal parameter — an interactive request without one downgrades to background, counted) — a rejected acquire is terminal (no attempt, no classify, no health report), and `/api/synthesize/daily` carries a concurrency cap of 1 per principal (429 fail-fast; the E-U2 coupling that keeps the daily path interactive). Wave MW8 maps interactive rejections to the client: HTTP 429 with a `Retry-After` header (decision B1 — the estimate is clamp(queue-depth × p95-wait, 1 s..30 s) from the MW7 wait window via the optional `RetryHinter` capability; an unknown origin or the E-U2 concurrency 429 honestly OMITS the header instead of fabricating a value), the error body stays B6-generic; a waiting stream emits a `queued` SSE keepalive every 20 s and a mid-stream saturation surfaces as a backend-nameless `saturated` event. The chat SPA consumes both since wave MW8b (decision B2): a queue indicator with a cancel option while `queued` events arrive, and a saturation card on a pre-stream 429 — with a jittered countdown + auto-retry when `Retry-After` is present, a generic manual-retry state when it is honestly absent. The synthesis path’s heartbeat-200 makes a Retry-After header inert there — the body carries the signal (documented boundary). Wave MW10 (migration 091) persists the lease telemetry on the non-stream rows — `queue_wait_ms`/`dispatch_class`/`dispatch_abort` in `context_llm_log`, a wait-free `duration_ms`, and the K9 rejection line for never-admitted background acquires (`acquire_expired`/`queue_full`, `duration_ms` NULL) — pure telemetry, deliberately wired BEFORE the data activation so the MW13 gate rates against persisted waits instead of WARN lines (details: [architecture](architecture.md), LLM log). Wave MW12 (K13) exposes the admission registry on the status surface: the server-admin path carries the FULL dispatch section (per-target queue depths, MW7 wait aggregates, preempt/aged counters, last guard/digest/overview run stamps, and the D1a embed-token 24h rollup fed by the serving attempt’s prompt_tokens now persisted on embed llmlog rows), while the tenant path gets a structurally separate, coarsened shape — no fair-key field on the struct (F-B3), queue depth bucketed leer/niedrig/hoch (E-A5-6b) — and the llmlog list handler exposes the three 091 telemetry columns (lists stay body-free). Wave MW12b adds the D1b consumption: `GET /api/llmlog/{id}` is the strictly gated prompt/reply detail fetch (server-admin any row, tenant-admin only rows attributed to its own keys; foreign/unknown/malformed ids answer a uniform 404 — no existence oracle; `body_state` present/sealed/evicted never leaks a sealed or evicted body), and the SPA renders the dispatch tile (both shapes incl. the D1a embed-token table) plus the clickable history row → detail card (no body caching, fetch dropped on close). Since wave MW5 the remaining wire sites are attached too — chat stream (lease spans the whole stream, usage charged from the end-of-stream usage), embed incl. the query-path backfill (cache hits acquire NO lease; backfill is interactive per E-U5(a), the overtake relief is the per-attempt acquire) and the rerank cross-encoder — so ALL 6 wire sites run under a lease. **Behaviour-neutral beyond that until `slots` policy rows are activated (MW13); preemption additionally needs `preempt_background=true` on the target row (MW21, only after the MW20 measurement gate)** — with an empty policy every acquire is a pass-through | +| `CTX_DISPATCH_ENABLED` / `_BACKGROUND_QUEUE_MAX` / `_INTERACTIVE_QUEUE_PER_PRINCIPAL` / `_INTERACTIVE_QUEUE_PER_TENANT` / `_INTERACTIVE_QUEUE_MAX` / `_LEASE_REAP_GRACE` / `_LEASE_MAX_AGE` / `_PREEMPT_RELEASE_TIMEOUT` / `_BACKGROUND_AGING_AFTER` | `true` / `32` / `8` / `16` / `64` / `30` (s) / `900` (s) / `2` (s) / `0` (s, off) | hot | Inference dispatch admission layer (`internal/dispatch`, Vorhaben E wave MW1): per-origin lease queue with two classes (interactive > background, FIFO per class, herald term), the cap ladder principal→tenant→aggregate, and the lease reaper (grace past the request deadline; `_LEASE_MAX_AGE` is the fallback for deadline-less contexts). `_ENABLED=false` is the kill switch (pass-through). `_PREEMPT_RELEASE_TIMEOUT` (wave MW18) is the preempt watchdog: a background victim canceled for interactive demand whose release stays out past it is force-released (third legitimate release path, ERROR + divergence counter). Wave MW19 pins the consumer-side preempt semantics in the dream pipeline: a preempted eval takes the plain transient cooldown (~5 min, `dream_eval_count` untouched — a preempt is a scheduling decision, not completed work) and never spills to the next backend or reports health; a preempted recurrence pair is a per-pair non-fatal skip (verdict lost, candidate loop continues, next pair re-acquires) — detected exclusively via `errors.Is` on `dispatch.ErrPreempted` over the returned error chain. All keys global-only. The `Enforcing()` predicate (wave MW14) reports whether the layer actually enforces — enabled AND non-empty policy AND every background-reachable local target slot-capped; a partial policy WARNs per uncovered origin and keeps the legacy yield fallbacks active (waves MW15–MW17 consume this). Wave MW22 adds the fairness usage meter: per HomeScope×origin sliding window charging TOKENS (prompt+completion from backend usage, decision E-F3/C1; missing usage charges 0 and bumps an `uncharged_calls` counter — visibility over estimation; rerank backends reporting only `usage.total_tokens` (Voyage) are charged that measured value instead of counting as uncharged — a one-time INFO line marks the switch). Measurement only; the budget consumer arrives with MW23/MW24. The demand herald (wave MW2) counts interactive demand from HTTP ingress — the query mounts and `/api/synthesize/daily` (third entry) raise it; since wave MW15 (A5-W0) the herald is the ONLY demand signal — the scheduler’s `activeQueries` mirror is gone, the `SchedulerNotifier` seam is the dispatcher’s idempotent `InteractiveArrived() func()` closure, and all five LLM-free consumers (guard, digest, overview, dream, audit) defer on `InteractiveDemand()` (a running daily call now visibly defers a guard batch — the closed inventory gap). Wave MW16 (A5-W2) retires the audit arm’s 2 s self-polling under `Enforcing()`: the classify call waits AT THE TARGET in its background lease instead; with enforcement off the legacy 2 s wait-loop is byte-identical (P-Fallback), and the kill switch stays safe because `UpdateSettings`/`UpdatePolicy` synchronously wake and pass-through every parked waiter (pinned by the drain tests — the design’s §9 wake-on-flip concern is resolved in code). Wave MW17 (A5-W3) does the same for the dream arm and adds the vor-pick demand check: under `Enforcing()` a cycle launch is skipped (loop cadence, non-blocking) while interactive demand stands, so the `PickBlock` transient claim and the error-path `dream_checked_at` stamp never rotate without work under sustained load — wire calls of an already-running cycle keep waiting in their leases, and with enforcement off the legacy 2 s dream yield loop stays byte-identical — and the dispatcher is constructed, policy-derived and hot-reloaded from boot (settings NOTIFY + backend-pool reload). `_BACKGROUND_AGING_AFTER` (wave MW25, default 0 = off) is the starvation escape: a background waiter aged past it may pass the herald term once — never past a waiting interactive, and never on an interactive-role target without `preempt_background` (the E-F5 coupling invariant); aged admits/preempts count per target as the activation gate's inputs. Wave MW7 adds admitted-wait aggregates per target×class (K=512 ring, p95/max/samples in the snapshot) — the empirical inputs of the MW13 activation gate. Wave MW9 (Q-I3) moves the scheduler backfill's lease BEFORE its DB transaction (no admission wait under a held row lock) — follow-up chain links go through a non-blocking TryAcquire and defer to the next cycle when busy. Wave MW11 extends the 091 telemetry columns to every remaining pipeline (stream, embed both roles, rerank, and the five dream sites through ONE funnel) — including the K9 rejection rows for the background embed sites, so backfill starvation under permanent interactive demand is visible instead of producing zero rows. Since wave MW3 every non-stream chain call (translate, temporal, rerank-judge, classify/audit, synthesize, dream router, daily) acquires a per-attempt lease before the wire call — interactive from the auth principal, background for the autonomous loops; since wave MW4 the interactive class is a ctx-bound privilege (the principal derives from the authenticated request context, there is no principal parameter — an interactive request without one downgrades to background, counted) — a rejected acquire is terminal (no attempt, no classify, no health report), and `/api/synthesize/daily` carries a concurrency cap of 1 per principal (429 fail-fast; the E-U2 coupling that keeps the daily path interactive). Wave MW8 maps interactive rejections to the client: HTTP 429 with a `Retry-After` header (decision B1 — the estimate is clamp(queue-depth × p95-wait, 1 s..30 s) from the MW7 wait window via the optional `RetryHinter` capability; an unknown origin or the E-U2 concurrency 429 honestly OMITS the header instead of fabricating a value), the error body stays B6-generic; a waiting stream emits a `queued` SSE keepalive every 20 s and a mid-stream saturation surfaces as a backend-nameless `saturated` event. The chat SPA consumes both since wave MW8b (decision B2): a queue indicator with a cancel option while `queued` events arrive, and a saturation card on a pre-stream 429 — with a jittered countdown + auto-retry when `Retry-After` is present, a generic manual-retry state when it is honestly absent. The synthesis path’s heartbeat-200 makes a Retry-After header inert there — the body carries the signal (documented boundary). Wave MW10 (migration 091) persists the lease telemetry on the non-stream rows — `queue_wait_ms`/`dispatch_class`/`dispatch_abort` in `context_llm_log`, a wait-free `duration_ms`, and the K9 rejection line for never-admitted background acquires (`acquire_expired`/`queue_full`, `duration_ms` NULL) — pure telemetry, deliberately wired BEFORE the data activation so the MW13 gate rates against persisted waits instead of WARN lines (details: [architecture](architecture.md), LLM log). Wave MW12 (K13) exposes the admission registry on the status surface: the server-admin path carries the FULL dispatch section (per-target queue depths, MW7 wait aggregates, preempt/aged counters, last guard/digest/overview run stamps, and the D1a embed-token 24h rollup fed by the serving attempt’s prompt_tokens now persisted on embed llmlog rows), while the tenant path gets a structurally separate, coarsened shape — no fair-key field on the struct (F-B3), queue depth bucketed leer/niedrig/hoch (E-A5-6b) — and the llmlog list handler exposes the three 091 telemetry columns (lists stay body-free). Wave MW12b adds the D1b consumption: `GET /api/llmlog/{id}` is the strictly gated prompt/reply detail fetch (server-admin any row, tenant-admin only rows attributed to its own keys; foreign/unknown/malformed ids answer a uniform 404 — no existence oracle; `body_state` present/sealed/evicted never leaks a sealed or evicted body), and the SPA renders the dispatch tile (both shapes incl. the D1a embed-token table) plus the clickable history row → detail card (no body caching, fetch dropped on close). Since wave MW5 the remaining wire sites are attached too — chat stream (lease spans the whole stream, usage charged from the end-of-stream usage), embed incl. the query-path backfill (cache hits acquire NO lease; backfill is interactive per E-U5(a), the overtake relief is the per-attempt acquire) and the rerank cross-encoder — so ALL 6 wire sites run under a lease. **Behaviour-neutral beyond that until `slots` policy rows are activated (MW13); preemption additionally needs `preempt_background=true` on the target row (MW21, only after the MW20 measurement gate)** — with an empty policy every acquire is a pass-through | | `CTX_RERANK_ENABLED` / `_HOST` / `_*` | `true` | hot | Post-RRF rerank (default-on since Wave 3.5, fail-open). `_HOST` / `_MODEL` / `_API_KEY` are **Bootstrap-only since 053** (seed the `herbert-rerank` row, then inert); `_ENABLED` / `_MAX_DOCS` / `_BLEND_WEIGHT` stay live query knobs. `_HOST` empty → LLM-as-judge on the chat model; default `http://ctx-rerank:8082` → local bge-reranker-v2-m3 sidecar. `_MAX_DOCS` (default 50; CPU ≈1s/doc), `_BLEND_WEIGHT` (default 0.5; 1.0 = pure cross-encoder). See `docker-compose.yml` for the sidecar | | `CTX_RETRIEVAL_SELECTOR_ENABLED` / `_EXACT_MAX` / `_GREY_MAX` / `_GREY_SCAN_TUPLES` / `_STATS_TTL` | `false` / `4096` / `65536` / `60000` / `60` (s) | hot | Settings group `retrieval.selector.*` (Achse 02): the per-request dispatch between the two arms of the migration-112 dual-arm semantic CTE (exact brute force vs. HNSW). `_ENABLED` is the master gate and **ships false** — while it is closed the query path issues the legacy statement, runs no cardinality probe, and neither the `rrf search complete` log line nor the `context_access_log` metadata differ by a byte from the pre-selector state; the gate opens as a settings flip once the Achse-01 recall evidence is in, not as a deploy. Armed, a LIMIT-capped index probe decides: estimate ≤ `_EXACT_MAX` → exact arm (the probe limit and the SQL cap guard are the same value); above it a TTL-cached pg_stats estimate ≤ `_GREY_MAX` → grey zone, i.e. the ANN arm budgeted with `hnsw.max_scan_tuples = _GREY_SCAN_TUPLES`; beyond that plain ANN. `_STATS_TTL` is the age of that pg_stats snapshot (bare seconds); a snapshot older than 10× it, a never-analysed table or a failed probe degrade loudly to plain ANN — a failing estimate costs the strategy, never the query. **Policy is data, its range is code:** `_EXACT_MAX` is clamped to [64, 65536] and `_GREY_SCAN_TUPLES` to [1000, 200000] with a WARN (the ceiling is mirrored as a `RAISE` in the migration-112 SQL body); an out-of-range value is clamped, never rejected, so a hot reload cannot break the query path. `_GREY_MAX` is a pure comparison threshold and is not clamped | | `CTX_RECALL_CHECK_ENABLED` / `_INTERVAL` / `_OFFPEAK_HOUR` / `_K_LIST` / `_QUERIES` / `_STRATA_BOUNDS` / `_EXACT_BUDGET_MS` / `_EXACT_TOUCH_BUDGET` / `_LEG_TIMEOUT_MS` / `_PARK_MAX_MS` / `_EF_SEARCH` / `_EPSILON` / `_RETENTION_DAYS` | `true` / `86400` (s) / `4` / `10,75` / `20` / `4096,65536` / `300000` / `0` (auto) / `60000` / `600000` / `0` / `0` / `365` | hot | Settings group `recall_check.*` (Achse 01): the scheduler arm that measures how closely the production HNSW index tracks a brute-force reference (`context_recall_runs`, aggregates only). `_ENABLED` **defaults on** — "erst messen"; a default-off measuring system recreates the very blind spot the axis closes — and a disabled arm writes no row at all. Cadence: the cheap strata run on `_INTERVAL`, the expensive strata anchor to the local wall-clock hour `_OFFPEAK_HOUR` (4, deliberately offset from the 03:00 daily synthesis) with one expensive stratum per off-peak run, round-robin. There is no boot run. `_K_LIST` is the measured k set (10 for comparability with the R@10 history, 75 = the productive semantic window), `_QUERIES` the target sample size per stratum (settings key `queries_per_stratum`; a budget may cut it, floor 5 — below that the row refuses to record a pseudo-statistic), `_STRATA_BOUNDS` the class boundaries `small,medium` pinned to the selector thresholds above so the measurement calibrates the dispatch buckets directly. Two budget dimensions bound one run: `_EXACT_BUDGET_MS` is the wall clock over all exact legs including stratification, and `_EXACT_TOUCH_BUDGET` (settings key `exact_touch_budget_bytes`) the heap+TOAST read volume — `0` means auto-derive 25 % of the live `shared_buffers`, and a failed resolution degrades to touch-unbounded with a WARN rather than skipping the measurement. `_LEG_TIMEOUT_MS` is the hard single-leg cap so one large leg cannot eat the rest. Interactive demand defers a launch entirely; demand arriving mid-run parks it before each probe for up to `_PARK_MAX_MS`, then the remaining rows abort visibly as `demand_deferred`. `_EF_SEARCH` (`0` = pgvector default 40) tunes the ANN leg for live probes without a deploy, `_EPSILON` is the tie tolerance of the distance-based recall definition (hit = ANN distance ≤ last-exact distance + ε) and is stamped into every row's `meta`. `_RETENTION_DAYS` is the janitor horizon. Every row also stamps the pgvector/PG version and the HNSW indexdef, so a series stays interpretable across extension bumps and index rebuilds | diff --git a/go/internal/rerank/rerank.go b/go/internal/rerank/rerank.go index a2998c12..f2aca7f4 100644 --- a/go/internal/rerank/rerank.go +++ b/go/internal/rerank/rerank.go @@ -15,6 +15,13 @@ // 0-1 probability. Score() returns it verbatim; the caller decides whether // to sigmoid / normalize (the rrf layer does, see RerankCrossEncoder). // +// Voyage AI (rerank-2.5/-lite) is supported as an alternative backend. Its +// wire format differs in two coupled ways: results live under "data" (OpenAPI +// RerankingObject) and relevance_score is a calibrated [0,1] relevance — not +// a logit. Score() maps those probabilities through the logit function at the +// wire boundary so the return contract above (raw logits) holds for every +// backend and the caller's sigmoid reconstructs the original probability. +// // Source: https://github.com/GottZ/ctx package rerank @@ -24,7 +31,10 @@ import ( "encoding/json" "fmt" "io" + "log/slog" + "math" "net/http" + "sync" "time" "github.com/GottZ/ctx/internal/httpx" @@ -56,18 +66,31 @@ type rerankRequest struct { } type rerankResult struct { - Index int `json:"index"` - RelevanceScore float64 `json:"relevance_score"` + Index int `json:"index"` + // RelevanceScore is a pointer so an absent field is distinguishable + // from a legitimate 0.0: sibling rerank dialects name this field + // "score" (mixedbread, some gateways), and silently scoring every + // document 0 would neutralize the reranker while telemetry reports a + // successful call. Absent ⇒ error ⇒ caller fails open. + RelevanceScore *float64 `json:"relevance_score"` } type rerankResponse struct { Results []rerankResult `json:"results"` + // Voyage AI returns results under "data" instead of "results" + // (OpenAPI RerankingObject schema). Kept raw and decoded lazily: + // "data" is also the generic OpenAI-style list container, so a + // backend serving valid "results" alongside a non-array "data" + // field must not fail the whole decode over a field we ignore. + Data json.RawMessage `json:"data"` // Usage is llama.cpp's Jina-style token accounting on the rerank // response. prompt_tokens covers ALL query+document pairs of the request // — the MW22/C1 charge dimension of the sidecar call. Absent field ⇒ 0 ⇒ // the call site charges nothing (uncharged, never an estimate). + // Voyage reports total_tokens instead of prompt_tokens. Usage struct { PromptTokens int `json:"prompt_tokens"` + TotalTokens int `json:"total_tokens"` } `json:"usage"` } @@ -113,11 +136,42 @@ func Score(ctx context.Context, host, apiKey, model, query string, docs []string return nil, 0, fmt.Errorf("rerank: %w", httpx.NewStatusError(resp, errBody)) } + // Buffer the body (rerank responses are a few KB — MAX_DOCS entries of + // ~100 bytes; 1 MiB is a generous ceiling) so an unrecognized dialect + // can echo a snippet of what the server actually sent. The original + // Voyage incident hid behind a generic "count mismatch" for six + // production calls because the response was decoded blind. + raw, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) + if err != nil { + return nil, 0, fmt.Errorf("rerank: read body: %w", err) + } var result rerankResponse - if err := json.NewDecoder(resp.Body).Decode(&result); err != nil { + if err := json.Unmarshal(raw, &result); err != nil { return nil, 0, fmt.Errorf("rerank: decode: %w", err) } + // Voyage AI returns results under "data" (OpenAPI RerankingObject), + // while cohere/llama.cpp uses "results". Prefer "results"; fall back + // to "data" when "results" is absent. The container name is coupled to + // the score domain: "data" backends (Voyage) report calibrated [0,1] + // relevance, "results" backends (cohere/llama.cpp) report raw logits. + entries := result.Results + fromData := false + if len(entries) == 0 && len(result.Data) > 0 { + if err := json.Unmarshal(result.Data, &entries); err != nil { + return nil, 0, fmt.Errorf("rerank: decode data: %w; body: %.256s", err, raw) + } + fromData = true + } + if len(entries) == 0 { + // Neither container decoded any entries: an unknown backend + // dialect, a gateway error object behind HTTP 200, or a renamed + // results field. Name the schema break and echo a body snippet — + // "count mismatch" would misdiagnose this as a counting problem + // (the exact failure mode that masked the Voyage incident). + return nil, 0, fmt.Errorf("rerank: response contains neither \"results\" nor \"data\" entries (unknown backend dialect?); body: %.256s", raw) + } + // Re-align strictly by result.Index into input order. The server sorts // results by score descending (and may resize to top_n), so array position // is meaningless — only Index maps a score back to its document. Validate @@ -126,7 +180,7 @@ func Score(ctx context.Context, host, apiKey, model, query string, docs []string scores := make([]float64, len(docs)) seen := make([]bool, len(docs)) got := 0 - for _, r := range result.Results { + for _, r := range entries { if r.Index < 0 || r.Index >= len(docs) { return nil, 0, fmt.Errorf("rerank: result index %d out of range [0,%d)", r.Index, len(docs)) } @@ -134,12 +188,60 @@ func Score(ctx context.Context, host, apiKey, model, query string, docs []string return nil, 0, fmt.Errorf("rerank: duplicate result index %d", r.Index) } seen[r.Index] = true - scores[r.Index] = r.RelevanceScore + if r.RelevanceScore == nil { + return nil, 0, fmt.Errorf("rerank: result index %d missing relevance_score (unknown score field name?)", r.Index) + } + score := *r.RelevanceScore + if fromData { + // Voyage's calibrated [0,1] relevance would be sigmoided a + // second time by the caller (rrf treats every score as a + // logit), compressing the signal into [0.5,0.73] and letting + // RRF outvote the reranker at blend_weight < 1. Map the + // probability to its logit here so the downstream sigmoid + // reconstructs it exactly. A score outside [0,1] breaks the + // documented Voyage schema — error → caller fails open. + if score < 0 || score > 1 { + return nil, 0, fmt.Errorf("rerank: data score %g at index %d outside [0,1] (voyage schema violation)", score, r.Index) + } + score = probabilityToLogit(score) + } + scores[r.Index] = score got++ } if got != len(docs) { return nil, 0, fmt.Errorf("rerank: result count mismatch: got %d scores for %d documents", got, len(docs)) } - return scores, result.Usage.PromptTokens, nil + // Voyage reports total_tokens; llama.cpp reports prompt_tokens. + promptTokens := result.Usage.PromptTokens + if promptTokens == 0 && result.Usage.TotalTokens > 0 { + promptTokens = result.Usage.TotalTokens + // This flips the MW22 meter semantics for total_tokens-only + // backends from "uncharged" to a real measured charge — make + // the switch findable in the log, once per process (rerank + // runs per query; per-call INFO would be noise). + totalTokensFallbackOnce.Do(func() { + slog.Info("rerank: backend reports usage.total_tokens only; charging it as prompt tokens (previously uncharged)") + }) + } + + return scores, promptTokens, nil +} + +var totalTokensFallbackOnce sync.Once + +// probabilityToLogit maps a calibrated probability to its logit, +// logit(p) = ln(p/(1-p)), the exact inverse of the caller's sigmoid. +// p is clamped away from the exact endpoints so 0 and 1 (legitimately +// producible by a quantized probability) yield large finite logits +// instead of ±Inf, which would poison the downstream blend arithmetic. +func probabilityToLogit(p float64) float64 { + const eps = 1e-7 + if p < eps { + p = eps + } + if p > 1-eps { + p = 1 - eps + } + return math.Log(p / (1 - p)) } diff --git a/go/internal/rerank/rerank_test.go b/go/internal/rerank/rerank_test.go index 6e4a1c1c..3076f914 100644 --- a/go/internal/rerank/rerank_test.go +++ b/go/internal/rerank/rerank_test.go @@ -3,8 +3,10 @@ package rerank import ( "context" "encoding/json" + "math" "net/http" "net/http/httptest" + "strings" "testing" ) @@ -128,3 +130,394 @@ func TestScore_EmptyDocs(t *testing.T) { t.Errorf("empty docs: got (%v, %v), want (nil, nil)", scores, err) } } + +// Voyage AI format compatibility tests. + +// logit is the test-side mirror of probabilityToLogit: the expected wire +// transform for Voyage's calibrated [0,1] scores (no clamping — tests use +// interior probabilities). +func logit(p float64) float64 { + return math.Log(p / (1 - p)) +} + +// voyageResults builds a Voyage-style response body: results under "data" +// (OpenAPI RerankingObject schema) with "total_tokens" usage. +func voyageResults(rs ...map[string]any) map[string]any { + return map[string]any{ + "object": "list", + "data": rs, + "model": "rerank-2.5", + "usage": map[string]any{"total_tokens": 42}, + } +} + +// TestScore_VoyageDataFallback is the load-bearing regression test: Voyage AI +// returns results under "data" instead of "results". Before the fix, Score +// decoded an empty Results slice and failed with "got 0 scores for N documents". +func TestScore_VoyageDataFallback(t *testing.T) { + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + return http.StatusOK, voyageResults( + map[string]any{"index": 0, "relevance_score": 0.95}, + map[string]any{"index": 1, "relevance_score": 0.42}, + map[string]any{"index": 2, "relevance_score": 0.11}, + ) + }) + + scores, ptoks, err := Score(context.Background(), srv.URL, "", "rerank-2.5", "q", []string{"a", "b", "c"}) + if err != nil { + t.Fatalf("Voyage data fallback failed: %v", err) + } + // Voyage scores are calibrated [0,1] probabilities; Score() maps them + // to logits at the wire boundary so its raw-logit return contract + // holds for every backend (the caller's sigmoid reconstructs p). + want := []float64{logit(0.95), logit(0.42), logit(0.11)} + for i := range want { + if scores[i] != want[i] { + t.Errorf("scores[%d] = %v, want %v (logit of Voyage probability)", i, scores[i], want[i]) + } + } + // Voyage reports total_tokens; Score must surface it as promptTokens. + if ptoks != 42 { + t.Errorf("promptTokens = %d, want 42 (total_tokens fallback)", ptoks) + } +} + +// TestScore_VoyageDataReAlignsByIndex verifies index re-alignment works through +// the "data" path too (Voyage sorts descending by score, not input order). +func TestScore_VoyageDataReAlignsByIndex(t *testing.T) { + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + // Server returns sorted DESC: doc2 (best), doc0, doc1 (worst). + return http.StatusOK, voyageResults( + map[string]any{"index": 2, "relevance_score": 0.99}, + map[string]any{"index": 0, "relevance_score": 0.50}, + map[string]any{"index": 1, "relevance_score": 0.05}, + ) + }) + + scores, _, err := Score(context.Background(), srv.URL, "", "rerank-2.5", "q", []string{"apple", "bear", "exact"}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + // Must be re-aligned to INPUT order (as logits of the Voyage + // probabilities): doc0=0.50, doc1=0.05, doc2=0.99. + want := []float64{logit(0.50), logit(0.05), logit(0.99)} + for i := range want { + if scores[i] != want[i] { + t.Errorf("scores[%d] = %v, want %v (index re-alignment via data broken)", i, scores[i], want[i]) + } + } +} + +// TestScore_PrefersResultsOverData: when both "results" and "data" are present +// (hypothetical server that sends both), "results" wins (cohere/llama.cpp path). +func TestScore_PrefersResultsOverData(t *testing.T) { + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + return http.StatusOK, map[string]any{ + "object": "list", + "results": []map[string]any{ + {"index": 0, "relevance_score": 1.0}, + {"index": 1, "relevance_score": 2.0}, + }, + "data": []map[string]any{ + {"index": 0, "relevance_score": 9.0}, + {"index": 1, "relevance_score": 8.0}, + }, + // total_tokens deliberately differs: the assertion below is + // only discriminating if both fields are present and unequal. + "usage": map[string]any{"prompt_tokens": 10, "total_tokens": 999}, + } + }) + + scores, ptoks, err := Score(context.Background(), srv.URL, "", "m", "q", []string{"a", "b"}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + // Must use "results" values (1.0, 2.0), NOT "data" values (9.0, 8.0). + if scores[0] != 1.0 || scores[1] != 2.0 { + t.Errorf("scores = %v, want [1.0 2.0] (results must take precedence over data)", scores) + } + if ptoks != 10 { + t.Errorf("promptTokens = %d, want 10 (prompt_tokens preferred over total_tokens)", ptoks) + } +} + +// TestScore_UsageAbsentChargesNothing pins the documented metering contract: +// "Absent field ⇒ 0 ⇒ the call site charges nothing (uncharged, never an +// estimate)". A regression here would silently invent charges. +func TestScore_UsageAbsentChargesNothing(t *testing.T) { + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + return http.StatusOK, okResults(map[string]any{"index": 0, "relevance_score": 1.0}) + }) + + _, ptoks, err := Score(context.Background(), srv.URL, "", "m", "q", []string{"a"}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if ptoks != 0 { + t.Errorf("promptTokens = %d, want 0 (absent usage must charge nothing)", ptoks) + } +} + +// TestScore_AuthorizationHeader: Voyage is a remote authenticated backend, so +// the bearer header becomes load-bearing — sent when a key is configured, +// absent otherwise (local llama.cpp sidecars reject no auth, they get none). +func TestScore_AuthorizationHeader(t *testing.T) { + var gotAuth string + handler := func(r *http.Request) (int, any) { + gotAuth = r.Header.Get("Authorization") + return http.StatusOK, okResults(map[string]any{"index": 0, "relevance_score": 1.0}) + } + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + status, body := handler(r) + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(status) + _ = json.NewEncoder(w).Encode(body) + })) + t.Cleanup(srv.Close) + + if _, _, err := Score(context.Background(), srv.URL, "sk-test-key", "m", "q", []string{"a"}); err != nil { + t.Fatalf("unexpected error: %v", err) + } + if gotAuth != "Bearer sk-test-key" { + t.Errorf("Authorization = %q, want %q", gotAuth, "Bearer sk-test-key") + } + + if _, _, err := Score(context.Background(), srv.URL, "", "m", "q", []string{"a"}); err != nil { + t.Fatalf("unexpected error: %v", err) + } + if gotAuth != "" { + t.Errorf("Authorization = %q, want empty when no apiKey configured", gotAuth) + } +} + +// TestScore_VoyageTotalTokensFallback: when prompt_tokens is absent but +// total_tokens is present (Voyage), Score returns total_tokens as the usage. +func TestScore_VoyageTotalTokensFallback(t *testing.T) { + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + return http.StatusOK, map[string]any{ + "object": "list", + "results": []map[string]any{ + {"index": 0, "relevance_score": 0.5}, + }, + "usage": map[string]any{"total_tokens": 77}, + } + }) + + _, ptoks, err := Score(context.Background(), srv.URL, "", "m", "q", []string{"a"}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if ptoks != 77 { + t.Errorf("promptTokens = %d, want 77 (total_tokens fallback)", ptoks) + } +} + +// TestScore_VoyageDataRejectsCountMismatch: the "data" path must enforce the +// same count-mismatch validation as "results" (no silent partial scoring). +func TestScore_VoyageDataRejectsCountMismatch(t *testing.T) { + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + // Only 1 result in "data" for 3 docs. + return http.StatusOK, voyageResults( + map[string]any{"index": 0, "relevance_score": 0.9}, + ) + }) + + if _, _, err := Score(context.Background(), srv.URL, "", "rerank-2.5", "q", []string{"a", "b", "c"}); err == nil { + t.Fatal("expected count mismatch error via data path, got nil") + } +} + +// TestScore_VoyageDataRejectsDuplicateIndex: duplicate index validation applies +// through the "data" fallback path too. +func TestScore_VoyageDataRejectsDuplicateIndex(t *testing.T) { + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + return http.StatusOK, voyageResults( + map[string]any{"index": 0, "relevance_score": 0.9}, + map[string]any{"index": 0, "relevance_score": 0.8}, // dup + ) + }) + + if _, _, err := Score(context.Background(), srv.URL, "", "rerank-2.5", "q", []string{"a", "b"}); err == nil { + t.Fatal("expected duplicate index error via data path, got nil") + } +} + +// TestScore_NeitherResultsNorData: a response with neither "results" nor +// "data" must fail with a dedicated schema error naming both fields — not a +// "count mismatch", which misdiagnoses a dialect break as a counting problem +// (the failure mode that masked the original Voyage incident). +func TestScore_NeitherResultsNorData(t *testing.T) { + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + return http.StatusOK, map[string]any{"object": "list", "model": "m"} + }) + + _, _, err := Score(context.Background(), srv.URL, "", "m", "q", []string{"a"}) + if err == nil { + t.Fatal("expected error when neither results nor data present, got nil") + } + if !strings.Contains(err.Error(), "neither") { + t.Errorf("error should name the schema break, got: %v", err) + } + if strings.Contains(err.Error(), "count mismatch") { + t.Errorf("schema break must not masquerade as a count mismatch, got: %v", err) + } +} + +// TestScore_SchemaErrorEchoesBodySnippet: a gateway that answers HTTP 200 +// with an error object must surface what it actually sent — the operator's +// only trace is the fail-open log line carrying this error string. +func TestScore_SchemaErrorEchoesBodySnippet(t *testing.T) { + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + return http.StatusOK, map[string]any{"detail": "quota exceeded for rerank-2.5"} + }) + + _, _, err := Score(context.Background(), srv.URL, "", "m", "q", []string{"a"}) + if err == nil { + t.Fatal("expected schema error, got nil") + } + if !strings.Contains(err.Error(), "quota exceeded") { + t.Errorf("error should echo a body snippet for diagnosis, got: %v", err) + } +} + +// TestScore_VoyageLogitRoundTrip pins the cross-backend score contract: the +// caller's sigmoid (rrf.RerankCrossEncoder) applied to what Score() returns +// for a Voyage "data" response must reconstruct the original probability. +// This is the regression fence against double-sigmoiding calibrated scores, +// which compresses the rerank signal into [0.5,0.73] and lets RRF outvote +// the reranker at blend_weight < 1. +func TestScore_VoyageLogitRoundTrip(t *testing.T) { + probs := []float64{0.05, 0.10, 0.20, 0.98} + rs := make([]map[string]any, len(probs)) + for i, p := range probs { + rs[i] = map[string]any{"index": i, "relevance_score": p} + } + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + return http.StatusOK, voyageResults(rs...) + }) + + scores, _, err := Score(context.Background(), srv.URL, "", "rerank-2.5", "q", []string{"a", "b", "c", "d"}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + for i, p := range probs { + got := 1.0 / (1.0 + math.Exp(-scores[i])) + if math.Abs(got-p) > 1e-9 { + t.Errorf("sigmoid(scores[%d]) = %v, want %v (logit round-trip broken)", i, got, p) + } + } +} + +// TestScore_VoyageEndpointScoresStayFinite: exact 0 and 1 are legitimately +// producible by a quantized probability; they must clamp to large finite +// logits, never ±Inf (which would poison the downstream blend arithmetic). +func TestScore_VoyageEndpointScoresStayFinite(t *testing.T) { + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + return http.StatusOK, voyageResults( + map[string]any{"index": 0, "relevance_score": 0.0}, + map[string]any{"index": 1, "relevance_score": 1.0}, + ) + }) + + scores, _, err := Score(context.Background(), srv.URL, "", "rerank-2.5", "q", []string{"a", "b"}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if math.IsInf(scores[0], 0) || math.IsInf(scores[1], 0) || math.IsNaN(scores[0]) || math.IsNaN(scores[1]) { + t.Errorf("endpoint probabilities must clamp to finite logits, got %v", scores) + } + if scores[0] >= 0 || scores[1] <= 0 { + t.Errorf("clamped endpoint logits lost their sign: got %v, want [negative, positive]", scores) + } +} + +// TestScore_ForeignDataFieldDoesNotBreakResultsPath: "data" is also the +// generic OpenAI-style list container. A backend serving valid "results" +// alongside a non-array "data" field (object, string, whatever) must decode +// exactly as before the Voyage fallback existed — the primary llama.cpp +// path may never regress over a field we ignore. +func TestScore_ForeignDataFieldDoesNotBreakResultsPath(t *testing.T) { + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + return http.StatusOK, map[string]any{ + "results": []map[string]any{ + {"index": 0, "relevance_score": -2.0}, + {"index": 1, "relevance_score": 5.97}, + }, + "data": map[string]any{"note": "not an array"}, + "usage": map[string]any{"prompt_tokens": 10}, + } + }) + + scores, ptoks, err := Score(context.Background(), srv.URL, "", "m", "q", []string{"a", "b"}) + if err != nil { + t.Fatalf("foreign data field broke the results path: %v", err) + } + if scores[0] != -2.0 || scores[1] != 5.97 || ptoks != 10 { + t.Errorf("results path changed: scores=%v ptoks=%d, want [-2 5.97] 10", scores, ptoks) + } +} + +// TestScore_NonArrayDataWithoutResultsErrors: when "data" is the only +// container present but is not an array of rerank entries, the decode error +// must say so (and echo the body) instead of pretending a count mismatch. +func TestScore_NonArrayDataWithoutResultsErrors(t *testing.T) { + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + return http.StatusOK, map[string]any{"data": "unexpected string"} + }) + + _, _, err := Score(context.Background(), srv.URL, "", "m", "q", []string{"a"}) + if err == nil { + t.Fatal("expected decode error for non-array data, got nil") + } + if !strings.Contains(err.Error(), "decode data") { + t.Errorf("error should name the data decode failure, got: %v", err) + } +} + +// TestScore_MissingRelevanceScoreFailsOpen: sibling rerank dialects name the +// score field "score" instead of "relevance_score" (mixedbread, gateways). +// Decoding such a body must error — never silently score every document 0.0, +// which would neutralize the reranker while telemetry reports success. Both +// container paths are covered; the "results" case guards the pre-existing +// lattice, the "data" case the one this PR's fallback newly reaches. +func TestScore_MissingRelevanceScoreFailsOpen(t *testing.T) { + for name, body := range map[string]any{ + "results": map[string]any{ + "results": []map[string]any{ + {"index": 0, "score": 0.9}, + {"index": 1, "score": 0.5}, + }, + }, + "data": map[string]any{ + "data": []map[string]any{ + {"index": 0, "score": 0.9}, + {"index": 1, "score": 0.5}, + }, + }, + } { + t.Run(name, func(t *testing.T) { + b := body + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + return http.StatusOK, b + }) + if _, _, err := Score(context.Background(), srv.URL, "", "m", "q", []string{"a", "b"}); err == nil { + t.Fatalf("%s path: expected missing relevance_score error, got nil", name) + } + }) + } +} + +// TestScore_VoyageDataRejectsOutOfRangeScore: a "data" entry outside [0,1] +// breaks the documented Voyage schema — Score must error (caller fails open) +// rather than feed a bogus value through the probability→logit transform. +func TestScore_VoyageDataRejectsOutOfRangeScore(t *testing.T) { + srv := rerankServer(t, func(_ rerankRequest) (int, any) { + return http.StatusOK, voyageResults( + map[string]any{"index": 0, "relevance_score": 4.2}, + ) + }) + + if _, _, err := Score(context.Background(), srv.URL, "", "rerank-2.5", "q", []string{"a"}); err == nil { + t.Fatal("expected error for data score outside [0,1], got nil") + } +}