blog: Tuning Recall — Hindsight's Four Retrieval Arms, Explained

[benfrank241]

A pillar post on tuning recall: what each of the four retrieval arms does, the knob and default that controls each, and how to match the pipeline to the shape of a bank.

What's in it

• The recall pipeline end to end: query analysis → four arms → RRF → cross-encoder rerank
• Each arm with its real thresholds and defaults (semantic 0.3, graph seed 0.3, temporal 0.1, semantic-link 0.7, BM25 min score 0)
• The four per-bank toggles, including the new enable_text_search from feat(recall): add a per-bank enable_text_search toggle for pure vector recall #3819
• Recall budget mapping (fixed 100/300/1000 vs adaptive), strategy boosts, recency decay
• Reranker failover chain, and the local reranker's bucket-batching / FP16 wins
• Retain-side pairing (retain_extraction_mode: chunks, enable_observations: false) for plain-retrieval banks
• A tune-by-bank-shape table and 8 FAQs

Verification

Every default, threshold, and env var name is taken from hindsight-docs/docs/developer/configuration.md and the hindsight-api-slim source, not restated from memory. The enable_text_search behavior is checked against engine/search/retrieval.py:190 and the knowledge-page path in engine/memory_engine.py.

The post deliberately contains no CLI commands: set_config in hindsight-cli/src/commands/bank.rs accepts only the LLM, retain, mission, and disposition params, so none of the four recall toggles are settable via a CLI flag. Configuration is shown through the config API and environment variables only.

Files

• hindsight-docs/blog/2026-08-27-tuning-recall-retrieval-arms.md
• hindsight-docs/static/img/blog/tuning-recall-retrieval-arms.png (1600x900)

Noted separately, not fixed here

The "plain-retrieval (RAG) banks" curl example in configuration.md still lists only the three original recall toggles and omits enable_text_search: false, though that section's premise is making a bank behave like a conventional vector store. #3819 updated the table above it but not the example. Left for its own docs PR.

🤖 Generated with Claude Code

[strix-security]

Strix Security Review

No security issues found.

_{Updated for 462ee56.}

---

Reviewed by Strix
_{Re-run review · Configure security review settings}

[Sanderhoff-alt]

The env var names, defaults and the failover/decay/budget descriptions all match config.py and configuration.md. One factual correction in the FAQ: the similarity thresholds are not per-bank overridable.

[Sanderhoff-alt]

The thresholds are not hierarchical. semantic_min_similarity, graph_seed_min_similarity, temporal_semantic_min_similarity, semantic_link_min_similarity and bm25_min_score are not in _CONFIGURABLE_FIELDS (hindsight-api-slim/hindsight_api/config.py:2911-2975), so a config-API or bank-template write naming one of them is rejected in hindsight-api-slim/hindsight_api/config_resolver.py:452-467 ("Only configurable fields can be overridden per-bank"). Only the four toggles and the recall_budget_* fields are per-bank here.

Suggested: "All four toggles and the recall budget mapping are hierarchical: environment variables set the deployment default and any bank overrides them through the config API or a bank template. The similarity thresholds are server-level only."