feat(server): native tool-prefix caching for multi-turn agents - #492

Merged

davide221 merged 25 commits into

Luce-Org:mainfrom

davidmroth:feat/tool-split-agent-cache

Jul 27, 2026

Commits on May 11, 2026

Merge branch 'feature/dual-gpu-dflash-safe-p2p-port' into feature/dual-gpu-dflash-safe-p2p-tool-use
davidmroth
committed

Commits on Jul 3, 2026

fix(dflash): multi-turn tool-split cache restore and usage timings
davidmroth
committed
fix(dflash): tool-split VRAM budget and ship tool_split package
davidmroth
committed
fix(dflash): reject stale prefix-cache lookups after in-place slot refresh
davidmroth
committed
Merge branch 'feature/dual-gpu-dflash-safe-p2p-tool-use' into main
davidmroth
committed
feat(server): tool-split KV cache for multi-turn agent tool use
davidmroth
committed
fix(tool-split): address Copilot review on slot LRU and snap failure
davidmroth
committed
Merge main into feat/tool-split-agent-cache
davidmroth
committed
docs: reframe tool-split goal around PFlash-friendly tool isolation
davidmroth
committed

Commits on Jul 4, 2026

Commits on Jul 7, 2026

fix(tool-split): address PR review — cache eviction, abort wiring, and hardening
davidmroth
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(server): native tool-prefix caching for multi-turn agents - #492

feat(server): native tool-prefix caching for multi-turn agents#492
davide221 merged 25 commits into
Luce-Org:mainfrom
davidmroth:feat/tool-split-agent-cache

Commits on May 10, 2026

Align server_tools.py with server.py dual-GPU daemon flags

Accept --daemon in server_tools.py for Compose parity with server.py

Improve OpenAI tool parity: tool_choice, max_completion_tokens, ignore extra fields

Fix tool use: default enable_thinking off, force off with tools, fix parse_reasoning

Commits on May 11, 2026

Merge branch 'feature/dual-gpu-dflash-safe-p2p-port' into feature/dual-gpu-dflash-safe-p2p-tool-use

Commits on Jul 3, 2026

fix(dflash): multi-turn tool-split cache restore and usage timings

fix(dflash): tool-split VRAM budget and ship tool_split package

fix(dflash): reject stale prefix-cache lookups after in-place slot refresh

Merge branch 'feature/dual-gpu-dflash-safe-p2p-tool-use' into main

feat(server): tool-split KV cache for multi-turn agent tool use

fix(tool-split): address Copilot review on slot LRU and snap failure

Merge main into feat/tool-split-agent-cache

docs: reframe tool-split goal around PFlash-friendly tool isolation

Commits on Jul 4, 2026

fix(tool-split): stop encoding system-only prompts for Qwen3.6 tools

fix(test_dflash): store prefix snapshots in system RAM via snapshot backend

fix(tool-split): legacy daemon opt-in + dict tool_call arguments coercion

test(tool-split): Qwen-shaped tokenizer stub for split boundary tests

fix(tool-split): drop 1-slot prefix-cache VRAM clamp

feat(server): expose spec-decode telemetry in API usage timings

Commits on Jul 7, 2026

fix(tool-split): address PR review — cache eviction, abort wiring, and hardening

Commits on Jul 27, 2026

Merge remote-tracking branch 'origin/main' into codex/pr492-ready

fix(server): use native cache for tool-heavy agent turns

test(server): add reproducible tool-prefix benchmark

fix(server): harden tool-prefix benchmark accounting

fix(server): preserve prefix cache defaults in containers