Agent workloads repeatedly encode tool and skill schemas that recur across requests in different combinations and orders, so standard prefix caching cannot reuse their KV states. ReCache caches each resource's representation independently: resource-wise attention removes cross-resource interactions and assigns resource-local positions, producing composition-invariant KV blocks; visibility is restricted to contribution-selected layer-KV-head-group routes, and structural plus semantic pruning keeps only invocation-critical fields. On a benchmark assembled from seven public tool/skill-use datasets (including resource-disjoint tests), resource-wise attention matches dense invocation performance (82.3% vs 82.4% Inv-F1) while delivering a 3.655x time-to-first-token speedup; the full framework cuts allocated KV-tensor memory by 92.43% and accelerates attention by 1.423x. Code is available at github.com/EIT-NLP/ReCache.
ReCache makes KV cache reusable for tool-heavy LLM agents
Agent workloads repeatedly encode tool and skill schemas that recur across requests in different combinations and orders, so standard prefix caching cannot reuse their KV states. ReCache caches each resource's representation independently: resource-wise attention removes cross-resource interactions and assigns resource-local positions, producing composition-invariant KV blocks; visibility is restricted to contribution-selected layer-KV-head-group routes, and structural plus semantic pruning keeps only invocation-critical fields. On a benchmark assembled from seven public tool/skill-use datasets (including resource-disjoint tests), resource-wise attention matches dense invocation performance (82.3% vs 82.4% Inv-F1) while delivering a 3.655x time-to-first-token speedup; the full framework cuts allocated KV-tensor memory by 92.43% and accelerates attention by 1.423x. Code is available at github.com/EIT-NLP/ReCache.
Imported from the existing research archive. Migration preserves the recorded judgment and is not a fresh verification.
What this means for you
Prefix caching fails exactly where agent traffic is heaviest: tool schemas recombined on every request. For teams running MCP or tool-heavy gateways, this is a concrete serving-cost lever on both TTFT and KV memory, complementing the correctness side of agent x KV-cache interactions (ev-20260818-03). The caveats: it requires attention-level changes (resource-wise attention), so it is not a drop-in cache configuration; integration into production serving stacks carries real cost, and the evaluation is academic-benchmark scale.
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Problem | tool/skill schemas recur across requests in varying combinations/orders; prefix caching cannot reuse their KV states |
|---|---|
| Mechanism | resource-wise attention: remove cross-resource interactions + resource-local positions -> composition-invariant KV blocks · contribution-selected layer-KV-head-group visibility routes · structural + semantic pruning of invocation-critical fields |
| Results | Inv-F1 82.3% vs 82.4% dense (parity) · 3.655x TTFT speedup · -92.43% allocated KV-tensor memory · 1.423x attention speedup |
| Benchmark | seven public tool/skill-use datasets, incl. resource-disjoint tests |
| Code | github.com/EIT-NLP/ReCache |
| Relation To Kb | cost dimension of agent x KV-cache line; correctness dimension opened by ev-20260818-03 (arXiv 2608.15939) |