KVMem virtualizes million-token agent workspaces on a consumer GPU

KVMem (arXiv 2609.04852, Mon 7 Sep digest) is a KV-context virtualization system for long-running agents: instead of compacting overflowed workspace history into summaries or re-retrieving it as text (losing fine-grained execution evidence and re-prefilling already-processed content), it preserves overflowed history as paged KV state across GPU memory, host memory and NVMe. Lightweight model-native attention-space indexes select relevant historical blocks, and a query-dependent execution view bounded by the model's native context window is materialized on demand. On long-context agent benchmarks spanning histories up to one million tokens — LongMemEval, MemoryAgentBench, AgentLongBench — KVMem generally achieves higher task utility and greater inference efficiency than compaction-based approaches, the de-facto standard for context overflow (paper reports a DeepSWE long-context test with Qwen3.8-27B).

This attacks the biggest practical gap in long-running agents: compaction is lossy and re-retrieval is expensive, while the KV the model already computed is thrown away. Paging KV to host/NVMe with attention-space indexing is the memory-hierarchy move the agent-memory space has been missing, and it runs on a single consumer GPU. Teams building persistent workspaces (cloud agents, long coding sessions) get a design template that keeps full-fidelity history without paying unbounded context costs.