智能体
x.com ·
@omarsar0
KVMem：将百万级Token Agent工作区虚拟化到消费级GPU
KVMem是一种提升推理效率的新方法，将溢出内容以分页KV状态存储，利用注意力索引选择相关历史块，在DeepSWE测试中任务成功率从43.8%提升至48.4%，并可在24GB RTX 5090上运行1M token工作区。
AI推荐理由 介绍KVMem方法，通过分页KV状态和注意力索引提升长上下文agent推理效率，在DeepSWE基准上从43.8%提升至48.4%，并支持消费级GPU上1M token工作区。信息具体，有数据支撑，但来源为个人X帖，非官方发布。
78
浏览 7,141
点赞 50
评论 22
转发 6
正文
Nice paper to improve inference efficiency.
It's been a while we haven't seen good work on efficiency.
Here is why it matters:
A long-running agent's workspace outgrows its context window long before the task finishes.
The first approach commonly used, compaction, loses the fine-grained execution evidence. And text retrieval re-prefills content the model already processed.
KVMem keeps the overflow as paged KV state instead, spread across GPU memory, host memory and NVMe.
Lightweight attention-space indexes, native to the model, pick the relevant historical blocks and materialize a query-dependent view that fits inside the native context window.
On the DeepSWE long-context test with Qwen3.8-27B, task success goes from 43.8% under compaction to 48.4%.
The local deployment result stands out. It runs Qwen3.6/3.8-27B NVFP4 with MTP on a laptop with a 24GB RTX 5090, virtualizing an agent workspace up to 1M tokens, four times the model's native 256K window, at around 50 tokens per second.
Paper: [academy.dair.ai/papers/kvmem-v…](https://academy.dair.ai/papers/kvmem-virtualizing-million-token-agent-workspaces-on-a-consumer-gpu-2609.04852)
