论文研究
dev.to ·
@Papers Mache
GVA 将 KV cache 削减约 45%，精度几乎不变
一篇 DEV 社区文章介绍 Grouped Value Attention（GVA）：只存储分组后的 value，并在解码时即时重建 key，从而避免物化完整 key 矩阵。作者称在 350M 参数模型上，持久 cache 标量相对匹配的 GQA 减少约 45–47%，16 维位置变体在五个任务上平均准确率 44.35，对比 GQA 的 44.36 和 MLA 的 43.88。该思路针对长上下文自回归推理中随序列长度线性增长的 KV 内存瓶颈，但摘要未给出论文、代码或独立复现证据。
AI推荐理由 提出 Grouped Value Attention（GVA），在 350M 参数模型上将持久 KV cache 标量减少约 45–47%，五个任务平均准确率 44.35 对 GQA 的 44.36，属可验证的推理内存优化方向；但来源为个人博客，摘要中引用的 DeepSeek-V4.1 信息被截断，缺少论文与代码链接，效果尚未经独立复现。
72
正文
Grouped Value Attention slashes transformer KV memory by roughly 45 % without hurting benchmark scores. By storing only grouped values and reconstructing keys on the fly, it eliminates the need to materialize a full key matrix during decode, cutting the persistent cache scalars dramatically.
Before these works, long‑context decoding relied on full key–value caches that grew linearly with sequence length. Even techniques like grouped‑query attention still kept a key for every token, so memory pressure remained the dominant bottleneck in autoregressive inference.
On a 350 M‑parameter model, GVA cuts persistent cache scalars by about 46 % while keeping average task accuracy within 0.01 points of GQA. The authors report “this representation reduces persistent cache scalars by approximately 45–47 % relative to matched GQA” and note that “the 16‑dimensional positional variant reaches 44.35 average accuracy across five tasks, compared with 44.36 for GQA and 43.88 for MLA” [1].
DeepSeek‑V4.1‑Flash pushes KV compression further, shrinking the global HBM footprint to 890 bytes per token – roughly one quarter of its predecessor. Moreover, “through a dedicated deployment optimization known as SWA Bounded Replay … reduces its persistent KV cache footprint … to roughly 1/8 of that of DeepSeek‑V4‑Flash,” and despite these cuts the model “delivers substantially better performance than the baseline” [2].
Both papers leave open how these memory savings translate into end‑to‑end latency reductions on commodity GPUs. GVA’s custom decoding kernels are still under evaluation, so real‑world throughput gains remain unproven; DeepSeek’s design includes FP4 caching and cross‑layer reuse, which could benefit from hardware supporting low‑precision arithmetic.
If the reported cuts hold in production, the KV cache allocation for decoder‑only models could potentially be reduced substantially while maintaining quality. These reductions suggest that re‑evaluating long‑context benchmarks with smaller cache budgets may enable longer token contexts on more modest hardware.
References
Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression