RedKnot：基于头部感知 KV 复用和 SegPagedAttention 的高效长上下文 LLM 服务

Computer Science > Artificial Intelligence arXiv:2606.06256 (cs) [Submitted on 4 Jun 2026 (v1), last revised 21 Sep 2026 (this version, v4)] Title:RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention Authors:Yang Liu, Zhaokai Luo, Huayi Jin, Zhiyong Wang, Ruozhou He, Boyu Wang, Guanjie Chen, Yifei Liu, Tao Xie, Junhao Hu View a PDF of the paper titled RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention, by Yang Liu and 9 other authors View PDF HTML (experimental) Abstract:As the input length of large language model (LLM) serving continues to grow, the KV cache has become a dominant bottleneck in AI infrastructure. It limits GPU memory capacity, serving concurrency, cache reuse, and distributed scalability. Multiple important problems, including position-independent KV cache, prefix KV cache compression, hot/cold KV cache separation, and distributed KV cache management, all depend on how the KV cache is represented and managed. However, existing serving systems largely rely on a monolithic KV cache abstraction, where the KV cache is treated as a homogeneous sequence of token-level memory blocks and managed with similar policies across attention heads and serving scenarios. We observe that KV cache utility is highly structured across KV heads: different heads exhibit different functional roles, attention distances, and runtime importance. Therefore, a full KV cache is not always necessary for every head, token range, or serving scenario. We present RedKnot, a head-aware KV cache management system for LLM serving. RedKnot breaks the conventional monolithic KV cache abstraction by decomposing the KV cache along KV heads, whose importance and effective attention ranges vary significantly across serving scenarios. This head-level decomposition turns the KV cache from a monolithic tensor abstraction into a structured memory object, enabling RedKnot to uniformly support position-independent KV reuse, prefix KV compression, hot/cold KV separation, and distributed KV placement while preserving output fidelity and improving resource efficiency, without requiring model retraining or fine-tuning. RedKnot establishes a new foundation for AI infrastructure by transforming the KV cache from a monolithic, passive runtime artifact into a dynamic, model-aware runtime substrate for scalable LLM serving. Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2606.06256 [cs.AI] (or arXiv:2606.06256v4 [cs.AI] for this version) https://doi.org/10.48550/arXiv.2606.06256 Focus to learn more arXiv-issued DOI via DataCite Submission history From: Junhao Hu [view email] [v1] Thu, 4 Jun 2026 14:57:07 UTC (5,852 KB) [v2] Fri, 26 Jun 2026 06:47:29 UTC (3,193 KB) [v3] Wed, 29 Jul 2026 11:03:07 UTC (3,193 KB) [v4] Mon, 21 Sep 2026 15:06:27 UTC (3,193 KB) Full-text links: Access Paper: View a PDF of the paper titled RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention, by Yang Liu and 9 other authors View PDF HTML (experimental) TeX Source view license Current browse context: cs.AI new | recent | 2026-06 Change to browse by: cs References & Citations NASA ADS Google Scholar Semantic Scholar export BibTeX citation Loading... BibTeX formatted citation × loading... Data provided by: Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Code, Data and Media Associated with this Article alphaXiv Toggle alphaXiv (What is alphaXiv?) Links to Code Toggle CatalyzeX Code Finder for Papers (What is CatalyzeX?) DagsHub Toggle DagsHub (What is DagsHub?) GotitPub Toggle Gotit.pub (What is GotitPub?) Huggingface Toggle Hugging Face (What is Huggingface?) ScienceCast Toggle ScienceCast (What is ScienceCast?) Demos Demos Replicate Toggle Replicate (What is Replicate?) Spaces Toggle Hugging Face Spaces (What is Spaces?) Spaces Toggle TXYZ.AI (What is TXYZ.AI?) Related Papers Recommenders and Search Tools Link to Influence Flower Influence Flower (What are Influence Flowers?) Core recommender toggle CORE Recommender (What is CORE?) Author Venue Institution Topic About arXivLabs arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs. Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)