大模型
arxiv.org ·
从推理引擎到推理控制平面：连接 vLLM、llm-d 与高效分布式 LLM 服务的演进
该 arXiv 预印本将 LLM 推理从引擎内优化问题重新表述为分布式控制问题，涉及可复用状态、阶段放置、异构加速器、网络、自动扩缩容、可靠性与 SLO。作者把 vLLM 与 llm-d 视为互补层：前者通过 PagedAttention、连续批处理、内核、量化与并行优化执行，后者作为推理控制平面决定执行的位置、时机与策略。论文提出 Inference Execution Planner，选择可行执行方案（聚合/分离拓扑、KV 来源与传输、硬件变体、路由与准入策略、扩缩容决策），并提供来源本地基准图谱、瓶颈迁移分类、部署指南与基于 SLO-goodput 的评估框架。作者明确本文为综合而非新基准，所有性能与部署结果均归属原始来源。
AI推荐理由 arXiv 预印本，作者自述为综述性综合（synthesis）而非新基准，所有性能与部署结果均归属原始来源，未声称已验证或官方首发。新信息在于提出 Inference Execution Planner 概念，把 vLLM（PagedAttention、连续批处理、量化、并行）与 llm-d 视为互补层，并给出瓶颈迁移分类、SLO-goodput 评估框架及面向 agentic/多模态/异构推理的研究问题。局限是缺乏新实验数据，结论依赖二手证据，实际价值需后续验证。
72
正文
Computer Science > Artificial Intelligence
arXiv:2609.23130 (cs)
[Submitted on 19 Sep 2026]
Title:From Inference Engine to Inference Control Plane: Connecting vLLM, llm-d, and the Evolution of Efficient Distributed LLM Serving
Authors:Twinkll Sisodia
View a PDF of the paper titled From Inference Engine to Inference Control Plane: Connecting vLLM, llm-d, and the Evolution of Efficient Distributed LLM Serving, by Twinkll Sisodia
View PDF
HTML (experimental)
Abstract:Large language model (LLM) inference is evolving from an engine-local optimization problem into a distributed control problem involving reusable state, phase placement, heterogeneous accelerators, networking, autoscaling, reliability, and service-level objectives. This paper connects that transition across peer-reviewed systems research, open-source implementations, and documented production studies. It treats vLLM and llm-d as complementary layers: model-serving engines optimize execution through mechanisms such as PagedAttention, continuous batching, kernels, quantization, and parallelism, while an inference control plane can optimize where, when, and under what policy execution occurs across a fleet. The contribution is synthesis rather than a new benchmark; all reported performance and deployment results remain attributed to their original sources. The combined evidence suggests that the scarce resource in modern inference is shifting from raw FLOPs alone toward managed state, placement, network movement, reliability, and decision quality. We propose an Inference Execution Planner that selects feasible execution plans rather than only endpoints, including aggregated versus disaggregated topology, KV source and transfer action, hardware variant, routing/admission policy, and slower scaling decisions. We also provide a source-local benchmark atlas, a bottleneck-migration taxonomy, practical deployment guidance, an evaluation framework based on SLO-goodput, and research questions for agentic, multimodal, heterogeneous, and resilient inference.
Comments:
Subjects:
Artificial Intelligence (cs.AI); Performance (cs.PF)
Cite as:
arXiv:2609.23130 [cs.AI]
(or
arXiv:2609.23130v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.23130
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Twinkll Sisodia [view email]
[v1]
Sat, 19 Sep 2026 16:58:20 UTC (80 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled From Inference Engine to Inference Control Plane: Connecting vLLM, llm-d, and the Evolution of Efficient Distributed LLM Serving, by Twinkll Sisodia
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.AI
new
|
recent
| 2026-09
Change to browse by:
cs
cs.PF
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)