大模型
arxiv.org ·
MAPS：面向大语言模型服务的内存感知预测调度框架
该 arXiv 论文提出 MAPS，一种面向 prefill-decode 分离式 LLM 服务的调度框架。它利用设备端辅助的输出长度推测并与云端 prefill 重叠执行，以极低延迟开销获得预测；再通过不确定性感知校准推导满足目标覆盖率的输出长度上界，据此进行全局-局部分层调度，缓解解码器间队列堆积与解码器内队头阻塞。作者称在两个真实负载和两个 LLM 上的实验显示，相比三个现有最优系统，平均端到端延迟降低 42.6，尾延迟最高降低 84.8。
AI推荐理由 arXiv 新提交论文（2026-09-14），提出 MAPS 内存感知预测调度框架，针对 prefill-decode 分离式 LLM 服务中 decode 实例负载不均问题：用设备端辅助的输出长度推测与云端 prefill 重叠，并以不确定性感知校准给出带目标覆盖率的输出长度上界，再做全局-局部分层调度。摘要给出在两个真实负载、两个 LLM 上相比三个 SOTA 系统平均端到端延迟降低 42.6、尾延迟最高降低 84.8 的结果，属可验证的量化证据；但仅为论文自述、尚无独立复现或代码链接，且未说明具体模型与负载细节，故未给更高分。
72
正文
Computer Science > Artificial Intelligence
arXiv:2609.15359 (cs)
[Submitted on 14 Sep 2026]
Title:MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Serving
Authors:Tiancheng Zhang, Yulin Chen, Yunfeng Zhao, Shaoyuan Huang, Cheng Zhang, Xiaofei Wang
View a PDF of the paper titled MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Serving, by Tiancheng Zhang and 5 other authors
View PDF
HTML (experimental)
Abstract:The surge of large language model (LLM) applications on personal devices imposes massive, bursty workloads on cloud serving infrastructure. While prefill-decode disaggregation improves throughput and scalability, memory-bound decode instances often suffer from persistent load imbalance, as output lengths are unknown when requests arrive at the cloud. To address this, we propose MAPS, a Memory-Aware Predictive Scheduling framework tailored for disaggregated LLM serving. MAPS performs device-assisted speculative output length prediction overlapped with cloud-side prefilling, incurring negligible latency overhead. To handle generation uncertainty, MAPS applies uncertainty-aware calibration to derive output-length upper bounds with target coverage, enabling safe scheduling decisions. Building on these bounds, MAPS employs a hierarchical global-local scheduling strategy to mitigate inter-decoder queue buildup and intra-decoder head-of-line blocking. Extensive experiments on two real-world workloads and two LLMs show that MAPS significantly outperforms three state-of-the-art systems, reducing average end-to-end latency by 42.6 and tail latency by up to 84.8.
Comments:
Subjects:
Artificial Intelligence (cs.AI)
Cite as:
arXiv:2609.15359 [cs.AI]
(or
arXiv:2609.15359v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.15359
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Tiancheng Zhang [view email]
[v1]
Mon, 14 Sep 2026 10:43:42 UTC (1,515 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled MAPS: Memory-Aware Predictive Scheduling Framework for Large Language Model Serving, by Tiancheng Zhang and 5 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.AI
new
|
recent
| 2026-09
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)