智能体
arxiv.org ·
RoofLang：赋能 AI 驱动的 LLM 推理系统架构设计
该 arXiv 预印本提出领域特定语言 RoofLang，用于支持 AI 驱动的 LLM 推理系统架构设计。作者认为现有 AI 优化多基于 profiling，搜索空间受限于已有软件栈，难以发现更优架构，因此需要通用工作负载表示、可验证变异空间和实现无关评估器。评估显示 DeepSeek V4 系列模型峰值解码吞吐可比其他代表性模型高 3.5-39.5 倍，差距主要来自紧凑 KV-cache 设计带来的更大批量和更低内存流量；持续运行的优化器 agent 还在 NVIDIA B300 上为 DeepSeek V4 Pro 发现新架构，吞吐与交互性提升 6.23-50.1%。
AI推荐理由 arXiv 预印本，作者与摘要信息明确，提出面向 LLM 推理系统架构设计的 DSL RoofLang，并给出可验证的评估结果（DeepSeek V4 系列峰值解码吞吐提升 3.5-39.5 倍、优化器 agent 在 NVIDIA B300 上提升 6.23-50.1%）。新意在于用通用工作负载表示、可验证变异空间和实现无关评估器替代 profiling 驱动的搜索，对推理系统与 Agent 自动优化方向有参考价值；局限是仅为预印本，结果未经同行评审或独立复现，且摘要未披露完整实验设置。
78
正文
Computer Science > Distributed, Parallel, and Cluster Computing
arXiv:2609.12551 (cs)
[Submitted on 11 Sep 2026]
Title:RoofLang: Enabling AI-Driven Architecting of LLM Inference Systems
Authors:Ziyue Yang, Yuting Jiang, Lei Qu, Peng Cheng
View a PDF of the paper titled RoofLang: Enabling AI-Driven Architecting of LLM Inference Systems, by Ziyue Yang and 3 other authors
View PDF
HTML (experimental)
Abstract:AI is beginning to make substantive contributions to LLM inference optimization. Existing AI optimizations are predominantly profiling-based. Profiling-bound feedback confines the search to the capabilities and performance of an existing software stack, preventing a fundamentally better architecture of LLM inference systems from being identified. To enable the AI-driven LLM inference system architecting loop, we argue that a general workload representation, a verifiable mutation space, and an implementation-independent evaluator are required. We present the RoofLang domain-specific language (DSL) that provides these features. In our evaluation, RoofLang reveals that DeepSeek V4-series models could achieve 3.5-39.5$\times$ higher peak decode throughput than other representative models. This gap is disproportionate to their total parameter counts and arises largely from compact KV-cache designs that support larger batches and reduce memory traffic. A persistent optimizer agent further discovered several new architectures that improved both throughput and interactivity of DeepSeek V4 Pro on NVIDIA B300 by 6.23-50.1%.
Subjects:
Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI)
Cite as:
arXiv:2609.12551 [cs.DC]
(or
arXiv:2609.12551v1 [cs.DC] for this version)
https://doi.org/10.48550/arXiv.2609.12551
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Ziyue Yang [view email]
[v1]
Fri, 11 Sep 2026 07:57:28 UTC (249 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled RoofLang: Enabling AI-Driven Architecting of LLM Inference Systems, by Ziyue Yang and 3 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.DC
new
|
recent
| 2026-09
Change to browse by:
cs
cs.AI
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)