大模型
arxiv.org ·
DeepStack：面向分布式 LLM 推理的 3D DRAM 堆叠加速器协同设计探索框架
该论文提出 DeepStack，一个用于分布式 3D 堆叠 LLM 推理的精确性能模型与高效设计空间探索（DSE）框架。硬件层建模事务感知内存带宽、bank 激活约束、缓冲限制及热与功耗行为；系统层纳入并行化策略与执行调度。借助双阶段网络抽象和 tile 级计算-通信重叠建模，作者称其评估速度比现有模拟器快最多 100,000 倍且精度相当，并与自研 3D 设计、NS-3 后端（2.12% 误差）以及 8 张 B200 GPU 上的 vLLM 服务（12.92% 误差）交叉验证。结合分层搜索，可探索约 2.5×10^14 个设计点，覆盖堆叠 DRAM 层数、垂直互连、互连网络、计算-内存分配及热与面积约束下的分布式调度；消融实验显示受限 DSE 基线可能漏掉最多 9.5 倍建模吞吐。论文还给出分布式 3D AI 系统的设计启示，并公开源码与工件。
AI推荐理由 arXiv 硬件架构论文（v4），提出面向分布式 3D DRAM 堆叠加速器 LLM 推理的性能模型与设计空间探索框架 DeepStack。具体新信息：硬件层建模事务感知内存带宽、bank 激活约束、缓冲限制与热/功耗行为，系统层纳入并行策略与执行调度；通过双阶段网络抽象和 tile 级计算-通信重叠建模，声称评估速度比现有模拟器快最多 100,000 倍且精度相当；与自研 3D 设计、NS-3 后端（2.12% 误差）及 8 张 B200 上 vLLM 服务（12.92% 误差）交叉验证；结合分层搜索探索约 2.5×10^14 个设计点，消融显示受限 DSE 基线可能漏掉最多 9.5 倍建模吞吐。局限：加速比与误差均为作者自述，未提供第三方复现；属建模与 DSE 工作，非实测芯片或生产部署效果。
78
正文
Computer Science > Hardware Architecture
arXiv:2604.04750 (cs)
[Submitted on 6 Apr 2026 (v1), last revised 11 Sep 2026 (this version, v4)]
Title:DeepStack: Facilitating Co-Design Exploration of 3D DRAM-Stacked Accelerators for Distributed LLM Inference
Authors:Zhiwen Mo, Guoyu Li, Hao Mark Chen, Yu Cheng, Zhengju Tang, Qianzhou Wang, Lei Wang, Shuang Liang, Lingxiao Ma, Xianqi Zhou, Yuxiao Guo, Wayne Luk, Jilong Xue, Hongxiang Fan
View a PDF of the paper titled DeepStack: Facilitating Co-Design Exploration of 3D DRAM-Stacked Accelerators for Distributed LLM Inference, by Zhiwen Mo and 13 other authors
View PDF
HTML (experimental)
Abstract:Advances in hybrid bonding and packaging have driven growing interest in 3D DRAM-stacked AI accelerators. As large language models (LLMs) scale to hundreds of billions or trillions of parameters, distributed inference across multiple 3D chips has become essential for AI serving. This trend makes cross-stack co-design critical because system-level parallelization and scheduling choices are tightly coupled with hardware characteristics such as memory organization, interconnects, and thermal constraints. We present DeepStack, an accurate performance model and efficient design space exploration (DSE) framework for distributed 3D-stacked LLM inference. At the hardware level, DeepStack captures transaction-aware memory bandwidth, bank activation constraints, buffering limitations, and thermal and power behavior. At the system level, it incorporates comprehensive parallelization strategies and execution scheduling. Through a dual-stage network abstraction and tile-level compute-communication overlap modeling, DeepStack achieves up to 100,000x faster evaluation than state-of-the-art simulators at comparable accuracy. We cross-validate DeepStack against our in-house 3D designs, an NS-3 backend with 2.12% error, and vLLM serving on eight B200 GPUs with 12.92% error. Combined with hierarchical search, DeepStack efficiently explores about 2.5 x 10^14 design points spanning the number of stacked DRAM layers, DRAM vertical connectivity, interconnects, compute-memory allocation, and distributed scheduling under thermal and area constraints. A search-space ablation shows that restricted DSE baselines can miss up to 9.5x modeled throughput. Beyond modeling and DSE, DeepStack derives design implications for distributed 3D AI systems and guides performance optimization across the stack. Source code and artifacts are available at this https URL.
Comments:
Subjects:
Hardware Architecture (cs.AR); Distributed, Parallel, and Cluster Computing (cs.DC)
Cite as:
arXiv:2604.04750 [cs.AR]
(or
arXiv:2604.04750v4 [cs.AR] for this version)
https://doi.org/10.48550/arXiv.2604.04750
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: Zhiwen Mo [view email]
[v1]
Mon, 6 Apr 2026 15:16:35 UTC (16,902 KB)
[v2]
Thu, 9 Apr 2026 14:13:19 UTC (16,899 KB)
[v3]
Fri, 21 Aug 2026 13:51:10 UTC (2,496 KB)
[v4]
Fri, 11 Sep 2026 14:11:13 UTC (3,264 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled DeepStack: Facilitating Co-Design Exploration of 3D DRAM-Stacked Accelerators for Distributed LLM Inference, by Zhiwen Mo and 13 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.AR
new
|
recent
| 2026-04
Change to browse by:
cs
cs.DC
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)