大模型
arxiv.org ·
迈向训练私有 LLM：在 Apple Silicon 上通过 Thunderbolt RDMA 微调语言模型
该论文研究 Apple Silicon 能否作为私有 LLM 微调的实用平台。作者刻画了 Mac Studio 节点上 RDMA-over-Thunderbolt 的通信表现，发现实测带宽远低于标称规格；随后通过多链路通信、持久 worker 线程和 CPU 侧梯度重叠进行优化。在四节点 Mac Studio 集群微调 Qwen3-9B 时，优化后弱扩展吞吐较单链路、无重叠基线最高提升 1.6 倍，序列长度 17408 下达 936 tokens/s。论文还将 Apple Silicon 与 NVIDIA H100 平台对比，量化内存容量、吞吐与购置成本之间的权衡，认为 Apple Silicon 可为私有 LLM 微调提供具成本效益的方案。
AI推荐理由 arXiv 论文（cs.DC），摘要给出明确实验设置与量化结果：在四节点 Mac Studio 集群上微调 Qwen3-9B，通过多链路通信、持久 worker 线程和 CPU 侧梯度重叠，弱扩展吞吐较单链路无重叠基线最高提升 1.6 倍，序列长度 17408 时达 936 tokens/s，并与 NVIDIA H100 平台对比内存容量、吞吐与购置成本。新信息在于把 RDMA-over-Thunderbolt 用于私有 LLM 微调的实测特征（实测带宽远低于标称规格）及优化手段；局限是仅摘要级证据、未提供完整基准细节，且属探索性研究而非生产验证。
78
正文
Computer Science > Distributed, Parallel, and Cluster Computing
arXiv:2609.18066 (cs)
[Submitted on 16 Sep 2026 (v1), last revised 17 Sep 2026 (this version, v2)]
Title:Towards Training Private LLMs: Exploring Fine-Tuning Language Models on Apple Silicon with RDMA over Thunderbolt
Authors:En-Ming Huang, Yao-Ting Hsieh, Hsiang-Yu Tsou, Mu-Chi Chen, Shih-Hao Hung, H.T. Kung
View a PDF of the paper titled Towards Training Private LLMs: Exploring Fine-Tuning Language Models on Apple Silicon with RDMA over Thunderbolt, by En-Ming Huang and 5 other authors
View PDF
HTML (experimental)
Abstract:Private large language model (LLM) fine-tuning is increasingly important for organizations that need to adapt models using sensitive data, but it often exceeds the memory capacity of commodity datacenter accelerators. Apple Silicon offers a different design point through large unified memory and lower complete-system cost, while recent Apple software support enables distributed execution over RDMA-over-Thunderbolt (TB). This paper studies whether Apple Silicon can serve as a practical platform for private LLM fine-tuning. We characterize RDMA-over-TB communication on Mac Studio nodes, showing that the measured bandwidth is far below nominal TB specifications. Next, we extend Apple's implementation with multi-trunk communication, persistent worker threads, and CPU-side gradient overlap to better exploit multiple direct TB links for LLM fine-tuning workloads. Finally, on a four-node Mac Studio cluster that fine-tunes a Qwen3-9B, our optimizations improve weak-scaling throughput by up to 1.6X over the single-trunk, non-overlapped baseline and reach 936 tokens/s for sequence length 17408. We further compare Apple Silicon with an NVIDIA H100 platform to quantify the trade-off between memory capacity, throughput, and acquisition cost, showing that Apple Silicon can provide a cost-effective solution for private LLM fine-tuning.
Comments:
Subjects:
Distributed, Parallel, and Cluster Computing (cs.DC)
Cite as:
arXiv:2609.18066 [cs.DC]
(or
arXiv:2609.18066v2 [cs.DC] for this version)
https://doi.org/10.48550/arXiv.2609.18066
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: En-Ming Huang [view email]
[v1]
Wed, 16 Sep 2026 03:11:40 UTC (713 KB)
[v2]
Thu, 17 Sep 2026 17:04:46 UTC (713 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Towards Training Private LLMs: Exploring Fine-Tuning Language Models on Apple Silicon with RDMA over Thunderbolt, by En-Ming Huang and 5 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.DC
new
|
recent
| 2026-09
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)