算力商业
developer.nvidia.com ·
@Yang Liu
在 NVIDIA DGX Spark 上提升 MoE 推理性能的三种方法
NVIDIA 技术博客介绍在 DGX Spark 上优化 MoE 推理的三种方式，并以 Inclusion AI 的 Ling-3.0-flash 在 batch-size-1 条件下的评估为例，说明需先定位推理性能瓶颈再提速。
AI推荐理由 NVIDIA 官方技术博客，聚焦 DGX Spark 上 MoE 推理优化，摘要提到以 Inclusion AI 的 Ling-3.0-flash 在 batch-size-1 场景评估性能瓶颈，属于可复现的工程实践信息；但正文片段有限，具体三种方法的效果数据未在摘要中给出。
78
正文
Generating tokens faster starts with identifying what limits inference performance. In the batch-size-1 evaluation of Ling-3.0-flash from Inclusion AI on NVIDIA...
Generating tokens faster starts with identifying what limits inference performance. In the batch-size-1 evaluation of Ling-3.0-flash from Inclusion AI on NVIDIA DGX Spark, profiling showed that decoding was memory-bound: reading model weights drove execution time. Two operations stood out—the mixture-of-experts (MoE) computation and the output-layer projection that produces scores across the model’…
Source