大模型
arxiv.org ·
探索扩散大语言模型在代码生成中的潜力
该 arXiv 论文（v3，2026-09-13 修订）称首次系统实证研究扩散 LLM 用于代码生成，涉及 9 个代表性扩散 LLM 与 4 个常用基准。作者指出自回归 LLM 存在逐 token 生成效率低、只能从左到右生成两点局限，而扩散 LLM 通过多 token 预测与灵活生成顺序应对。主要发现包括：现有扩散 LLM 与同规模自回归 LLM 具竞争力；长度外推能力更强、长代码理解更好；并分析影响效果与效率的因素、给出实践建议与后续方向。代码、数据与结果已开源。
AI推荐理由 arXiv 论文（v3，2026-09-13 修订），声称是首个针对扩散 LLM 代码生成的系统性实证研究：覆盖 9 个代表性扩散 LLM、4 个常用基准，结论包括与同规模自回归 LLM 竞争力相当、长度外推与长代码理解更强，并给出影响效果与效率的因素及实践建议，且开源代码与数据。信息密度高、可验证性强，对代码生成与模型架构方向有参考价值；局限是摘要未给出具体指标数值，结论强度需看正文。
78
正文
Computer Science > Software Engineering
arXiv:2509.11252 (cs)
[Submitted on 14 Sep 2025 (v1), last revised 13 Sep 2026 (this version, v3)]
Title:Exploring the Potential of Diffusion Large Language Models in Code Generation
Authors:Chengze Li, Yitong Zhang, Jia Li, Liyi Cai, Ge Li
View a PDF of the paper titled Exploring the Potential of Diffusion Large Language Models in Code Generation, by Chengze Li and 4 other authors
View PDF
HTML (experimental)
Abstract:LLMs have become the mainstream approaches to code generation. Existing LLMs mainly employ autoregressive generation, i.e. generating code token-by-token from left to right. However, the underlying autoregressive generation has two limitations in code generation. First, autoregressive LLMs only generate a token at each step, showing low efficiency in practice. Second, programming is a non-sequential process involving back-and-forth editing, while autoregressive LLMs only employ the left-to-right generation order. These two intrinsic limitations hinder the further development of LLMs in code generation. Recently, diffusion LLMs have emerged as a promising alternative. Diffusion LLMs address the above limitations with two advances, including multi-token prediction (i.e. generating multiple tokens at each step) and flexible generation order (i.e. flexibly determining which positions to generate tokens). However, there is no systematic study exploring diffusion LLMs in code generation. To bridge the knowledge gap, we present the first empirical study of diffusion LLMs for code generation. Our study involves 9 representative diffusion LLMs and conduct experiments on 4 widely used benchmarks. Based on the results, we summarize the following findings. (1) Existing diffusion LLMs are competitive with autoregressive LLMs with similar sizes. (2) Diffusion LLMs have a stronger length extrapolation ability than autoregressive LLMs and perform better in long code understanding. (3) We explore factors impacting the effectiveness and efficiency of diffusion LLMs, and provide practical guidance. (4) We discuss several promising further directions to improve diffusion LLMs on code generation. We open-source all source code, data, and results to facilitate the following research. The code is publicly available at this https URL.
Comments:
Subjects:
Software Engineering (cs.SE); Artificial Intelligence (cs.AI)
Cite as:
arXiv:2509.11252 [cs.SE]
(or
arXiv:2509.11252v3 [cs.SE] for this version)
https://doi.org/10.48550/arXiv.2509.11252
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: Yitong Zhang [view email]
[v1]
Sun, 14 Sep 2025 12:51:06 UTC (852 KB)
[v2]
Sun, 2 Nov 2025 02:15:42 UTC (852 KB)
[v3]
Sun, 13 Sep 2026 07:24:51 UTC (1,477 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Exploring the Potential of Diffusion Large Language Models in Code Generation, by Chengze Li and 4 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.SE
new
|
recent
| 2025-09
Change to browse by:
cs
cs.AI
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)