论文研究
arxiv.org ·
迈向高效推理：为扩散语言模型学习因果捷径
该论文针对扩散语言模型（DLM）在双向注意力下探索空间远大于自回归模型、随机掩码时难以聚焦推理引导 token 的问题，定义“因果捷径”为覆盖全序列并显式指向正确推理轨迹的 token 链，并分析其对推理准确率与收敛速度的影响。作者提出 Causal Shortcut Learning（CSL）框架：先用逐步 token 提取流程从数据中抽取因果捷径，再在训练中对这些 token 施加并行优先掩码，使模型沿因果捷径高效收敛到正确答案。在多个推理基准和两个基座模型上的实验显示，CSL 稳定优于现有 SFT 变体基线，较仅 SFT 平均提升 1.92%，在 MATH-500 上最高提升 4.20%，代码已公开。
AI推荐理由 arXiv 新论文，提出面向扩散语言模型（DLM）的 Causal Shortcut Learning（CSL）框架：通过逐步 token 提取从数据中抽取覆盖全序列的因果捷径 token 链，并在训练中对这些 token 施加并行优先掩码，以提升推理收敛效率与准确率。摘要给出具体量化结果：在多个推理基准和两个基座模型上平均较仅 SFT 提升 1.92%，MATH-500 上最高提升 4.20%，并提供代码链接。信息密度较高、方法描述明确，属可验证的研究进展；局限是仅基于摘要，未提供独立复现或第三方验证，提升幅度相对有限，且 DLM 推理效率仍处早期探索阶段。
72
正文
Computer Science > Computation and Language
arXiv:2609.28272 (cs)
[Submitted on 23 Sep 2026]
Title:Towards Efficient Reasoning: Learning Causal Shortcuts for Diffusion Language Models
Authors:Dian Jin, Kairong Han, Baohong Li, Xinpeng Dong, Zijing Hu, Nuanqiao Shan, Fei Wu, Kun Kuang
View a PDF of the paper titled Towards Efficient Reasoning: Learning Causal Shortcuts for Diffusion Language Models, by Dian Jin and 7 other authors
View PDF
HTML (experimental)
Abstract:Diffusion Language Models (DLMs) have attracted significant attention for their strong reasoning ability. However, under a bidirectional attention mechanism, DLMs operate over an exponentially large exploration space compared to autoregressive models (ARMs), making it challenging to focus on reasoning-guiding tokens under random masking. We define causal shortcuts as token chains that cover the full sequence and provide explicit guidance towards correct reasoning trajectories. We analyze the effects of causal shortcuts on the reasoning accuracy and convergence speed of DLMs, and find that they largely improve answer convergence efficiency and generation accuracy. Motivated by this, we propose a Causal Shortcut Learning (CSL) Framework for DLMs. Specifically, we introduce a step-by-step token extraction procedure to extract causal shortcuts from data, and apply parallel prioritized masking on these tokens during training to enable efficient and accurate convergence to correct answers via causal shortcuts. Extensive experiments across multiple reasoning benchmarks and two base models demonstrate that CSL consistently outperforms existing SFT-variant baselines, achieving an average improvement of $1.92\%$ over SFT-only models, and up to $4.20\%$ on MATH-500. The code is available at the \href{this https URL}{this https URL
Subjects:
Computation and Language (cs.CL)
Cite as:
arXiv:2609.28272 [cs.CL]
(or
arXiv:2609.28272v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.28272
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Dian Jin [view email]
[v1]
Wed, 23 Sep 2026 15:27:05 UTC (6,182 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Towards Efficient Reasoning: Learning Causal Shortcuts for Diffusion Language Models, by Dian Jin and 7 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CL
new
|
recent
| 2026-09
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)