大模型
arxiv.org ·
LLM 能否模拟学生的错误推理？一项关于干扰项生成的案例研究
该 arXiv 论文研究 LLM 在为选择题生成干扰项时如何推理学生误解。作者提出基于学习科学文献与实证观察的干扰项生成推理策略分类法，并应用于数学和科学选择题的 LLM 推理轨迹。结果显示：数学数据集上模型遵循误解驱动流程（先恢复正确解、表述学生错误、模拟错误、筛选候选），具有潜在诊断价值；科学数据集上则倾向基于与正确答案的语义相似度，稳健性较低。最常见失败模式是模型无法生成正确解，或在选择阶段丢弃合理的干扰项候选。在提示中提供正确解后，与人工编写干扰项的对齐相对提升 6.4%，说明将干扰项生成锚定到正确解很关键。
AI推荐理由 arXiv 论文（v2，2026-09-15 修订），作者与摘要明确，属可验证的研究工作。核心新信息：提出基于学习科学文献的干扰项生成推理策略分类法，并用于分析 LLM 在数学与科学选择题上的推理轨迹；发现数学数据集上模型走“先解出正确答案—表述学生错误—模拟—筛选”的误解驱动流程，科学数据集上则更多依赖与正确答案的语义相似度；主要失败模式为无法生成正确解或在选择阶段丢弃合理干扰项；在提示中提供正确解可使与人工编写干扰项的对齐相对提升 6.4%。局限：仅为干扰项生成这一具体任务的案例研究，未提供跨模型、跨数据集的泛化证据，也未说明绝对性能水平。对 AI 教育应用与 LLM 推理可解释性有参考价值，但非通用能力突破。
72
正文
Computer Science > Computation and Language
arXiv:2603.15547 (cs)
[Submitted on 16 Mar 2026 (v1), last revised 15 Sep 2026 (this version, v2)]
Title:Can LLMs Model Incorrect Student Reasoning? A Case Study on Distractor Generation
Authors:Yanick Zengaffinen, Andreas Opedal, Donya Rooein, Kv Aditya Srivatsa, Shashank Sonkar, Mrinmaya Sachan
View a PDF of the paper titled Can LLMs Model Incorrect Student Reasoning? A Case Study on Distractor Generation, by Yanick Zengaffinen and 5 other authors
View PDF
HTML (experimental)
Abstract:Modeling student misconceptions in a realistic manner is critical for AI in education. In this work, we examine how large language models (LLMs) reason about misconceptions when generating distractor answers for multiple-choice questions (MCQs), a task that requires producing answers that are incorrect, yet plausible. We introduce a taxonomy over reasoning strategies for distractor generation that is grounded in learning-science literature and empirical observation, which we apply to LLM-generated reasoning traces across math and science MCQs. On the math dataset, we find that models follow a misconception-based process with potentially high diagnostic value: they recover the correct solution, articulate student errors, simulate them, and select plausible candidates. On the science dataset, on the other hand, they tend to follow a less robust approach based on semantic similarity to the correct answer. We find the most frequent failure modes to be that the model is unable to generate a correct solution or that it discards plausible distractor candidates when performing selection. Providing the correct solution in the prompt yields a relative improvement of 6.4% in alignment with human-authored distractors, highlighting the critical role of anchoring distractor generation to the correct solution. Together, our findings offer an interpretable view of how LLMs model incorrect student reasoning.
Comments:
Subjects:
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
Cite as:
arXiv:2603.15547 [cs.CL]
(or
arXiv:2603.15547v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2603.15547
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: Yanick Zengaffinen [view email]
[v1]
Mon, 16 Mar 2026 17:09:41 UTC (227 KB)
[v2]
Tue, 15 Sep 2026 17:27:55 UTC (573 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Can LLMs Model Incorrect Student Reasoning? A Case Study on Distractor Generation, by Yanick Zengaffinen and 5 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CL
new
|
recent
| 2026-03
Change to browse by:
cs
cs.AI
cs.HC
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)