大模型
arxiv.org ·
LLM 在被训练预测自身准确率时学会不同形式的元认知
论文训练 10 个开放权重 LLM 在回答事实性多选题前预测自身准确率，发现训练后的置信度反映两种不同信号：在接近训练数据的问题上追踪模型真实准确率，在其他领域则追踪输出一致性（答案分布的集中度）。输出一致性追踪在训练早期出现并可跨数据集泛化，准确率追踪出现较晚且局限于训练分布。作者据此指出，校准训练可能无法让模型普遍识别其自信犯下的错误。
AI推荐理由 arXiv 论文，作者为 Nicolas Yax、Stefano Palminteri、Pierre-Yves Oudeyer，摘要明确给出实验设置（训练 10 个开放权重 LLM 在回答事实性多选题前预测自身准确率）和两项可验证发现：训练后的置信度同时反映真实准确率与输出一致性，其中输出一致性追踪在训练早期出现且可跨数据集泛化，准确率追踪出现较晚且局限于训练分布。该结论对 LLM 校准训练的实际效果提出具体限制，即校准训练未必能让模型普遍识别其自信犯下的错误，对模型可靠性与 Agent 自我评估方向有参考价值。局限在于仅基于摘要，未提供具体模型清单、数据集和量化指标，且论文为预印本，未经同行评审。
78
正文
Computer Science > Computation and Language
arXiv:2609.33886 (cs)
[Submitted on 27 Sep 2026]
Title:LLMs learn different forms of metacognition when trained to predict their own accuracy
Authors:Nicolas Yax, Stefano Palminteri, Pierre-Yves Oudeyer
View a PDF of the paper titled LLMs learn different forms of metacognition when trained to predict their own accuracy, by Nicolas Yax and 2 other authors
View PDF
HTML (experimental)
Abstract:Large language models are trained to always produce an answer, regardless of whether they possess the relevant knowledge, which leads them to fabricate facts. Prior work has shown that LLMs' confidence estimates correspond poorly to their actual performance, and that fine-tuning can substantially improve them. However, what models actually learn during such training remains poorly understood. We investigate how LLMs acquire metacognitive monitoring, the ability to know what one knows, by training 10 open-weight LLMs to predict their own accuracy on factual multiple-choice questions before answering them. We find that trained confidence reflects two distinct signals. While on questions close to the training data, it tracks the model's true accuracy, in other domains, it instead tracks output consistency: the concentration of the model's answer distribution. Output consistency tracking emerges early in training and generalizes across datasets, whereas accuracy tracking develops later and remains local to the training distribution. These results suggest that calibration training may not teach models to generally detect errors they commit confidently, and they raise broader questions about the nature of metacognition in artificial systems.
Subjects:
Computation and Language (cs.CL)
Cite as:
arXiv:2609.33886 [cs.CL]
(or
arXiv:2609.33886v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.33886
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Nicolas Yax [view email]
[v1]
Sun, 27 Sep 2026 20:02:33 UTC (6,578 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled LLMs learn different forms of metacognition when trained to predict their own accuracy, by Nicolas Yax and 2 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CL
new
|
recent
| 2026-09
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)