大模型
arxiv.org ·
大型语言模型的能力自我评估
论文研究 LLM 的能力自我评估（CSA），即模型判断自身能否解决问题或应否求助。由于同一模型既做评估又做解题，自我评估训练会改变其被评估的能力，作者系统比较监督微调（SFT）与强化学习（RL），在多个模型家族、规模和领域上评估自我评估与更新后能力的一致性及解题能力保留情况。结果显示 RL 在评估准确性与能力保留之间取得最佳权衡，CSA 还能迁移到所评估的分布外场景，并可用于推理时的本地-云端路由和训练中的定向数据选择。
AI推荐理由 arXiv 正式论文（v2，2026-09-26 修订），作者与摘要明确，属可验证的一手研究。核心新信息：系统比较 SFT 与 RL 在 LLM 能力自我评估（CSA）上的评估-解题权衡，发现 RL 在评估准确性与保留解题能力之间取得最佳折中，并验证 CSA 可迁移到分布外场景，可用于推理时的本地-云端路由与训练数据筛选。局限：仅摘要可见，未提供具体模型规模、数据集与量化指标，实际效果需看全文。
78
正文
Computer Science > Artificial Intelligence
arXiv:2606.00251 (cs)
[Submitted on 29 May 2026 (v1), last revised 26 Sep 2026 (this version, v2)]
Title:Capability Self-Assessment in Large Language Models
Authors:Haoyan Yang, Reza Shirkavand, Yukai Jin, Jiawei Zhou, Shangqian Gao, Heng Huang
View a PDF of the paper titled Capability Self-Assessment in Large Language Models, by Haoyan Yang and 5 other authors
View PDF
HTML (experimental)
Abstract:The ability to recognize one's own limitations and decide whether to solve a problem or seek help is fundamental to reliable intelligent systems. Yet modern large language models tend to overestimate their competence and attempt queries they cannot solve. We study how models can learn Capability Self-Assessment (CSA) while preserving their problem-solving ability. Because the same model both assesses and solves queries, self-assessment training can change the capability it must recognize. We systematically compare supervised fine-tuning (SFT) and reinforcement learning (RL), evaluating each trained model's self-assessment against its updated capability and measuring how well it retains its solving ability. Across model families, scales, and domains, RL provides the strongest assessment-retention trade-off. We also find that CSA transfers to the out-of-distribution settings we evaluate, and demonstrate its practical value for local-cloud routing at inference time and targeted data selection during training.
Comments:
Subjects:
Artificial Intelligence (cs.AI)
Cite as:
arXiv:2606.00251 [cs.AI]
(or
arXiv:2606.00251v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.00251
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: Haoyan Yang [view email]
[v1]
Fri, 29 May 2026 18:32:14 UTC (2,226 KB)
[v2]
Sat, 26 Sep 2026 18:21:53 UTC (3,373 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Capability Self-Assessment in Large Language Models, by Haoyan Yang and 5 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.AI
new
|
recent
| 2026-06
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)