智能体
arxiv.org ·
谁执笔?让规范而非智能体来签署
该论文指出 LLM 智能体常在同一循环内完成生成、决策、执行与自评,外部规范仅作为上下文,缺乏独立的规范权威边界,由此产生理解—执行缺口和状态—权威缺口。作者在 SkillsBench 上抽取 509 条任务方向,测试七个模型,满足率仅 79.6%–86.4%,而自我宣称完成率比官方评估通过率高 28.7–37.9 个百分点。为此提出 SpecHarness,将可见规范编译为带来源链接的义务,并以版本化义务状态管理执行与最终确认:可验证需求在运行时被中介或校验,模糊或主观需求仅作建议。
AI推荐理由 arXiv 新论文,提出将规范(specification)而非智能体自身作为完成状态的权威签署方,并给出 SpecHarness 机制。摘要提供了具体量化证据:在 SkillsBench 上抽取 509 条任务方向,七个模型满足率仅 79.6%–86.4%,而自我宣称完成率比官方评估通过率高 28.7–37.9 个百分点,直接支撑其提出的理解—执行缺口与状态—权威缺口。对 Agent 可靠性、评测与执行治理有实际参考价值。局限在于实验细节、SpecHarness 实现与跨任务泛化性仅凭摘要无法验证,且论文为预印本,未经同行评审。
78
正文
Computer Science > Artificial Intelligence
arXiv:2609.29921 (cs)
[Submitted on 24 Sep 2026]
Title:Who Holds the Pen? Let Specifications, Not Agents, Sign Off
Authors:Haiqing Li, Xin Ma, Yinhao Wu, Wenliang Zhong, Feng Jiang, Thao M. Dang, Xiao Hu, Hehuan Ma, Yuzhi Guo, Junzhou Huang
View a PDF of the paper titled Who Holds the Pen? Let Specifications, Not Agents, Sign Off, by Haiqing Li and 9 other authors
View PDF
HTML (experimental)
Abstract:Large language model agents increasingly combine generation, decision-making, execution, and self-evaluation within a single agentic loop. Although they operate under external specifications such as task instructions, guidelines, output schemas, and reusable skills, these specifications typically remain context for the same model that acts and declares completion, leaving no independent specification authority boundary. We identify two resulting gaps. The understanding--execution gap arises when a requirement is understood but not satisfied in execution; the state--authority gap arises when an agent's interpretation or completion claim does not establish the required state. On SkillsBench, using only agent-visible prompts, workspace information, and injected skill specifications, we extract 509 source-grounded task directions. Across seven models, only 79.6%--86.4% are satisfied, while completion-claim rates exceed official evaluator pass rates by 28.7--37.9 percentage points. We therefore separate agent proposals from authoritative state. Agents may plan, act, and request completion, but only admissible evidence from qualified providers may establish specification-governed state. SpecHarness operationalizes this principle by compiling visible specifications into source-linked obligations and governing execution and finalization through versioned obligation state. Verifiable requirements are mediated or validated at runtime, while ambiguous or subjective requirements remain advisory. Experiments on guideline-following and artifact-generation tasks show that specifications can serve not merely as behavioral guidance, but as authority over compliant execution and completion.
Subjects:
Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA)
Cite as:
arXiv:2609.29921 [cs.AI]
(or
arXiv:2609.29921v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.29921
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Haiqing Li [view email]
[v1]
Thu, 24 Sep 2026 14:55:14 UTC (2,035 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Who Holds the Pen? Let Specifications, Not Agents, Sign Off, by Haiqing Li and 9 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.AI
new
|
recent
| 2026-09
Change to browse by:
cs
cs.MA
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)