AI编码
arxiv.org ·
SkillGym：将人类技能内化到 LLM 中以解决现实世界问题
该论文提出 SkillGym 框架，把人类编写的 agent skill 转化为可执行、可验证的训练环境：通过 skill-to-task 流水线生成具体任务，用代码检查器验证结果，并以对比执行评估技能依赖程度。作者构建并发布覆盖 12 个类别的 2756 个环境，从多个模型与 harness 收集 8364 条成功轨迹（平均 49 次工具调用、超 6 万 token 日志），用于监督微调和基于结果奖励的强化学习。在 Claude Code 下，监督微调使 Qwen3.5-35B-A3B 在 GDPval-AA v2 提升 199 Elo、Terminal-Bench 2.1 提升 19.10 个百分点，SkillsBench v1.1 在有/无技能条件下分别提升 28.13 和 12.38 分；35B 的 SkillGym-Agent 在技能辅助 SkillsBench 上达到 51.47%，作者称超过 Claude Sonnet 4.6、GPT-5.4 Mini 和 DeepSeek V4 Pro 的已报告成绩。
AI推荐理由 arXiv 新论文，提出 SkillGym 框架，将人类编写的 agent skill 转化为可执行、可验证的训练环境，而非仅作为推理期外部指令。摘要给出具体规模（12 类 2756 个环境、8364 条成功轨迹、平均 49 次工具调用）和可核验的评测数字（Qwen3.5-35B-A3B 在 GDPval-AA v2 提升 199 Elo、Terminal-Bench 2.1 提升 19.10 个百分点等），并声称 35B SkillGym-Agent 在 SkillsBench 上超过若干更大模型。信息密度高、方法可复现，对 agent 训练与技能内化方向有实际参考价值；局限在于所有结果均为论文自报，尚无第三方复现，且对比模型的具体版本与评测配置需查原文确认。
82
正文
Computer Science > Computation and Language
arXiv:2609.27717 (cs)
[Submitted on 23 Sep 2026]
Title:SkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving
Authors:Zhilong Ge, Yuting Shao, Yutao Yang, Yuxuan Cai, Jie Zhou, Kai Chen, Bo Zhang, Qin Chen, Liang He
View a PDF of the paper titled SkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving, by Zhilong Ge and 8 other authors
View PDF
HTML (experimental)
Abstract:Human-written agent skills encode rich workflows for real-world problem solving, but are typically used as external inference-time instructions rather than internalized as reusable model capabilities. We introduce \texttt{SkillGym}, a framework that transforms these skills into executable, verifiable training environments for large language model agents. Its skill-to-task pipeline instantiates concrete tasks, verifies outcomes with code-based checkers, and assesses empirical skill dependence through contrastive executions. We construct and release 2,756 environments across 12 categories and collect 8,364 successful trajectories from multiple models and harnesses, averaging 49 tool calls and over 60k logged text tokens. These resources support supervised fine-tuning on verified workflows and reinforcement learning with outcome-based rewards. Under Claude Code, supervised fine-tuning improves Qwen3.5-35B-A3B by 199 Elo on GDPval-AA v2, 19.10 percentage points on Terminal-Bench 2.1, and 28.13 and 12.38 points on SkillsBench v1.1 with and without skills, respectively. Our 35B \texttt{SkillGym-Agent} reaches 51.47\% on skill-assisted SkillsBench, exceeding reported scores for Claude Sonnet 4.6, GPT-5.4 Mini, and DeepSeek V4 Pro. Without skills, it also surpasses skill-assisted bases under Codex and Claude Code, suggesting reusable procedural competence.
Subjects:
Computation and Language (cs.CL)
Cite as:
arXiv:2609.27717 [cs.CL]
(or
arXiv:2609.27717v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.27717
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Jie Zhou [view email]
[v1]
Wed, 23 Sep 2026 11:35:34 UTC (2,452 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled SkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving, by Zhilong Ge and 8 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CL
new
|
recent
| 2026-09
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)