大模型
arxiv.org ·
DRIFT：通过同策略归因进行 LLM 指令调优的数据选择
该论文提出 DRIFT（Data Refinement via On-Policy Influence Functions for Supervised Fine-Tuning），一种基于影响函数的数据归因方法，用于指令微调的数据选择。与以往以效率为目标不同，作者研究数据选择能否提升性能上限：先让 LLM 在全量语料上完成 SFT，再从同一语料中筛选样本继续训练，任何增益都意味着超过全量训练的上限。DRIFT 采样模型对验证查询的自身回答并按正确性加权构成验证目标，据此对原语料样本按估计影响排序，再对排名靠前的子集继续 SFT，并修正影响分数对梯度范数的依赖。在 Olmo3-7B-Instruct-SFT 与 OpenR1-Distill-7B 上，八项基准平均准确率分别从 39.17 提升至 40.14、从 57.23 提升至 58.66，且在未用于归因的基准上也有增益。
AI推荐理由 arXiv 论文，摘要明确给出方法（基于影响函数的 on-policy 数据归因）、实验设置（在已完成全量 SFT 的模型上继续训练）和具体数字（Olmo3-7B-Instruct-SFT 八项基准平均从 39.17 升至 40.14，OpenR1-Distill-7B 从 57.23 升至 58.66），信息密度高、可验证。局限在于提升幅度较小，且摘要未说明基准构成与统计显著性，也未提供代码或复现细节。
78
正文
Computer Science > Machine Learning
arXiv:2606.18307 (cs)
[Submitted on 16 Jun 2026 (v1), last revised 26 Sep 2026 (this version, v2)]
Title:DRIFT: Data Selection for LLM Instruction Tuning via On-Policy Attribution
Authors:Zefan Wang, Lincheng Li, Tianyu Yu, Yuan Yao
View a PDF of the paper titled DRIFT: Data Selection for LLM Instruction Tuning via On-Policy Attribution, by Zefan Wang and 3 other authors
View PDF
HTML (experimental)
Abstract:Data selection for instruction tuning is usually studied for efficiency, reaching strong performance with a fraction of the training data. We ask whether data selection can instead raise the performance ceiling, producing a better model than training on all the available data. To test this, we take a large language model (LLM) that has already completed supervised fine-tuning (SFT) on its full corpus and train it further on examples selected from that same corpus. Any gain therefore lifts the model above the ceiling set by full-data training. Existing selection methods yield little or no gain in this setting. We propose DRIFT (Data Refinement via On-Policy Influence Functions for Supervised Fine-Tuning), a data attribution method built on influence functions. DRIFT samples the model's own responses to validation queries and weights them by correctness to form a validation objective. It ranks the examples of the original corpus by their estimated influence on this objective, and the model then continues SFT on the top-ranked subset. This design is motivated by the proximity gap in influence function theory: we hypothesize that these on-policy responses are better validation targets than external references in terms of locality. DRIFT also corrects influence scores for their dependence on gradient norm within each validation task. DRIFT outperforms all evaluated data selection baselines on Olmo3-7B-Instruct-SFT and OpenR1-Distill-7B, raising average accuracy across eight benchmarks from 39.17 to 40.14 and from 57.23 to 58.66, respectively, with gains on benchmarks not used for attribution.
Subjects:
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as:
arXiv:2606.18307 [cs.LG]
(or
arXiv:2606.18307v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.18307
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: Zefan Wang [view email]
[v1]
Tue, 16 Jun 2026 07:21:49 UTC (1,022 KB)
[v2]
Sat, 26 Sep 2026 13:43:45 UTC (1,287 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled DRIFT: Data Selection for LLM Instruction Tuning via On-Policy Attribution, by Zefan Wang and 3 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.LG
new
|
recent
| 2026-06
Change to browse by:
cs
cs.AI
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
IArxiv recommender toggle
IArxiv Recommender
(What is IArxiv?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)