大模型
arxiv.org ·
微调 LLM 用于翻译：通用遗忘缓解无法保持机器翻译特定的指令遵循能力
该 arXiv 论文研究翻译微调中的灾难性遗忘问题，指出通用基准上的遗忘缓解评估未必适用于机器翻译特定指令遵循（如正式度、语法性别、长度控制）。作者先在 Llama 3.2 1B Instruct 上做筛选，再在双向阿英或西英数据上微调 Llama 3.1 8B Instruct，比较基于辅助数据、模型输出和基座参数的方法。结果显示 Elastic Weight Consolidation 在通用能力保持上最好，8B 西语模型通用基准平均仅降 1.7 分，而标准微调降 11.0 分，但其正式度与语法性别控制得分仍接近标准微调；只有混入控制任务样本的数据混合能保住这些控制，且增益无法迁移到同任务的未见提示。
AI推荐理由 arXiv 正式提交的 cs.CL 论文，作者含 Hermann Ney 等机器翻译领域研究者，摘要给出可核验的实验设置与量化结果：在 Llama 3.2 1B Instruct 筛选后，于 Llama 3.1 8B Instruct 上用双向阿英或西英数据微调，Elastic Weight Consolidation 在通用基准上平均仅降 1.7 分（标准微调降 11.0 分），但其对正式度和语法性别控制的保持接近标准微调；只有混入控制任务样本的数据混合能保住这些控制能力，且增益无法迁移到同任务的未见提示。新信息在于指出通用遗忘缓解评估结论不能直接外推到机器翻译特定指令遵循，对做翻译微调与持续学习的开发者有直接参考价值。局限是仅覆盖两个语言对和两个模型规模，摘要未给出完整数据与统计显著性，结论需以全文为准。
72
正文
Computer Science > Computation and Language
arXiv:2609.28395 (cs)
[Submitted on 23 Sep 2026]
Title:Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following
Authors:Niklas Scholz, David Thulke, Abdallah Nasir, Will Allred, Evgeny Matusov, Hermann Ney
View a PDF of the paper titled Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following, by Niklas Scholz and 5 other authors
View PDF
HTML (experimental)
Abstract:Fine-tuning large language models on parallel data improves translation quality but can cause catastrophic forgetting. Mitigation methods are generally evaluated by retention on general benchmarks. We ask whether these findings transfer to machine translation (MT) fine-tuning and to MT-specific instruction following (MT-IF): instructions that modify a translation, such as formality, grammatical gender, and length control. We compare methods anchored to auxiliary data, to model outputs, and to the base model parameters, first in a screening study with Llama 3.2 1B Instruct, then on Llama 3.1 8B Instruct fine-tuned on bidirectional Arabic-English or Spanish-English data. Elastic Weight Consolidation preserves general capabilities best in both stages; on the 8B Spanish model the average score on general benchmarks drops 1.7 points versus 11.0 for standard fine-tuning, yet its scores for formality and grammatical gender control remain close to standard fine-tuning. Only data mixing with control-task examples preserves these controls, but its gains do not transfer to unseen prompts for the same task.
Comments:
Subjects:
Computation and Language (cs.CL); Machine Learning (cs.LG)
ACM classes:
I.2.7
Cite as:
arXiv:2609.28395 [cs.CL]
(or
arXiv:2609.28395v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.28395
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: David Thulke [view email]
[v1]
Wed, 23 Sep 2026 17:02:59 UTC (105 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following, by Niklas Scholz and 5 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CL
new
|
recent
| 2026-09
Change to browse by:
cs
cs.LG
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)