论文研究
arxiv.org ·
FADE：通过降低大型视觉语言模型中的语言先验主导性来缓解幻觉
该 arXiv 论文研究大型视觉语言模型（LVLM）的幻觉问题，指出语言先验压过视觉输入是主因。作者逐层分析 transformer 信息流，发现 attention 模块持续聚合视觉证据，而关键层的 FFN 模块是语言先验来源，可覆盖视觉证据使中间层正确预测漂移为错误输出。据此提出免训练方法 FADE（FFN Attenuation for DEcoding），通过衰减 FFN 输出降低语言先验主导性。在 POPE、CHAIR、MME 基准上对 LLaVA-1.5、mPLUG-Owl2、InstructBLIP 的评测显示其可缓解幻觉并保持推理效率。
AI推荐理由 arXiv 论文（v5，2026-09-15 修订），提出免训练方法 FADE：通过衰减 FFN 输出降低语言先验主导性以缓解 LVLM 幻觉。作者通过逐层信息流分析指出 attention 聚合视觉证据、关键层 FFN 是语言先验来源，并在 POPE、CHAIR、MME 上对 LLaVA-1.5、mPLUG-Owl2、InstructBLIP 评测，声称在保持推理效率的同时缓解幻觉。机制解释与免训练特性对多模态模型落地有参考价值；但摘要未给出具体指标数值，效果需以原文实验为准。
78
正文
Computer Science > Artificial Intelligence
arXiv:2606.29431 (cs)
[Submitted on 28 Jun 2026 (v1), last revised 15 Sep 2026 (this version, v5)]
Title:FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Models
Authors:Yichen Guo, Kai Tang, Jinhao You, Fenglai Lin, Yiding Sun, Dongxu Zhang, Wenya Wang, Lin William Cong, Shanghang Zhang
View a PDF of the paper titled FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Models, by Yichen Guo and 8 other authors
View PDF
HTML (experimental)
Abstract:Despite the impressive capabilities of Large Vision-Language Models (LVLMs), they remain susceptible to hallucination, generating content inconsistent with the input image. Recent studies attribute this to the dominance of language priors over visual inputs and employ contrastive decoding methods to mitigate this dominance, but the mechanistic origin remains unexplored. We investigate the information flow through each transformer layer and find that attention modules consistently aggregate visual evidence, while FFN modules at critical layers act as the source of language priors. These priors can override visual evidence, causing correct predictions in intermediate layers to drift toward incorrect outputs. Based on this insight, we propose FADE (FFN Attenuation for DEcoding), a training-free method that attenuates FFN outputs to reduce language-prior dominance. Evaluations on POPE, CHAIR, and MME benchmarks across LLaVA-1.5, mPLUG-Owl2, and InstructBLIP show that FADE effectively mitigates hallucinations while preserving inference efficiency.
Comments:
Subjects:
Artificial Intelligence (cs.AI)
Cite as:
arXiv:2606.29431 [cs.AI]
(or
arXiv:2606.29431v5 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.29431
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: Yichen Guo [view email]
[v1]
Sun, 28 Jun 2026 14:48:08 UTC (3,935 KB)
[v2]
Wed, 1 Jul 2026 04:28:30 UTC (4,495 KB)
[v3]
Mon, 6 Jul 2026 07:27:45 UTC (4,496 KB)
[v4]
Tue, 7 Jul 2026 03:19:40 UTC (4,496 KB)
[v5]
Tue, 15 Sep 2026 11:52:44 UTC (4,498 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Models, by Yichen Guo and 8 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.AI
new
|
recent
| 2026-06
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)