大模型
arxiv.org ·
事实锚定注意力：通过注意力层知识集成消除大语言模型中的幻觉
arXiv 论文提出 Fact Grounded Attention（FGA），一种在 transformer 的 pre-softmax 注意力分数层面注入可验证知识的架构改动，而非生成后修补或前置检索文本。作者称在覆盖智能手机、笔记本和电动车的 1,107 条技术查询上，Llama 3.2 的准确率从 6.3% 提升至 99.7%，知识更新可在不到一秒内完成且无需重训练。论文为 v3 修订版，尚未经同行评审，'完全消除幻觉'为作者主张。
AI推荐理由 arXiv 预印本，提出 Fact Grounded Attention（FGA），在 pre-softmax 注意力分数层面注入可验证知识，声称在 1,107 条技术查询上将 Llama 3.2 准确率从 6.3% 提升至 99.7%，且知识更新无需重训练、耗时不到一秒。摘要给出具体数字与机制，信息密度较高；但仅两名作者、无同行评审，'完全消除幻觉'的表述属作者自述，缺乏第三方复现，需保留不确定性。
62
正文
Computer Science > Artificial Intelligence
arXiv:2509.25252 (cs)
[Submitted on 27 Sep 2025 (v1), last revised 18 Sep 2026 (this version, v3)]
Title:Fact Grounded Attention: Eliminating Hallucination in Large Language Models Through Attention Level Knowledge Integration
Authors:Aayush Gupta, Manish Choudhary
View a PDF of the paper titled Fact Grounded Attention: Eliminating Hallucination in Large Language Models Through Attention Level Knowledge Integration, by Aayush Gupta and 1 other authors
View PDF
HTML (experimental)
Abstract:"The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge." Large Language Models have conquered natural language but remain prisoners of their own probabilistic nature--confidently hallucinating facts they never truly knew. We present Fact Grounded Attention (FGA), a novel architectural modification that transforms unreliable language models into deterministic truth tellers by injecting verifiable knowledge directly into the attention mechanism. Unlike existing approaches that patch hallucinations after generation or prepend retrieved text, FGA intervenes at the mathematical heart of the transformer--the pre-softmax attention scores--creating a model that cannot hallucinate when facts exist in its knowledge base. Our experiments across 1,107 technical queries spanning smartphones, laptops, and electric vehicles demonstrate a transformation from 6.3% accuracy in vanilla Llama 3.2 to 99.7% accuracy with FGA. More critically, knowledge updates occur in under one second without retraining, compared to hours for parameter editing approaches. FGA doesn't just reduce hallucination--it eliminates it entirely for verifiable facts, marking a fundamental shift from probabilistic approximation to deterministic precision in neural language generation.
Comments:
Subjects:
Artificial Intelligence (cs.AI)
MSC classes:
68T50, 68T05, 68T30
ACM classes:
I.2.7; I.2.6; H.3.3
Cite as:
arXiv:2509.25252 [cs.AI]
(or
arXiv:2509.25252v3 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2509.25252
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: Aayush Gupta [view email]
[v1]
Sat, 27 Sep 2025 09:55:21 UTC (17 KB)
[v2]
Thu, 2 Oct 2025 06:19:43 UTC (17 KB)
[v3]
Fri, 18 Sep 2026 11:32:49 UTC (17 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Fact Grounded Attention: Eliminating Hallucination in Large Language Models Through Attention Level Knowledge Integration, by Aayush Gupta and 1 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.AI
new
|
recent
| 2025-09
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)