论文研究
arxiv.org ·
当量化保持精度却不保持证据：面向医疗大语言模型的可解释性感知训练后量化
该 arXiv 预印本研究医疗多选题问答中的训练后量化（PTQ）问题：常规 PTQ 以重建、困惑度或答案准确率为优化目标，可能保持最终答案正确却削弱支撑答案的推理依据。作者提出一种解释感知的 PTQ 目标，从全精度教师模型的 rationale 构建离线 faithfulness cache，在优化中保留支撑答案的证据 token 及证据条件下的答案行为，并在 OSTQuant 的 W4A4KV4 量化下，对四个 7B–8B 医疗与指令微调模型在 MedExQA、MedExpQA、ChallengeClinicalQA 上评估。结果显示，同校准的 OSTQuant 基线虽保持任务准确率，却可能明显削弱支撑答案的 rationale；所提方法能更好保留全精度模型的答案行为与 rationale 对答案的支持。作者据此主张，面向解释关键场景的 PTQ 应评估支撑答案证据的保留情况，而非仅看答案准确率。
AI推荐理由 arXiv 预印本，作者与摘要信息完整，问题设定明确：现有 PTQ 通常只优化重建、困惑度或答案准确率，但在医疗等需要解释的场景中，量化可能保持答案准确率却削弱支撑答案的推理依据。论文提出 explanation-aware 的 PTQ 目标，基于全精度教师 rationale 构建离线 faithfulness cache，在 OSTQuant 的 W4A4KV4 设置下于 MedExQA、MedExpQA、ChallengeClinicalQA 上评估四个 7B–8B 医疗/指令模型，并称相比同校准基线能更好保留全精度模型的答案行为与 rationale-to-answer 支持。局限是仅预印本、未提供具体数值，且作者明确目标是保留全精度行为而非提升 gold-label 准确率。
72
正文
Computer Science > Computation and Language
arXiv:2609.24799 (cs)
[Submitted on 21 Sep 2026]
Title:When Quantization Preserves Accuracy but Not Evidence: Explanation-Aware Post-Training Quantization for Medical LLMs
Authors:Yeji Kim, Mi-Young Kim, Randy Goebel
View a PDF of the paper titled When Quantization Preserves Accuracy but Not Evidence: Explanation-Aware Post-Training Quantization for Medical LLMs, by Yeji Kim and 2 other authors
View PDF
HTML (experimental)
Abstract:Post-training quantization (PTQ) enables efficient deployment of large language models, and PTQ methods are usually optimized and evaluated with generic reconstruction, perplexity, or answer accuracy. But in explanation-critical domains, preserving only the final answer may be insufficient, since users may also inspect generated rationales to judge whether a prediction is trustworthy. We study this issue in medical multiple-choice question answering, where rationales should provide evidence that supports the selected answer.
We propose an explanation-aware objective for transformation-based PTQ. Our method builds an offline faithfulness cache from full-precision teacher rationales and uses it during optimization to preserve answer-supporting evidence tokens and evidence-conditioned answer behavior. We instantiate it on OSTQuant under W4A4KV4 quantization and evaluate four 7B--8B medical and instruction-tuned LLMs on MedExQA, MedExpQA, and ChallengeClinicalQA. While a same-calibration OSTQuant baseline preserves task accuracy, it can substantially weaken answer-supporting rationales. Our objective is to preserve the full-precision model's answer-supporting behavior rather than improve gold-label accuracy, and our method better preserves the full-precision model's answer behavior and rationale-to-answer support. These results suggest that PTQ for explanation-critical settings should evaluate preservation of answer-supporting evidence, not only answer accuracy. Code and evaluation scripts are available at this https URL.
Subjects:
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as:
arXiv:2609.24799 [cs.CL]
(or
arXiv:2609.24799v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.24799
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Yeji Kim [view email]
[v1]
Mon, 21 Sep 2026 15:59:06 UTC (128 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled When Quantization Preserves Accuracy but Not Evidence: Explanation-Aware Post-Training Quantization for Medical LLMs, by Yeji Kim and 2 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CL
new
|
recent
| 2026-09
Change to browse by:
cs
cs.AI
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)