大模型
arxiv.org ·
ViSTA：一种将视觉对齐扩展到多模态 LLM 临床时间序列理解的简单桥接
该 arXiv 预印本提出 ViSTA，一个紧凑适配器，将不规则数值测量融入预训练视觉语言模型的图表表示，仅学习对视觉 token 的修正，保持所有预训练参数不变。在 MIMIC-IV 上，ViSTA 在 2-9B 参数模型的急性肾损伤与死亡率预测四项指标中取得对比适配方案的最高平均分；仅 0.516M 可训练参数的 2B 模型在 AKI 预测上 AUROC 达 0.7376，接近 GPT-5.6 Sol 文本输入高推理强度的 0.7380。时序问答训练在 4B 参数下准确率 69.27%，可训练参数比使用图表或数值文本的 LoRA 少 90% 以上，准确率差距 2.82-4.88 个百分点。
AI推荐理由 arXiv 预印本，摘要给出了具体方法（ViSTA 适配器，仅训练 0.516M 参数，冻结预训练参数）和可核验的量化结果（MIMIC-IV 上 AKI 预测 AUROC 0.7376，对比 GPT-5.6 Sol 文本输入 0.7380；时序问答 4B 参数下 69.27% 准确率）。新信息在于用轻量适配器把不规则临床时序数值接入视觉语言模型的图表表示，并声称在多个指标上优于对比适配方案。局限：仅预印本、未同行评审，结果限于 MIMIC-IV 单一数据集，摘要未提供完整基线细节，实际临床泛化能力未知。
72
正文
Computer Science > Computation and Language
arXiv:2609.31448 (cs)
[Submitted on 25 Sep 2026]
Title:ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs
Authors:Junyi Gao, Yu Shi, Pingzhao Hu, Ewen M Harrison
View a PDF of the paper titled ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs, by Junyi Gao and 3 other authors
View PDF
HTML (experimental)
Abstract:Clinical prediction models estimate risk from patient measurements, while large language models support medical text understanding and question answering. Yet their language capabilities do not ensure accurate prediction from structured, high-dimensional clinical time series. Improving this ability would connect risk estimation with flexible questions about a patient's evolving condition. We introduce ViSTA, a compact adapter that incorporates irregular numerical measurements into a pretrained vision-language model's chart representations. It learns corrections to visual tokens while leaving all pretrained parameters unchanged. On MIMIC-IV, ViSTA has the highest mean scores among the compared adaptations on all four metrics for acute kidney injury and mortality prediction across models with 2-9 billion parameters. With 0.516 million trainable parameters, the 2-billion-parameter model reaches an area under the ROC curve of 0.7376 for acute kidney injury, compared with GPT-5.6 Sol's 0.7380 with text input and high reasoning effort. Training for temporal question answering yields 69.27% accuracy at 4 billion parameters with over 90% fewer trainable parameters than low-rank adaptation using charts or numerical text, at a 2.82-4.88 percentage-point accuracy gap. ViSTA extends pretrained language models to numerical prediction and temporal questions.
Subjects:
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as:
arXiv:2609.31448 [cs.CL]
(or
arXiv:2609.31448v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.31448
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Junyi Gao [view email]
[v1]
Fri, 25 Sep 2026 16:06:29 UTC (465 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled ViSTA: A Simple Bridge Extends Visual Alignment to Clinical Time-Series Understanding in Multimodal LLMs, by Junyi Gao and 3 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CL
new
|
recent
| 2026-09
Change to browse by:
cs
cs.AI
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)