论文研究
arxiv.org ·
面向多模态模型合并的跨层影响追踪与协调
该论文提出 TAC-Merge，用于多模态模型合并中的跨层影响追踪与协调。作者指出，现有单模态与多模态合并方法多在单层内组合专家更新，但专家更新会改变传递给后续层的表示，使其影响沿深度传播并影响视觉与文本信息的交互，导致先后更新相互耦合。TAC-Merge 包含两个模块：多模态影响映射（MIM）构建更新效应图，并结合 Ricci 曲率与专家预测定义共享融合目标；耦合合并控制（CMC）对系数调整间的交互建模，联合优化区域权重以合成单一共享模型。论文称在多种多模态任务上验证了该方法整合互补专家能力并支持对未见任务泛化的效果，但摘要未提供具体指标与对比数据。
AI推荐理由 arXiv 新提交论文（2026-09-11），提出 TAC-Merge 方法，针对多模态模型合并中专家更新跨层传播导致视觉与语言更新相互耦合的问题，设计了多模态影响映射（MIM，用 Ricci 曲率与专家预测构建更新效应图）和耦合合并控制（CMC，联合优化区域权重）两个模块。属于模型合并这一具体技术方向的新方法，对多模态模型复用与任务专家整合有参考价值；但摘要仅称在多种多模态任务上验证有效性，未给出具体指标、基线或消融数据，实际收益无法从摘要判断，且尚未见同行评审或复现证据。
72
正文
Computer Science > Artificial Intelligence
arXiv:2609.12897 (cs)
[Submitted on 11 Sep 2026]
Title:Tracing and Coordinating Cross-Layer Influence for Multimodal Model Merging
Authors:Pengyang Zhou, Xiaobin Tu, Zhengxi Liu, Rongkun Xue, Haochen Li, Miancan Liu, Ziyuan Chen, Yinggui Wang, Jinkui Ren, Xiantao Zhang
View a PDF of the paper titled Tracing and Coordinating Cross-Layer Influence for Multimodal Model Merging, by Pengyang Zhou and 9 other authors
View PDF
HTML (experimental)
Abstract:Multimodal model merging aims to consolidate task experts into a single model that retains their complementary capabilities. Most unimodal model merging methods combine expert updates within individual layers, and multimodal approaches largely follow this design. However, an expert update changes the representations passed to subsequent layers, allowing its influence to propagate across depth and affect how visual and textual information interact. When visual and language updates are combined, later updates act on inputs already modified by earlier ones, coupling their effects. This poses two challenges: (1) how to characterize the multimodal influence of individual expert updates across depth, and (2) how to jointly combine expert updates based on their multimodal influence. To address these challenges, we propose TAC-Merge for tracing and coordinating cross-layer influence in multimodal model merging. It contains two modules, i.e., multimodal influence mapping (MIM) and coupled merge control (CMC). MIM constructs graphs of update effects and uses Ricci curvature together with expert predictions to define a shared fusion objective. CMC models interactions among coefficient adjustments and jointly optimizes regional weights to synthesize one shared model. Experiments across diverse multimodal tasks demonstrate the effectiveness of TAC-Merge in consolidating complementary expert capabilities and supporting generalization to unseen tasks.
Subjects:
Artificial Intelligence (cs.AI)
Cite as:
arXiv:2609.12897 [cs.AI]
(or
arXiv:2609.12897v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.12897
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Pengyang Zhou [view email]
[v1]
Fri, 11 Sep 2026 14:24:07 UTC (292 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Tracing and Coordinating Cross-Layer Influence for Multimodal Model Merging, by Pengyang Zhou and 9 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.AI
new
|
recent
| 2026-09
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)