大模型
arxiv.org ·
面向嵌入向后兼容性的多模态知识保持适配器
该论文提出 MKP-Adapter，一种面向多模态大语言模型（MLLM）的纯适配器式向后兼容训练（BCT）方法，无需更新主干模型即可让新查询嵌入与既有数据库嵌入兼容，从而避免昂贵的数据库重建索引。作者指出纯适配器 BCT 的核心挑战在于同时保持新嵌入知识与向后兼容，因此设计多级保持损失以维持嵌入空间几何结构，并加入 focal 重加权策略优先学习困难样本。实验在图像、文本、视觉文档和视频检索等多模态基准及多种模型类型上取得较强兼容性；方法仅基于预抽取嵌入训练，相对原始主干前向的额外延迟可忽略。
AI推荐理由 arXiv 新论文，提出 MKP-Adapter，是首个面向多模态大模型（MLLM）的纯适配器式向后兼容训练（BCT）方法，无需更新主干模型。核心新信息：识别出纯适配器 BCT 的主要难点是兼顾新嵌入知识保持与向后兼容，为此提出多级保持损失以维持嵌入空间几何结构，并引入 focal 重加权策略优先学习困难样本；实验覆盖图像、文本、视觉文档和视频检索任务，且仅用预抽取嵌入训练、额外延迟可忽略。局限：仅依据摘要，尚无第三方复现或生产验证，实际收益需进一步评估。
72
正文
Computer Science > Computer Vision and Pattern Recognition
arXiv:2609.16875 (cs)
[Submitted on 15 Sep 2026]
Title:Multi-modal Knowledge Preserving Adapter for Embedding Backward Compatibility
Authors:Jaeseok Byun, Gukyeong Kwon, Han-Kai Hsu, Meher Gitika Karumuri, Zhikang Zhang, Hao Yang, Davide Modolo
View a PDF of the paper titled Multi-modal Knowledge Preserving Adapter for Embedding Backward Compatibility, by Jaeseok Byun and 6 other authors
View PDF
HTML (experimental)
Abstract:Upgrading embedding models typically requires expensive database re-indexing, as new query embeddings are incompatible with existing database embeddings. While Backward Compatible Training (BCT) mitigates this by enforcing compatibility during training, existing approaches often require updating the backbone model. This is impractical because of significant training cost, the risk of performance regression, and limited access to proprietary model weights. We introduce Multi-modal Knowledge Preserving Adapter (MKP-Adapter), the first adapter-only BCT approach for Multi-modal Large Language Models (MLLMs) that requires no backbone updates. We identified that the primary challenge in adapter-only BCT is preserving the knowledge of the new embeddings while enforcing backward compatibility. Hence, we propose a multi-level preservation loss that maintains the geometric structure of the embedding spaces throughout BCT. Furthermore, a focal re-weighting strategy is integrated to prioritize learning from challenging samples. Experiments demonstrate that our method achieves strong backward compatibility across diverse multi-modal benchmarks (image, text, visual document, and video retrieval tasks) and model types. Notably, MKP-Adapter is trained solely on pre-extracted embeddings and requires only negligible additional latency relative to the original backbone forward pass, highlighting its efficiency.
Comments:
Subjects:
Computer Vision and Pattern Recognition (cs.CV)
Cite as:
arXiv:2609.16875 [cs.CV]
(or
arXiv:2609.16875v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2609.16875
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Jaeseok Byun [view email]
[v1]
Tue, 15 Sep 2026 09:06:00 UTC (529 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Multi-modal Knowledge Preserving Adapter for Embedding Backward Compatibility, by Jaeseok Byun and 6 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CV
new
|
recent
| 2026-09
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)