论文研究
arxiv.org ·
以帧级定位增强大型音频-语言模型，实现细粒度时间感知
该 arXiv 论文针对大型音频-语言模型（LALM）在细粒度时间感知、尤其是精确事件定位上的不足，指出现有做法主要靠后训练让模型把事件边界预测为时间戳 token，但这种生成式形式缺乏时间戳预测与细粒度声学证据之间的显式对应，限制了定位精度与可靠性。作者提出为 LALM 增加一个专用的帧级 grounding 模型，同时利用其语义建模能力表示事件查询：冻结的 LALM 以音频为上下文编码事件查询，grounding 模型将这些查询表示与细粒度音频特征结合，在帧级定位目标事件。论文称在多个时间定位基准上取得一致且明显的提升，并称该 grounding 模型可为下游推理提供时间证据。
AI推荐理由 arXiv 新提交论文（2026-09-14），提出用独立的帧级 grounding 模型增强大型音频-语言模型（LALM），以解决现有将事件边界预测为时间戳 token 的生成式方法缺乏时间戳与细粒度声学证据显式对应、定位精度和可靠性受限的问题。方法上冻结 LALM 编码事件查询与音频上下文，grounding 模型结合查询表示与细粒度音频特征在帧级定位目标事件，并在多个时间定位基准上报告了一致提升，还称 grounding 模型可为下游推理提供时间证据。局限：仅有摘要，未提供具体数据集、指标数值和消融细节，提升幅度无法核实，且为单篇预印本、未见同行评审或代码发布。
72
正文
Computer Science > Sound
arXiv:2609.15215 (cs)
[Submitted on 14 Sep 2026]
Title:Augmenting Large Audio-Language Models with Frame-Level Grounding for Fine-Grained Temporal Perception
Authors:Yanfeng Shi, Yan Song, Junhui Li, Tinggan Huang, Wu Guo, Haoyu Song, Ian McLoughlin
View a PDF of the paper titled Augmenting Large Audio-Language Models with Frame-Level Grounding for Fine-Grained Temporal Perception, by Yanfeng Shi and 6 other authors
View PDF
HTML (experimental)
Abstract:Large Audio-Language Models (LALMs) have substantially advanced general audio understanding, yet they remain limited in fine-grained temporal perception, particularly in precise event localization. Existing approaches primarily post-train LALMs to predict event boundaries as timestamp tokens. However, this generative formulation lacks explicit correspondence between the timestamp predictions and fine-grained acoustic evidence, limiting the precision and reliability of temporal localization. To address this issue, we augment the LALM with a dedicated frame-level grounding model while leveraging its semantic modeling capability to represent the event query. Specifically, the frozen LALM encodes the event query with audio as context, and the grounding model combines these query representations with fine-grained audio features to localize the target event at the frame level. Extensive experiments across diverse temporal grounding benchmarks demonstrate strong and consistent improvements over existing methods. Further evaluation shows that the grounding model can provide temporal evidence to support downstream reasoning.
Comments:
Subjects:
Sound (cs.SD); Artificial Intelligence (cs.AI)
Cite as:
arXiv:2609.15215 [cs.SD]
(or
arXiv:2609.15215v1 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2609.15215
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Yanfeng Shi [view email]
[v1]
Mon, 14 Sep 2026 08:35:35 UTC (203 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Augmenting Large Audio-Language Models with Frame-Level Grounding for Fine-Grained Temporal Perception, by Yanfeng Shi and 6 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.SD
new
|
recent
| 2026-09
Change to browse by:
cs
cs.AI
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)