大模型
arxiv.org ·
X-AuT：面向语音大语言模型的跨尺度蒸馏渐进式音频编码器压缩
该论文提出 X-AuT 渐进式压缩框架，通过短行为探针选择层组合，并用表征对齐、跨尺度蒸馏、学生策略监督和 LoRA 微调恢复剪枝后的语音 LLM。语言模型主干冻结，仅注意力 LoRA 适配器与绑定输出嵌入参与蒸馏。在十个中英公开基准上，Qwen3-ASR-0.6B 由 18 层压至 16 层，宏平均错误率从 5.61% 降至 5.27%；14 层模型为 5.75%，音频塔参数减少 20.7%。相同配方下 1.7B 教师蒸馏为 5.55%，自蒸馏为 8.45%，渐进式 18→14 剪枝优于直接剪枝（5.75% vs 6.73%）。作者说明这些为单次运行结果，精度影响因基准而异。
AI推荐理由 arXiv 新论文，提出 X-AuT 渐进式音频编码器压缩框架，通过短行为探针选层、表征对齐、跨尺度蒸馏、学生策略监督与 LoRA 微调恢复剪枝模型，语言模型主干保持冻结。在十个中英公开基准上，Qwen3-ASR-0.6B 从 18 层压到 16 层使宏平均错误率由 5.61% 降至 5.27%，14 层为 5.75% 且音频塔参数减少 20.7%；1.7B 教师蒸馏得 5.55%，自蒸馏为 8.45%，渐进式 18→14 剪枝优于直接剪枝（5.75% vs 6.73%）。局限是单次运行结果，作者也指出精度影响因基准而异，尚需复现验证。
72
正文
Computer Science > Sound
arXiv:2609.11412 (cs)
[Submitted on 10 Sep 2026]
Title:X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation
Authors:Haojun Zhang, Yi Zou, Min Chen, Qize Yu, Lianrui Fan, Xini Ding, Hao Li, Shuchang Zhou, Xianming Liu, Shiyu Huang
View a PDF of the paper titled X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation, by Haojun Zhang and 9 other authors
View PDF
HTML (experimental)
Abstract:Reducing audio-encoder depth lowers the inference cost of speech large language models, but removing complete blocks perturbs the embeddings consumed by the decoder and can cause deletion and premature end-of-sequence errors. We introduce X-AuT, a progressive framework that selects layer combinations through short behavioral probes and restores the pruned model through representation alignment, cross-scale distillation, scheduled student-policy supervision, and LoRA finetuning. The language-model backbone remains frozen, while attention LoRA adapters and the tied output embedding adapt during distillation. Training uses the highest-agreement tier from a transcript-consistency pipeline, followed by source reweighting during finetuning. On ten public Chinese--English benchmarks, compressing Qwen3-ASR-0.6B from 18 to 16 audio-encoder layers reduces macro-average error from 5.61% to 5.27%. The 14-layer model reaches 5.75% with 20.7% fewer audio-tower parameters. Under the matched recipe, the 1.7B teacher yields 5.55% mean error, compared with 8.45% for self-distillation, and progressive 18$\rightarrow$14 pruning outperforms direct pruning (5.75% vs. 6.73%). These single-run results establish two practical operating points and show that the accuracy effects vary across benchmarks. Project website: this https URL
Subjects:
Sound (cs.SD); Artificial Intelligence (cs.AI)
Cite as:
arXiv:2609.11412 [cs.SD]
(or
arXiv:2609.11412v1 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2609.11412
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Shiyu Huang [view email]
[v1]
Thu, 10 Sep 2026 11:43:17 UTC (2,781 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation, by Haojun Zhang and 9 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.SD
new
|
recent
| 2026-09
Change to browse by:
cs
cs.AI
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)