大模型
arxiv.org ·
评估语言模型用于显著类别识别
该论文研究语言模型能否直接从 commit 中识别显著类别（salient class），即多类修改中主要被修改、并引发其他类改动的类，从而帮助代码审查者定位变更入口。作者基于 ApacheCM 构建新数据集 ApacheJavaCM，包含 7,911 个 commit 和 25,914 个标注类，并与最强可复现 SOTA 基线对比。评估覆盖 GPT-5.4、DeepSeek-V3.2 两个 LLM、Qwen3.5-9B 一个 SLM，以及 zero-shot、few-shot、chain-of-thought 三种提示策略。结果显示 LM 显著优于基线，且在不同 commit 特征与模型间保持稳定；在 few-shot 下，9B 开源 SLM Qwen3.5-9B 的性能可与更大的闭源 GPT-5.4 相当，作者据此认为轻量本地可部署 SLM 是该任务的可行方案，可降低对闭源 LLM 的成本与隐私依赖。
AI推荐理由 arXiv 论文（cs.SE），有明确数据集 ApacheJavaCM（7,911 commits、25,914 标注类）与可复现 SOTA 基线对比，事实与实验设置清晰，信息密度较高。新信息在于：无需特征工程、图构建或训练，直接用 LM 从 commit 识别 salient class，并覆盖 GPT-5.4、DeepSeek-V3.2、Qwen3.5-9B 与 zero-shot/few-shot/CoT 三种提示策略；结论称 9B 开源 SLM 在 few-shot 下可媲美更大闭源模型，对本地部署与隐私/成本有参考价值。局限：摘要未给出具体指标数值与统计显著性，结论为作者自述，尚待复现；任务范围较窄（代码审查场景），对通用 Agent 与产业落地的直接价值有限。
72
正文
Computer Science > Software Engineering
arXiv:2606.21629 (cs)
[Submitted on 19 Jun 2026 (v1), last revised 10 Sep 2026 (this version, v2)]
Title:Assessing Language Models for Salient Class Identification
Authors:Bo Xiong, Chaoran Cai, Zongen Ren, Kaipeng Xiong, Chong Wang, Peng Liang
View a PDF of the paper titled Assessing Language Models for Salient Class Identification, by Bo Xiong and 5 other authors
View PDF
HTML (experimental)
Abstract:Code review requires reviewers to understand the core intent of code changes, which becomes difficult when a commit modifies multiple classes. In such commits, one or more primarily modified classes, referred to as salient classes, may induce modifications in other classes. Accurate identification of salient classes offers reviewers an effective entry point to navigate code changes and facilitates program comprehension. Existing state-of-the-art approaches rely on complex program-analysis procedures, including Abstract Syntax Tree (AST) parsing, class relation extraction, handcrafted feature engineering, or dependency graph construction. To this end, we study whether language models (LMs) can identify salient classes directly from commits without feature engineering, graph construction, or training. We first construct a new dataset ApacheJavaCM, derived from the ApacheCM dataset, containing 7,911 commits and 25,914 labeled classes. On this dataset, we systematically evaluate whether LMs can identify salient classes directly from commits and compare with the strongest reproducible state-of-the-art (SOTA) baseline. The evaluation covers two large language models (LLMs), GPT-5.4 and DeepSeek-V3.2, one small language model (SLM), Qwen3.5-9B, and three prompting strategies: zero-shot, few-shot, and chain-of-thought. The LMs substantially outperform the baseline while remaining stable across commit characteristics and selected LMs. We also found that, for salient class identification tasks, a 9B-parameter open-source SLM, Qwen3.5-9B, under few-shot prompting, achieves performance comparable to that of a much larger closed-source LLM, GPT-5.4. These results suggest that lightweight, locally deployable SLMs are feasible options for the salient class identification task and can reduce both cost and privacy barriers associated with relying on closed-source LLMs.
Comments:
Subjects:
Software Engineering (cs.SE)
Cite as:
arXiv:2606.21629 [cs.SE]
(or
arXiv:2606.21629v2 [cs.SE] for this version)
https://doi.org/10.48550/arXiv.2606.21629
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: Peng Liang [view email]
[v1]
Fri, 19 Jun 2026 17:33:56 UTC (35 KB)
[v2]
Thu, 10 Sep 2026 16:46:02 UTC (34 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Assessing Language Models for Salient Class Identification, by Bo Xiong and 5 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.SE
new
|
recent
| 2026-06
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)