大模型
arxiv.org ·
采样揭示风格：LLM 激活中提示条件风格轴的无监督、免训练发现
该论文提出一种免训练、提示条件的方法：对单个提示在较高温度下重复采样补全，对池化隐激活做 PCA，并依据两极生成自动标注所得轴。作者用 245 条人类风格标注做两阶段验证，在 Qwen-3.5-4B-Instruct 上 top-2 轴与人类自发请求维度匹配精度 72.8%、宏召回 43.6%，极性生成标签准确率 75.6%，标注者相邻一致率 90.9%。可发现性明显依赖模型：Qwen 系列与 Llama-3.2-3B 能暴露人类显著轴，而 DeepSeek-7B-Chat 精度降至 35.3%，其主成分由结构性而非风格性方差主导。结论是模型自身解码方差上的简单 PCA 可作为低成本风格结构探针，并暴露跨模型组织差异。
AI推荐理由 arXiv 论文，方法明确（对单提示高温重复采样、对池化隐激活做 PCA、自动标注轴），并给出可验证的量化结果：在 Qwen-3.5-4B-Instruct 上 top-2 轴与人类自发请求维度匹配精度 72.8%、宏召回 43.6%，极性生成标签准确率 75.6%，标注者相邻一致率 90.9%；同时报告模型依赖性差异（DeepSeek-7B-Chat 精度降至 35.3%，主成分被结构性方差主导）。新信息在于提出免训练、免监督对比数据的提示条件风格轴发现方法，并揭示跨模型风格结构组织差异。局限是仅摘要级证据、样本与模型范围有限，未提供代码或复现细节。
72
正文
Computer Science > Computation and Language
arXiv:2609.19150 (cs)
[Submitted on 20 Jul 2026 (v1), last revised 18 Sep 2026 (this version, v2)]
Title:Sampling Reveals Style: Unsupervised, Training-Free Discovery of Prompt-Conditional Stylistic Axes in LLM Activations
Authors:Ajit Mallavarapu, Ziwei Gu
View a PDF of the paper titled Sampling Reveals Style: Unsupervised, Training-Free Discovery of Prompt-Conditional Stylistic Axes in LLM Activations, by Ajit Mallavarapu and 1 other authors
View PDF
HTML (experimental)
Abstract:Large language models (LLMs) encode rich stylistic structure in their hidden activations, but discovering which stylistic dimensions are salient for a given prompt typically requires supervised contrastive data. We present a training-free, prompt-conditional alternative: we repeatedly sample completions of a single prompt at elevated temperature, apply Principal Component Analysis (PCA) to the pooled hidden activations, and label the resulting axes automatically from the pole generations. We validate the discovered axes against 245 human-elicited stylistic annotations in a two-phase study. On our strongest model (Qwen-3.5-4B-Instruct), the top two axes match spontaneously requested human dimensions with 72.8% precision and 43.6% macro-recall, and 75.6% of validity ratings judge the axes' polar generations accurate to their labels, with 90.9% adjacent inter-annotator agreement. Discoverability is strongly model-dependent: both Qwen models and Llama-3.2-3B expose human-salient axes, while DeepSeek-7B-Chat drops to 35.3% precision, its leading components dominated by structural rather than stylistic variance. Simple PCA over a model's own decoding variance is thus an effective, low-cost probe of stylistic structure in LLM representations, one that also exposes sharp cross-model differences in how that structure is organized.
Subjects:
Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as:
arXiv:2609.19150 [cs.CL]
(or
arXiv:2609.19150v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.19150
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: Ajit Mallavarapu [view email]
[v1]
Mon, 20 Jul 2026 11:55:26 UTC (480 KB)
[v2]
Fri, 18 Sep 2026 12:42:25 UTC (468 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Sampling Reveals Style: Unsupervised, Training-Free Discovery of Prompt-Conditional Stylistic Axes in LLM Activations, by Ajit Mallavarapu and 1 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CL
new
|
recent
| 2026-09
Change to browse by:
cs
cs.LG
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)