大模型
arxiv.org ·
LLM 训练前与训练后数据中的政治内容是什么？
该 arXiv 论文（v3）直接量化开源 LLM 预训练与后训练数据中的政治内容：结合大规模采样、政治倾向分类和立场检测，发现所有训练数据集系统性偏向左翼内容，预训练语料的政治参与度高于后训练语料；训练数据中的政治立场与模型行为存在强相关，且该相关性在多数基座模型中已出现并延续至后训练阶段。作者据此强调数据构成与模型行为的关联，并呼吁提升数据透明度以理解和监测模型行为。
AI推荐理由 arXiv 论文（v3，2026-09-21 修订），作者 Tanise Ceron 等，研究 LLM 训练前/后数据中的政治内容。具体新信息：对开源 LLM 训练数据做大规模采样、政治倾向分类与立场检测，发现所有训练数据集系统性偏左，预训练比后训练含更多政治参与文本，且训练数据立场与模型行为强相关，该相关性在多数基座模型中已存在并贯穿后训练阶段。用途：为数据透明度与模型行为监测提供可验证证据。局限：摘要未给出具体数据集名称、样本量与效应量，结论基于开源模型，不能外推到闭源模型。
78
正文
Computer Science > Computation and Language
arXiv:2509.22367 (cs)
[Submitted on 26 Sep 2025 (v1), last revised 21 Sep 2026 (this version, v3)]
Title:What Is The Political Content in LLMs' Pre- and Post-Training Data?
Authors:Tanise Ceron, Dmitry Nikolaev, Dominik Stammbach, Debora Nozza
View a PDF of the paper titled What Is The Political Content in LLMs' Pre- and Post-Training Data?, by Tanise Ceron and 3 other authors
View PDF
HTML (experimental)
Abstract:Large language models (LLMs) reflect politically-slanted opinions in their generated text. Even though it is widely assumed that model behavior stem from training data, there has been no study quantifying the extent to which political content is part of the training data. To bridge this gap, we aim to directly estimate (1)~the proportion of politically engaged texts in training data, (2)~respective data imbalance, (3)~cross-dataset similarity, and (4)~correlations between data composition and model behaviour. We analyze the political content of pre- and post-training datasets of open-source LLMs, combining large-scale sampling, political-leaning classification, and stance detection. We find that all LLM training datasets are systematically skewed towards left-leaning content, with pre-training containing more politically engaged than post-training corpora. We further observe a strong correlation between political stances in training data and model behavior, which is present already in most base models and persists across post-training stages. These findings highlight the role of data composition in correlating with model behavior and motivate the need for greater data transparency as a means to understand and monitor model behavior.
Comments:
Subjects:
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
Cite as:
arXiv:2509.22367 [cs.CL]
(or
arXiv:2509.22367v3 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2509.22367
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: Tanise Ceron [view email]
[v1]
Fri, 26 Sep 2025 14:00:51 UTC (4,408 KB)
[v2]
Fri, 3 Apr 2026 11:15:05 UTC (3,726 KB)
[v3]
Mon, 21 Sep 2026 16:42:44 UTC (739 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled What Is The Political Content in LLMs' Pre- and Post-Training Data?, by Tanise Ceron and 3 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CL
new
|
recent
| 2025-09
Change to browse by:
cs
cs.AI
cs.CY
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)