大模型
arxiv.org ·
开放权重 LLM 能否模拟人类调查群体？一项跨工具校准研究
该 arXiv 预印本评估三种开放权重 LLM 家族在跨工具校准任务上的表现：以真实受访者对某一心理测量工具的原话回答作为 persona 条件，再在另一构念距离受控的工具上测量，并与 2058 人人类面板对照。在 139 对网格中，各模型模拟的跨工具相关性与真实人类相关性达 r=0.70–0.73，主要由符号正确而非幅度精确驱动，且集中在中等构念距离的配对。作者认为这对基于 LLM 的行为模拟和数字孪生应用是积极信号，但能力并不随模型版本单调提升——三个 Llama 版本中最新版在三个核心指标中的两个上表现最差，因此实际落地需要针对具体发布版本和构念距离进行验证，而非一次性基准测试。
AI推荐理由 arXiv 预印本，摘要给出明确实验设计：在 2058 人人类面板上，用三种开放权重 LLM 家族做跨工具校准，139 对网格中模拟相关性与真实人类相关性达 r=0.70–0.73，且指出该相关性主要由符号正确而非幅度精确驱动，并发现最新 Llama 版本在三个核心指标中的两个上表现最差。新信息在于开放权重模型无需微调即可复现部分人类跨工具结构，局限是仅基于摘要、未经同行评审，且能力不随版本单调提升，需按发布版本和构念距离分别验证。
72
正文
Computer Science > Artificial Intelligence
arXiv:2609.32638 (cs)
[Submitted on 26 Sep 2026]
Title:Can Open-Weight Large Language Models (LLMs) Simulate Human Survey Populations? A Cross-Instrument Calibration Study
Authors:Grandee Lee, Wang Yue
View a PDF of the paper titled Can Open-Weight Large Language Models (LLMs) Simulate Human Survey Populations? A Cross-Instrument Calibration Study, by Grandee Lee and Wang Yue
View PDF
HTML (experimental)
Abstract:Large language models (LLMs) are increasingly used to generate synthetic survey respondents and digital twins of real people, but whether their output preserves real human statistical structure, rather than surface plausibility, remains unresolved, and most existing evidence comes from proprietary models rather than open-weight ones. We evaluate three open-weight LLM families on a cross-instrument calibration task: conditioning personas on real respondents' verbatim answers to one psychometric instrument and measuring them on a second, construct-distance-controlled instrument, checked against a 2,058-person human panel. Across a 139-pair grid, the simulated cross-instrument correlation tracks the real human correlation at r = 0.70 - 0.73 in every model, driven mainly by correct sign rather than precise magnitude and concentrated in pairs of moderate construct distance. A correlation of this magnitude, obtained from untuned open-weight models conditioned only on individual-level survey data, is a substantively encouraging result for LLM-based behavioral simulation and digital-twin applications: specific model families and releases already reproduce a meaningful share of real human cross-instrument structure without any fine-tuning. This capability does not, however, improve monotonically across model releases: on a matched panel, the newest of three tested Llama releases performs worst on two of three headline metrics, so realizing its promise in practice requires release-specific, distance-aware verification rather than a one-time benchmark.
Subjects:
Artificial Intelligence (cs.AI)
Cite as:
arXiv:2609.32638 [cs.AI]
(or
arXiv:2609.32638v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.32638
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Grandee Lee [view email]
[v1]
Sat, 26 Sep 2026 14:01:34 UTC (78 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Can Open-Weight Large Language Models (LLMs) Simulate Human Survey Populations? A Cross-Instrument Calibration Study, by Grandee Lee and Wang Yue
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.AI
new
|
recent
| 2026-09
Change to browse by:
cs
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)