大模型
arxiv.org ·
隐式和显式人口统计信号在基于大语言模型的学生评估中的作用
该 arXiv 论文研究学生人口统计信息如何影响 LLM 评估。作者用受控提示区分显式（直接提及人口统计细节）与隐式（以对话历史作为信号）两种设置，在自动作文评分、形成性反馈和元语言问答三类任务上测试六个 SOTA 模型。结果显示模型会捕捉人口统计线索并相应改变评分、反馈与答案：显式提及教育水平时反馈可读性常被调整；隐式条件下则出现不可预测偏差，例如低教育水平回答在问答中获得更低情感分数。论文据此指出 LLM 在教育评估任务中存在人口统计敏感性。
AI推荐理由 arXiv 论文（cs.CL/cs.AI/cs.CY），作者为 Donya Rooein、Luca Benedetto、Dirk Hovy，摘要明确给出实验设计：用受控提示测试显式与隐式人口统计信号，覆盖自动作文评分、形成性反馈、元语言问答三类任务，测试六个 SOTA LLM。具体新信息是：显式提及教育水平时模型会调整反馈可读性；隐式条件下产生不可预测偏差，如低教育水平回答在问答中获得更低情感分数。局限是仅摘要级证据，未提供具体模型名单、样本量与偏差幅度，结论为人口统计敏感性存在，而非已量化的实际部署效果。
72
正文
Computer Science > Computation and Language
arXiv:2609.16993 (cs)
[Submitted on 15 Sep 2026]
Title:The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment
Authors:Donya Rooein, Luca Benedetto, Dirk Hovy
View a PDF of the paper titled The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment, by Donya Rooein and 2 other authors
View PDF
HTML (experimental)
Abstract:Large Language Models are now common in student assessment, but we know little about how student demographics affect their use. Sometimes, considering student demographics may be necessary -- for example, to improve readability for users with lower educational levels. However, it also risks being a cause of discrimination, e.g., when assigning lower scores to students from lower socioeconomic backgrounds. We set up controlled prompts to test 1) explicit demographic effects, where we mention demographic details directly, and 2) implicit effects, where we use conversation history as a demographic signal. We test these settings in three tasks: Automated Essay Scoring, Formative Feedback, and Metalinguistic Question Answering. We test six state-of-the-art LLMs on these tasks. In both explicit and implicit cases, the models pick up on demographic cues and can change their scoring, feedback, and answers accordingly. We find that LLMs frequently adjust the readability of feedback to education levels when these are explicitly mentioned. On the other hand, implicit conditions produce unpredictable biases, such as in question answering, where responses from lower-education levels receive lower sentiment scores. Our results provide clear evidence of demographic sensitivity in LLMs for educational assessment tasks.
Comments:
Subjects:
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
Cite as:
arXiv:2609.16993 [cs.CL]
(or
arXiv:2609.16993v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.16993
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Luca Benedetto [view email]
[v1]
Tue, 15 Sep 2026 11:07:26 UTC (847 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment, by Donya Rooein and 2 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CL
new
|
recent
| 2026-09
Change to browse by:
cs
cs.AI
cs.CY
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)