大模型
arxiv.org ·
简短 AI 素养干预未能显著减少对 ChatGPT 的过度依赖，反而增加依赖不足：一项随机研究
该随机实验将高中生分为对照组（仅简要介绍 LLM）与干预组（额外讲解 LLM 原理、局限与使用策略），随后让其在 ChatGPT 建议下完成八道数学谜题，其中一半建议为错误。结果显示过度依赖普遍存在，错误建议被采纳的比例达 52.1%；干预未显著降低过度依赖，反而使学生更易拒绝正确建议，即依赖不足上升。作者认为这提供了初步证据，表明简短文本式干预难以培养恰当依赖，可能需要更全面、互动性更强的方法。
AI推荐理由 arXiv 同行预印本，随机对照实验，样本为高中生，给出具体数据（错误建议采纳率 52.1%），结论明确且反直觉：简短 AI 素养干预未显著降低过度依赖，反而增加对正确建议的拒绝（依赖不足）。对 AI 教育、人机协作与 LLM 使用规范有直接参考价值；局限是单一任务（数学谜题）、文本式干预、样本与场景有限，作者自称初步证据。
72
正文
Computer Science > Computers and Society
arXiv:2503.10556 (cs)
[Submitted on 13 Mar 2025 (v1), last revised 18 Sep 2026 (this version, v2)]
Title:A Brief AI Literacy Intervention Does Not Significantly Reduce Over-Reliance and Increases Under-Reliance on ChatGPT: A Randomized Study
Authors:Brett Puppart, Jaan Aru
View a PDF of the paper titled A Brief AI Literacy Intervention Does Not Significantly Reduce Over-Reliance and Increases Under-Reliance on ChatGPT: A Randomized Study, by Brett Puppart and Jaan Aru
View PDF
Abstract:In this study, we examined whether a brief AI literacy intervention influences high school students' reliance on recommendations from large language models (LLMs). In a randomized experiment, students were assigned to either a control group receiving a brief introduction to LLMs or an intervention group receiving additional information about how LLMs work, their limitations, and effective usage strategies. Participants then solved eight math puzzles with ChatGPT's advice, which was incorrect in half of the trials. Results indicated widespread over-reliance, with incorrect recommendations adopted in 52.1% of the trials. The intervention did not significantly reduce over-reliance. Instead, it led to an increase in under-reliance, as students were more likely to reject correct recommendations. These findings provide preliminary evidence that brief text-based interventions may be ineffective in fostering appropriate reliance. More comprehensive and interactive approaches may be required to meaningfully influence students' real-world reliance on LLMs.
Subjects:
Computers and Society (cs.CY); Neurons and Cognition (q-bio.NC)
Cite as:
arXiv:2503.10556 [cs.CY]
(or
arXiv:2503.10556v2 [cs.CY] for this version)
https://doi.org/10.48550/arXiv.2503.10556
Focus to learn more
arXiv-issued DOI via DataCite
Submission history
From: Jaan Aru [view email]
[v1]
Thu, 13 Mar 2025 17:10:33 UTC (966 KB)
[v2]
Fri, 18 Sep 2026 13:26:13 UTC (1,066 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled A Brief AI Literacy Intervention Does Not Significantly Reduce Over-Reliance and Increases Under-Reliance on ChatGPT: A Randomized Study, by Brett Puppart and Jaan Aru
View PDF
view license
Current browse context:
cs.CY
new
|
recent
| 2025-03
Change to browse by:
cs
q-bio
q-bio.NC
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)