研究：聊天机器人越健谈越易误导用户，验证工具有助纠偏

September 15, 2026 Chatty chatbots can mislead users, but verification tools help, researchers find by Jonathan F. McVerry, Pennsylvania State University edited by Lisa Lock, reviewed by Andrew Zinin Lisa Lock Scientific Editor Meet our editorial team Behind our editorial process Andrew Zinin Chief Editor Meet our editorial team Behind our editorial process Editors' notes This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility: fact-checked trusted source proofread The GIST Add as preferred source The verification website used in a study by Maggie Liao and S. Shyam Sundar. The researchers found that verification tools can help users become more skeptical when evaluating AI-generated information. Credit: Jonathan F. McVerry / Penn State As artificial intelligence (AI)-powered chatbots become more popular and more humanlike in the way they communicate, users are increasingly turning to them as trusted sources of information on topics ranging from personal finance to health. That may be problematic, according to Penn State researchers, who found in a recent study that the more conversational an AI chatbot is, the more likely users are to trust it—even when it shares misinformation. The team said verification options may help rebuild needed skepticism. In their study, published in the Journal of Computer-Mediated Communication, the researchers found that the more conversational or personable the chatbot, the more users curbed their "negative machine heuristic," or the tendency to believe that machines lack the human touch in terms of intuition and subjective decision-making. This was the case even when the information was "ridiculously wrong," they said. "The conversational delivery of information by generative AI is hyper-specific to your particular question, so it seems very tailored," said co-author S. Shyam Sundar, Evan Pugh University Professor and James P. Jimirro Professor of Media Effects at Penn State. "This is the first time in the history of technology where a machine can actually hold forth a conversation that has not been predicted by a programmer or has not been prescripted." Sundar and Maggie Liao, lead author and assistant professor at the University of Georgia, have conducted several studies on how people use and trust AI chatbots. Knowing that the technology can sometimes produce inaccurate information, they designed this study to find out whether a conversational chatbot makes users more likely to trust misinformation. "We tested whether the conversational nature of GenAI chatbots is driving users to trust misinformation coming from them," said Liao, who completed her doctoral studies with Sundar at Penn State. "And if that is the case, the natural next question is: What can we do about it?" The researchers conducted an online experiment with 477 participants who were told they were helping test a new AI health assistant. Each participant was asked to submit one prompt from a pool of sample prompts to the AI chatbot. During the interaction, the AI provided incorrect information. Participants were then asked to indicate how credible they thought the chatbot was. The study varied how conversational the AI was—from low to high—with the participants. A second part of the study tested whether adding a verification option to the chatbot experience offset misplaced trust in AI-produced information. Modeled after the standard human fact-checking process, the system gave users the opportunity to confirm whether the AI responses were backed by evidence. Offering a verification option, the scholars theorized, could counter a common assumption among users that AI is always more objective and accurate than humans. The four possible verification options were: a simple cue with a small icon warning that information may be incorrect; a required check with a button that users must click to proceed; an optional check with a button that users could click if they chose to check the information; and no verification option. The researchers found that users who could verify the information were more skeptical of the AI's responses. However, a verification cue alone wasn't enough. Trust in the AI system decreased only among users who actively clicked the verification buttons, the researchers said. "The verification option is a useful antidote to the conversational persuasiveness of GenAI," Sundar said. "That's what we say in the title of our study: 'Chat, but verify.' Enjoy the conversation by all means, but don't be lulled into believing all that you're getting." Participants who received highly conversational AI responses were more likely to find the information credible than those interacting with less conversational versions. This was the case even when the information was blatantly inaccurate. "Some of the misinformation we used in the study was ridiculous," Liao said. "One example was that ginger can be more effective than chemotherapy in treating cancer. It sounds ridiculous, but if it is given by a very conversational AI, people trust it more. I think that's worrisome." According to the researchers, users who verified information were less likely to accept false claims as accurate. When given the option, nearly half of the participants verified the information they were given. Sundar said the results from both parts of the study can serve as a warning sign and provide a solution to AI developers. "Hopefully, research showing that verification tools help combat misinformation will persuade AI platforms to take them seriously and develop more such checks on information generated by AI," he said. More information Mengqi Liao et al, Chat but verify: Combating misinformation in conversational Generative AI with verification affordance, Journal of Computer-Mediated Communication (2026). DOI: 10.1093/jcmc/zmag012 Key concepts AI chatbot safety Provided by Pennsylvania State University Who's behind this story? Lisa Lock BA art history, MA material culture. Former museum editor, paramedic, and transplant coordinator. Editing for Science X since 2021. Full profile → Andrew Zinin Master's in physics with research experience. Long-time science news enthusiast. Plays key role in Science X's editorial success. Full profile → Citation: Chatty chatbots can mislead users, but verification tools help, researchers find (2026, September 15) retrieved 15 September 2026 from https://techxplore.com/news/2026-09-chatty-chatbots-users-verification-tools.html This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only. Why do we trust chatbots and how can we use them more wisely? A psychologist explains Sep 2, 2026 The consequences of relying on AI for accurate news Jun 10, 2026 Popular AI chatbots are confidently dispensing medical misinformation, analysis shows Apr 14, 2026 AI chatbots can run with medical misinformation, highlighting need for stronger safeguards Aug 6, 2025 Here's how researchers are helping AIs get their facts straight Feb 10, 2025 Study finds source validation issues hurt ChatGPT reliability May 9, 2023 OpenAI, Anthropic and Google are working to create an AI standards body 1 hour ago AI imagines a harsher social world, expecting punishment where people expect inaction 5 hours ago AI can sound empathetic and human—but not at the same time 4 hours ago Rocket experiment tracks how liquid metal solidifies in microgravity 6 hours ago New cloud security vulnerability uncovered 5 hours ago Periodic voltage fluctuations can 'see' inside lithium batteries, no disassembly required 22 hours ago