最新资讯
dev.to 如何在专用 GPU 服务器上用 OpenAI 兼容 API 部署 LLM dev.to 审计 153 份 Claude Code 与 Codex 会话记录:User-Agent 与 URL 中的邮箱泄露 dev.to 你的 Agent Harness 应具备的 17 项能力 dev.to 离线 AI 记忆中 0.8 余弦阈值为何失效 dev.to Kimi K3.1 Pro 开放权重:自托管 2.8T 模型前的开发者清单 dev.to 如何通过工具调用引导模型以提升 Harness 质量 dev.to AI Agent 审计轨迹不等于对话记录 dev.to 精简 MCP server 工具数量:更少工具、更小影响面,双重收益 dev.to 为什么 git checkout 无法撤销 AI 编码 agent 的改动? wechat AI短剧门槛降降降!实测即梦短剧工作室:剧本分镜预演一条龙,伙伴计划长期护航 dev.to 让模型改写文档前,先量化事实保真度 dev.to 回放 153 个 Claude Code 与 Codex 会话,审计 Agent 的生产影响范围
多元动态
登录 注册
行业动态 techxplore.com ·

AI代理能记忆,黑客可‘投毒’其记忆——新的网络安全威胁

原文 我的收藏

新一代AI代理能记住先前交互信息、规划步骤并使用工具完成任务,但也面临记忆被恶意篡改的风险,构成新的网络安全挑战。

AI推荐理由 讨论AI代理的记忆功能及其被黑客‘投毒’的新安全威胁,对AI安全领域有实际参考价值。

75

正文

September 3, 2026 AI agents can now remember and hackers can 'poison' their memories—a new cybersecurity threat by Abbas Yazdinejad, The Conversation edited by Swati Mestri, reviewed by Andrew Zinin Swati Mestri Scientific Editor Meet our editorial team Behind our editorial process Andrew Zinin Chief Editor Meet our editorial team Behind our editorial process Editors' notes This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility: fact-checked peer-reviewed publication trusted source written by researcher(s) proofread The GIST Add as preferred source The important part of memory poisoning is the delay, as a poisoned AI agent may not immediately behave like a compromised system. Credit: Unsplash/Xavier Cee Artificial intelligence systems are starting to do more than answer questions. New AI "agents" can remember information from previous interactions, plan a series of steps and use digital tools to complete tasks. Memory is part of what makes these systems useful. But my recent research, conducted with my colleague Hadis Karimipour at the University of Calgary, shows that memory can also create a security weakness that is easy to overlook. Think of an AI agent as an assistant that keeps a notebook of what it learns. Each time it completes a task, useful information can be written into the notebook and consulted later. Now imagine that someone manages to slip a misleading instruction into that notebook. The attacker may not need to take control of the AI directly. The agent can continue working normally for some time. But days—or several interactions—later, it may open its notebook, retrieve the poisoned information and treat it as something it previously learned and can trust. This is known as memory poisoning, and the important part is the delay. A poisoned AI agent may not immediately behave like a compromised system. Like a notebook, an AI agent’s persistent memory can carry information from one interaction to the next. Credit: Pexels/Sergey Torbik An attack that waits Many familiar cybersecurity attacks produce effects relatively quickly. A malicious link is clicked, malware executes or a stolen password is used to access an account. Memory poisoning can work differently. In our research, we examined 2,614 simulated multi-step attack trajectories involving memory-enabled large language model agents. We studied four types of attacks: chain poisoning, policy rewriting, backdoor triggering and slow drift. Rather than asking only whether an attack succeeded, we examined what happened to the agent over time. That distinction matters. Imagine someone secretly adding a sentence to an employee's notebook saying: "Requests from this person have already been approved." Nothing necessarily happens when the sentence is written. The employee might complete several unrelated tasks normally. The problem emerges later, when a relevant request arrives and the employee consults the notebook. An AI agent with persistent memory can face a similar problem. Our experiments showed that some attacks remained difficult to distinguish from normal behavior through much of an interaction and became apparent only later. In particular, slow-drift and backdoor-trigger attacks could evade evaluations that looked only at individual steps until their effects appeared in later interactions. Checking once may not be enough This creates a problem for how we test AI security. Suppose a security team examines an AI agent immediately after it encounters suspicious information. The agent appears to behave normally, so the interaction is judged safe. That may be like inspecting a notebook immediately after someone has inserted a misleading entry but before anyone has acted on it. The absence of immediate harmful behavior does not necessarily mean the attack failed. Our results also showed that the risk did not always increase in a simple straight line. Some attacks produced what we call "non-monotonic" patterns: behavior could appear more concerning at one stage and less concerning at another before the attack ultimately developed. This means testing an AI agent one prompt or one interaction at a time can miss part of the picture. Instead, security evaluations may need to follow the agent across a sequence of interactions—essentially watching the whole story rather than examining individual photographs. A new security problem This issue is becoming more important as AI systems develop from chatbots that respond to individual questions to agents designed to perform longer tasks. An ordinary chatbot conversation can often be treated as relatively self-contained. A memory-enabled agent is different because yesterday's information may influence tomorrow's decision. When an AI agent can also use tools, the consequences of a poisoned memory can extend beyond generating an incorrect sentence. An agent might eventually use remembered information when deciding what action to take, which resource to access or which instruction to follow. This doesn't mean that memory-enabled AI agents are inherently unsafe. Memory provides important benefits: it allows an agent to maintain context, learn user preferences and work on tasks that cannot be completed in a single interaction. But it changes what defenders need to protect. Securing the prompt in front of an AI system is no longer necessarily enough. The information that the system carries forward may also need protection. From snapshots to stories There is a simple lesson from our research: When an AI system has memory, security also has a memory. If an attack can be planted at one moment and activated much later, evaluating only the moment when malicious information first appears—or only the moment when something goes wrong—can leave out what happened in between. That is why we argue for trajectory-aware security testing: evaluating how an AI agent's behavior develops across multiple interactions rather than judging each step in isolation. For non-specialists, the idea is perhaps easier to understand without the technical terminology. If you want to know whether an assistant's notebook has been compromised, you cannot simply watch the assistant write one page. You also need to pay attention to what the assistant remembers—and what happens when it eventually opens that notebook again. Publication details Abbas Yazdinejad et al, Temporal Dynamics of Memory Poisoning in Web3-Style LLM Agents, IEEE Access (2026). DOI: 10.1109/access.2026.3693560 Journal information: IEEE Access Key concepts Cybersecurity frameworksAI surveillance capitalism Provided by The Conversation Who's behind this story? Swati Mestri Swati Mestri holds a bachelor's degree in Electronics Engineering and has worked as a content editor since 2019. She has experience editing research documents across technology, health care, and materials science, and has a particular interest in technology and space. Full profile → Andrew Zinin Master's in physics with research experience. Long-time science news enthusiast. Plays key role in Science X's editorial success. Full profile → This article is republished from The Conversation under a Creative Commons license. Read the original article. Citation: AI agents can now remember and hackers can 'poison' their memories—a new cybersecurity threat (2026, September 3) retrieved 8 September 2026 from https://techxplore.com/news/2026-09-ai-agents-hackers-poison-memories.html This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only. Hidden prompts can plant false memories in AI agents, researchers warn Jul 19, 2026 How attackers persuade AI agents to break the rules Aug 19, 2026 AI agents open door to new hacking threats Nov 11, 2025 Researchers put six AI agents on Discord for two weeks, exposing risky failures Mar 10, 2026 New report identifies types of cyberattacks that manipulate behavior of AI systems Jan 4, 2024 Button‑pushing explorers: How to grasp that AI agents can do amazing things while knowing nothing May 12, 2026 Memristor chip breaks the capacity limit of brain-inspired associative memory 5 hours ago Invention may bring smart bionic eye into sight 5 hours ago Whole-body expansion and contraction make a robot seem more alive 6 hours ago A tiny crystal tilt may bring greener magnetic memory closer to everyday devices 7 hours ago Tracking energy flow reveals voltage pattern that slashes waste heat by over 60% 8 hours ago Editorial proposes limits on generative AI's role in scientific research 13 hours ago

来源报道 · 1 篇 · 以下链接均前往原始媒体

← 返回精选动态 原文

更多内容