我的 AI agent 连简单任务都失败：检索失败减少 49% 改变了我的调试方式

I used to blame the model. If an agent missed a refund rule, forgot a tool result from 10 seconds ago, or grabbed the wrong SKU from docs, I’d assume GPT-5 or Claude had a reasoning problem. I don’t think that anymore. A lot of "agent is dumb" bugs are retrieval bugs. That sounds obvious in hindsight, but it changes how you debug everything: n8n flows, OpenAI File Search, support bots, internal copilots, and custom agent stacks held together with Redis, Postgres, and optimism. Anthropic’s Contextual Retrieval writeup put hard numbers on something a lot of us have seen in production: 49% fewer failed retrievals with Contextual Retrieval 67% fewer failed retrievals when reranking is added That is not a small lift. That is a giant sign that many agent failures happen before the model even starts reasoning. The failure looked like reasoning. It wasn’t. Here’s the pattern I kept seeing. The agent could: summarize a long PDF call an API correctly produce a decent customer reply follow a multi-step workflow Then it would fail one painfully obvious step: miss the refund window in the policy doc use the wrong product SKU forget a prior tool result ignore a customer-specific exception When that happens, it feels like bad reasoning. But usually one exact fact was missing at one exact moment. That’s not a reasoning failure. That’s failed fetch. The debugging mistake: treating all memory as one thing A lot of teams say "memory" like it’s one subsystem. It isn’t. In practice, you usually have at least 3 different layers: | Memory type | Scope | Best for | |----------|----------| | Session/chat memory | Current conversation or run | Short-term continuity | | Durable memory | Across runs, users, or sessions | Preferences, state, long-lived facts | | Retrieval | Pulling external facts on demand | Docs, policies, tool outputs, exact references | If your n8n agent forgets a tool result from the same run, that’s probably session memory. If it loses a customer preference from yesterday, that’s durable memory. If it can’t find the refund rule that definitely exists in your docs, that’s retrieval. Different bug. Different fix. This is why debugging gets weird when people throw Redis, Postgres, vector search, chat history, and tool outputs into one bucket called "memory." Long context does not magically fix retrieval I still hear this one a lot: We gave the model the docs, so retrieval can’t be the issue. Not true. The Lost in the Middle result is still one of the most annoying realities in production: models often do worse when the relevant info is buried in the middle of a long prompt. So you can have both of these problems: The agent never retrieves the right fact The agent retrieves it, then buries it where the model is less likely to use it That means "the info was technically present" is not a useful defense. If the right fact is hidden in a wall of context, your agent can still fail in a way that looks like reasoning. Pure vector search loses stupid fights This is where I’ll be blunt. If your agent needs exact strings, pure embedding search is not enough. I’m talking about: order IDs policy titles product SKUs workflow names error codes ticket IDs Semantic retrieval is great until you need literal precision. That’s why hybrid retrieval keeps winning in real systems. Keyword search + semantic search + reranking is just more reliable than hoping embeddings will infer everything. Even OpenAI’s retrieval stack leans this way. That should tell you something. A practical OpenAI File Search setup If you’re using OpenAI-compatible tooling, retrieval should be in the agent loop instead of relying on the model to remember everything from prior turns. Example: from openai import OpenAI client = OpenAI() vector_store = client.vector_stores.create(name="Support FAQ") client.vector_stores.files.upload_and_poll( vector_store_id=vector_store.id, file=open("customer_policies.txt", "rb") ) Enter fullscreen mode Exit fullscreen mode That gives the agent something to query instead of forcing conversational recall to do all the work. You can also upload remote files directly: from io import BytesIO import requests from openai import OpenAI client = OpenAI() response = requests.get( "https://cdn.openai.com/API/docs/deep_research_blog.pdf", timeout=30, ) response.raise_for_status() file_content = BytesIO(response.content) result = client.files.create( file=("deep_research_blog.pdf", file_content), purpose="assistants", ) Enter fullscreen mode Exit fullscreen mode None of this is glamorous. That’s exactly why it matters. A lot of agent reliability comes from boring retrieval plumbing. Sometimes the right move is: stop doing RAG This is the part more people should talk about. Anthropic makes a strong point: if your knowledge base is under roughly 200,000 tokens, you may be better off putting the whole corpus in context instead of building a retrieval pipeline. That runs against the default instinct a lot of teams have now, which is: chunk everything embed everything build RAG first debug forever For smaller corpora, that can be overengineering. Anthropic also says prompt caching can reduce latency by more than 2x and costs by up to 90%. So the tradeoff looks more like this: Approach Best fit Common failure mode Long-context prompting Smaller corpora, roughly under 200k tokens Relevant fact is buried or badly ordered Retrieval pipeline Larger or changing corpora Wrong chunk retrieved or exact term missed I like this framing because it forces a better question: Do you actually need retrieval, or are you building retrieval because that’s what everyone does? How I’d debug this before blaming GPT-5 or Claude Here’s the checklist I wish more teams used. 1. Log the exact context the agent saw Not what was in your database. Not what you intended to send. The exact: retrieved chunks prior messages tool outputs system prompt injected memory If you can’t inspect the final prompt state, you’re debugging blind. 2. Separate session memory from durable memory from retrieval Ask: Did the agent forget something from this run? Did it fail to load something from prior runs? Did retrieval miss the source doc entirely? Those are 3 different incidents. Treat them that way. 3. Add keyword search for exact-match terms If the bug involves IDs, SKUs, policy names, or literal strings, semantic retrieval alone is a bad bet. Use hybrid retrieval. 4. Rerank before switching models Bad top-k results poison everything downstream. Reranking is often cheaper and more effective than migrating from one frontier model to another because you’re fixing the input, not arguing about benchmark deltas. 5. Test prompt position, not just prompt content Move the critical fact. Seriously. If it’s buried in the middle of a long prompt, put it near the end or surface it in a structured summary. 6. Check whether your corpus is small enough to skip RAG If the full knowledge base fits comfortably in context, try the simpler architecture first. Fewer moving parts means fewer ways to fail. Quick local experiment: inspect exact-match retrieval behavior If you want a simple sanity check, compare semantic-only retrieval against a hybrid strategy for exact identifiers. A toy example in Python: queries = [ "What is the refund window for SKU-8472?", "Find policy for order ID ORD-19384", "What does workflow billing_reversal_v2 do?", ] exact_terms = [ "SKU-8472", "ORD-19384", "billing_reversal_v2", ] for q, term in zip(queries, exact_terms): print(f"query={q}") print(f"must not lose exact term: {term}") print("---") Enter fullscreen mode Exit fullscreen mode That looks trivial, but it’s the whole point. If your retrieval layer can’t preserve exact identifiers reliably, the model is being asked to reason from incomplete evidence. The table that changed how I think about agent reliability Approach Retrieval method Reported failure reduction Needs reranking/BM25 Standard RAG Basic chunking plus semantic retrieval No specific reduction cited here Usually yes Anthropic Contextual Retrieval Contextualized chunks plus semantic retrieval and Contextual BM25 49% fewer failed retrievals Yes, benefits strongly Contextual Retrieval plus reranking Contextualized retrieval with reranked results 67% fewer failed retrievals Yes That table is the argument. A lot of agent unreliability is not model IQ. It’s underbuilt retrieval. Why this matters more when you run agents at scale This gets more painful when agents run continuously in automations. If you have workflows in: n8n Make Zapier OpenAI-compatible custom stacks internal cron-driven agents ...then retrieval misses turn into repeated production failures. And if you’re paying per token, debugging gets even more annoying because every retry, replay, and prompt experiment has a visible cost attached to it. That’s one reason I think predictable API infrastructure matters for agent teams. If you’re iterating on retrieval, memory, reranking, and long-running automations, flat-cost OpenAI-compatible compute is a lot easier to work with than watching token spend while trying to fix reliability. That’s the appeal of Standard Compute: same OpenAI-compatible API shape, but built for teams running agents and automations all day without per-token anxiety. When you’re testing retrieval fixes, prompt changes, and multi-step workflows repeatedly, predictable cost matters almost as much as model quality. My current rule When an agent fails an "obvious" task, I no longer start with: maybe GPT-5 is worse at this maybe Claude should replace it maybe we need a bigger model I start with: what did retrieval return? where did the fact appear in context? did exact-match search exist? was reranking applied? did the agent actually see the right thing in usable form? That question is less fun than debating models. It’s also the one that usually fixes the bug. If your agent keeps failing in dumb ways, there’s a good chance the model isn’t the first thing you should blame.