最新资讯
www.infoq.com Yelp 为 AI Assistant 融合原生与服务器驱动 UI computerworld.com Box 发布 Architect:用自然语言构建业务流程工作流 the-decoder.com METR 称 Anthropic 或已在 7 月越过自身 AI 加速预警阈值 towardsdatascience.com 用 Ollama 和 Python 构建你的第一个多工具 AI Agent the-decoder.com Thinking Machines 的 Inkling 2 Small 称霸西方开放权重模型 dev.to 为 CI/CD 构建 AI 质量门禁 dev.to Kakao 发布面向 AI Agent 的决策模型 Kanana-2-v-decide computerworld.com Microsoft 对 Copilot Chat 企业用户扩大未公开的使用限制 pub.towardsai.net Google 发布 Gemini 4 Carbon:主打网络安全能力的前沿模型 gradientflow.com METR 研究:AI 正在加速构建 AI 的团队本身 www.infoq.com GitLab 19.5 将 Agentic 开发纳入 DevOps 平台的治理体系 aiacceleratorinstitute.com 研究:AI Agent 的复杂流程编排可能反而是问题所在
多元动态
登录 注册
智能体 dev.to · @chenwengui23

同一知识库,语音Agent与聊天Agent给出不同答案

原文 我的收藏

作者从俄亥俄州一家牙科集团的语音电话Agent和网页聊天Agent各取同一周60条对话,按主题对比发现:改约是否收费、是否接受Delta Dental、洁牙时长等问题上两渠道回答明显不一致。作者认为渠道是独立的生成上下文,共享知识库不等于共享答案,并据此讨论多渠道Agent的一致性风险。

AI推荐理由 作者用同一牙科集团同一周各60条对话的对比,展示共享知识库下语音与网页聊天Agent对改约费用、保险、时长给出不一致回答,属于Agent落地中的具体一致性问题;但样本来自单一案例、未说明抽取与核对方法,结论应视为个案观察而非普遍规律。

62

正文

A dental group in Ohio runs four clinics. Their AI stack is tidy: one vendor, one knowledge base, two channels — an AI phone agent that answers after-hours calls, and a web chat agent on the booking page. Same pricing doc feeds both. The owner assumed "one knowledge base" meant "one set of answers." It doesn't. Channels are not pipes. Each channel is a generation context, and generation re-invents the answer every time. I pulled 60 conversations from each channel for the same week and sorted them by topic. On rescheduling, the two channels gave materially different answers to the same question: Question Voice agent (phone) Chat agent (web) "Can I move my appointment to Thursday?" "Of course, I've moved it — no charge." "Rescheduling within 24 hours incurs a $25 fee. Shall I proceed?" "Do you take Delta Dental?" "Yes, we're in-network with Delta." "We accept most major PPO plans. Please confirm with your provider." "How long is the cleaning?" "About 45 minutes." "Typically 60 minutes." "Can I get a same-day appointment?" Booked one, 2:30 PM. "Same-day availability is limited — please call us." (It didn't call.) Four topics, four contradictions. The fee contradiction is the expensive one: a customer who chatted first, heard "$25," then called and heard "no charge," now knows the AI makes it up as it goes. That's not a support problem anymore — that's the moment your pricing becomes negotiable in the customer's mind. Two customers that week opened with "well, the phone robot said it's free." Same knowledge base. Different behavior. Here's why, and here's the audit that catches it. Why channels drift apart Three mechanisms, all boring, all predictable. 1. The system prompts were written months apart by different hands. The chat agent's prompt says "always mention the late-rescheduling fee when a change is requested." The voice agent's prompt says "be accommodating and helpful." Neither contradicts the pricing doc, but one foregrounds the fee and the other foregrounds flexibility. Given a choice between strictness and smoothness, the voice agent was tuned for smoothness — because someone, sometime, complained that the phone bot was rigid. 2. Voice has a latency tax. Every extra clause costs seconds. Someone optimized the voice flow for short answers, and "no charge" is shorter than the compliant "there's no charge because you're outside the 24-hour window; within 24 hours it would be $25." The qualifier is where the policy lives, and qualifiers are the first thing cut when answers get trimmed. So the voice agent answers correctly but incompletely, and the incompleteness reads as a different policy. 3. The channels escalate differently, so the feedback loops never meet. Chat transcripts flow into one inbox; call summaries flow into another. Nobody ever put the two side by side and asked whether the AI said the same thing in both. The med-spa problem from last month's audit — Spanish vs. English — is this same bug wearing a different hat. Every axis that splits your audience (channel, language, location, hour of day) quietly forks your policy. The fix: one answer table, channel-checked The structure that works is boring: a canonical answer table with a per-channel confirmation column. One row per money- or commitment-bearing question. The rule the AI must follow: For any question touching fees, insurance, duration, availability, or guarantees: 1. Look up the answer in ANSWER_TABLE by intent key. 2. Reproduce the approved answer for the CURRENT channel verbatim, including the qualifier sentence. Do not shorten. 3. If the current channel is voice and the approved answer exceeds 40 words, use the VOICE_VARIANT column — but the VOICE_VARIANT must be pre-approved by a human, never generated. 4. If no row exists for the intent: say "Let me confirm that with a human" and escalate. Do not compose a new policy answer. Enter fullscreen mode Exit fullscreen mode And the table itself: intent_key | canonical answer (web/voice identical core) | voice variant (pre-approved) ------------------+------------------------------------------------------+---------------------------------------------- resched_fee | Rescheduling is free up to 24 hours before your | "No charge — you're outside the 24-hour | appointment. Inside 24 hours, there's a $25 fee. | window. Inside 24 hours it's $25." | Which day works for you? | insurance_delta | We are in-network with Delta Dental PPO at all | "Yes — in-network at all four clinics." | four clinics. We do not accept Delta HMO. | cleaning_duration | A standard cleaning takes 45 to 60 minutes depending | "45 to 60 minutes, depending on X-rays." | on whether X-rays are due. | same_day | Same-day slots open at 7 AM daily and are first- | "Same-day slots open at 7 AM — I can | come. The chat cannot book them; call or ask the | check, but booking happens by phone." | phone agent. | Enter fullscreen mode Exit fullscreen mode The last row is the one everyone forgets: channels must know each other's limits. Telling a chat customer "please call us" without saying why or when converts one channel's constraint into the customer's confusion. The cross-channel audit: 30 minutes, monthly This is the part you can copy-paste. Take the same 8 questions to every channel you run, in the same week, and diff the answers. Not for style — for substance. Can I reschedule, and does it cost anything? Do you take [top insurer]? How long does [core service] take? Can I get [core service] today? What's your cancellation policy? What does [core service] cost, all-in? What happens if I'm late? Do you offer refunds or a guarantee? Score each channel pair against three defect classes: FEE/POLARITY DRIFT — one channel charges, waives, includes, or excludes something the other doesn't. Always a defect. NUMBER DRIFT — durations, prices, windows, counts differ. Always a defect. CAPABILITY DRIFT — one channel can book/transact/answer what the other claims it can't. Usually a defect, occasionally intentional — but if it's intentional, the other channel must say so. At the dental group, the monthly audit took 26 minutes and caught a fifth contradiction the week it appeared: the voice agent had started quoting a "$99 new-patient special" that marketing had loaded into the phone flow but that the front desk had never agreed to honor. The audit found it in month one; before the audit existed, the median detection time for a channel contradiction in my deployment sample was nine weeks — usually discovered by a customer repeating one channel's answer to a human. The weekly scorecard One table, five rows, Monday morning: Week of: ______ Channels active: ______ 8-question audit run on all channels: __ / __ cells FEE/POLARITY drift findings: ____ NUMBER drift findings: ____ CAPABILITY drift findings: ____ Voice answers over 40 words (latency risk): ____ Escalations with cross-channel mentions: ____ ("the phone robot said...") → resolved: ____ Enter fullscreen mode Exit fullscreen mode The metric to watch is customer-cited contradictions — how many tickets or calls open with "the other channel told me X." It should be zero, and it's the only metric that captures the reputational cost, which never shows up in the AI vendor's dashboard. Nine weeks after the dental group installed the answer table: the rescheduling contradiction was gone (the fee qualifier became a pre-approved voice variant, 22 words), insurance answers matched across channels, and the phantom $99 special was either loaded everywhere or nowhere — their choice, made consciously, by a human. The two customers who'd negotiated off "the phone robot said it's free" had cost $150. The answer table cost one afternoon. The 12-line card [ ] One canonical answer table; channel variants pre-approved by a human [ ] Fee, insurance, duration, availability answers looked up, not composed [ ] Voice variants capped at 40 words, qualifiers never cut for latency [ ] Unknown intent → escalate with "let me confirm," never invent policy [ ] Channels know each other's limits and say why when redirecting [ ] 8-question cross-channel audit run monthly, same questions every time [ ] FEE/POLARITY, NUMBER, CAPABILITY drift each treated as defects [ ] New pricing or policy change → answer table updated same day, all channels [ ] Chat and call transcripts reviewed in one place, side by side [ ] Customer-cited contradictions tracked weekly, target zero [ ] New channel (WhatsApp, DMs, kiosk) onboarded with the same table [ ] One owner; audit plus review under 45 minutes/month Enter fullscreen mode Exit fullscreen mode If you only do one thing, run the 8-question audit once — chat agent, voice agent, whatever else you run — and put the answers side by side. Twenty minutes. In most deployments I've audited, at least two of the eight answers disagree across channels, and the owners had no idea until the table was on the screen. The 8-question cross-channel audit script, the answer table template with voice-variant columns, and the monthly scorecard are in the kit I run on deployments: Agent Audit Pack

来源报道 · 1 篇 · 以下链接均前往原始媒体

← 返回精选动态 原文

更多内容