AI 数据 Agent 护栏：如何防止对话式分析编造业务指标

Ai Data Ai Conversational Agents AI Data Agent Guardrails: How to Keep Conversational Analytics From Inventing Business Metrics Anna Jey 11 min read · Just now -- Listen Share Press enter or click to view image in full size AI Data Agent Guardrails Conversational analytics is becoming a first stop for business questions. The hard part is making sure its answers mean the same thing as the numbers Finance, Product, and Sales already trust. Someone asks an AI data agent, “Why did revenue fall last month?” The answer sounds sharp. It names a segment, suggests a cause, and ends with a tidy recommendation. Then Finance checks the dashboard and gets a different number. This is not usually a model-intelligence problem. It is a meaning problem. The agent may have counted bookings where Finance counts recognized revenue, used the calendar month instead of the fiscal month, or followed an old join that quietly double-counts a source. Fluent prose can hide all of that. The timing matters. OpenAI recently introduced a Data agent for ChatGPT Work that can investigate company data and create shareable dashboards. Its earlier account of the in-house data agent makes the right operating principle clear: the agent should inherit existing permissions, surface assumptions, and link people back to the underlying results. That is the bar worth designing for, regardless of whether you use ChatGPT Work, Databricks Genie, Power BI, Looker, a warehouse assistant, or a home-built text-to-SQL stack. The practical answer is an AI data agent guardrail: a set of machine-readable rules that turn a vague request into a governed metric query and return proof with the answer. This guide shows how to build that layer without waiting for a perfect semantic program. The trust gap is not “AI versus dashboards” A dashboard answers a small number of predesigned questions. An agent can explore. That is why it feels useful. It can move from “revenue fell” to “which region changed, which channel changed, and what changed inside that channel?” in one conversation. But exploration creates three new failure paths: Definition drift: “active customer,” “pipeline,” or “revenue” has several valid-looking definitions across the company. scope drift: the tool changes the time zone, cohort rule, entity grain, or filter while trying to be helpful. evidence drift: a summary is copied into a planning doc even though nobody can see the query, data freshness, or assumptions behind it. Recent practitioner discussions make this concrete. Teams report that a semantic layer helps, but they also worry about upstream schema changes that leave a technically correct metric producing the wrong business answer. Others describe dashboards shifting into a health-check role: a person sees a red signal, asks an agent to investigate, then verifies the answer against the evidence layer. That is a healthier frame than declaring dashboards dead. Your aim is not to make the agent sound certain. It is to make uncertainty visible and routine verification cheap. Start with metric contracts, not a giant catalog A metric contract is a compact, versioned agreement about one business number. It gives an agent more useful context than a warehouse schema and gives reviewers a stable thing to audit. It can live in dbt metadata, a semantic-layer tool, a repository, or a small service. The storage choice matters less than the contract being canonical and testable. For a first rollout, select five to ten high-value metrics that people ask for repeatedly. Good candidates are net revenue, active customers, conversion rate, retained accounts, support backlog, and qualified pipeline. Avoid “every table” as the first milestone. Each contract should answer six questions: What business concept does this metric represent? Which approved expression or view computes it? What grain, time zone, calendar, and allowed dimensions apply? Who owns the definition and when was it last reviewed? Which roles and data scopes can query it? What proof must the agent return with an answer? Here is a deliberately small example. The point is not YAML. The point is that the agent receives a constrained interface instead of raw tables plus a hopeful prompt. metric: net_revenue version: 3 owner: finance-analytics source: mart.finance_monthly expression: sum(recognized_revenue_usd) - sum(refunds_usd) grain: month, legal_entity, region calendar: fiscal_445 timezone: America/New_York allowed_filters: - region - legal_entity - product_family required_evidence: - metric_version - generated_sql - source_freshness - applied_filters - result_link confidence_rules: stale_after_hours: 30 escalation_on_missing_dimension: true Notice what is missing: a prose instruction telling the model to “be accurate.” A model cannot enforce that request by itself. A contract can limit what is queryable, reject an unsupported filter, and make the returned answer carry its own receipt. Keep business language separate from data mechanics People do not ask for recognized_revenue_usd. They ask for revenue, ARR, new business, or expansion. Add approved aliases to the contract, plus a short explanation of terms that are commonly confused. If “revenue” is ambiguous in your company, do not guess. Make the agent ask: “Do you mean recognized revenue or contracted bookings?” That clarification can feel slower in a demo. In a planning meeting, it is much faster than discovering a decision was based on the wrong measure. Make the semantic layer an execution boundary Many early AI analytics projects connect a model directly to a database with read-only credentials. Read-only is necessary, but it is not enough. The database can still expose confusing joins, sensitive columns, unapproved calculations, and a huge surface for expensive scans. Instead, expose a narrow set of tools that operate on governed objects: search_metrics(term) returns approved metrics and definitions. query_metric(metric, dimensions, filters, period) accepts only contract-approved arguments. compare_periods(metric, current, baseline, dimensions) applies a standard comparison method. get_evidence(run_id) returns query text, freshness, row counts, and source links. This design works through an MCP server, a conventional API, a warehouse semantic layer, or a tool wrapper in your agent framework. It also creates a clean place to enforce row-level security. Pass through the requesting user’s identity when possible; do not give a general-purpose agent a broad service account and hope it remembers who should see what. Let the model decide how to explain a governed result. Do not let it decide what your company’s revenue means. Press enter or click to view image in full size Return an evidence receipt with every consequential answer The best AI analytics answer is not a paragraph. It is a paragraph plus enough context to inspect and reproduce the claim. Call that an evidence receipt. For low-stakes exploratory questions, the receipt can stay collapsed behind a “show work” control. For executive reporting, finance, pricing, customer commitments, or automated alerts, show it by default. A useful receipt includes: the metric name and version; the precise period, time zone, and comparison baseline; filters, segments, and exclusions; source freshness and known data-quality warnings; a link to the query result or governed dashboard; the agent’s assumptions and unanswered questions. That last point is important. An agent should be allowed to say, “I can see a 14% decline in net revenue for the requested period, but the campaign-attribution feed is six hours late, so I cannot yet rank channels reliably.” A cautious incomplete answer is more useful than a complete-looking one that hides a broken dependency. Test analytics agents with questions people will actually ask Generic text-to-SQL benchmarks are not enough. They rarely include your fiscal calendar, duplicate customer identities, restricted account fields, or loaded words such as “customer,” “revenue,” and “active.” Build an evaluation bank from real analytics tickets, recurring executive questions, and past incidents. Each evaluation case should include the user question, authorized role, expected metric contract, allowed interpretations, a known-good result, required receipt fields, and a failure label. Start with about 50 cases. Add a case whenever someone catches an answer that is confusing, incorrect, overly broad, or missing proof. Get Anna Jey’s stories in your inbox Join Medium for free to get updates from this writer. Subscribe Subscribe Remember me for faster sign in Test four types of behavior: Grounding: Did the agent choose an approved metric rather than inventing a calculation? Scope: Did it apply the correct period, grain, filter, and security context? Truthfulness: Did the final prose match the executed result without adding unsupported causes? Recoverability: Can a reviewer recreate the answer from the receipt? Use deterministic assertions for numbers and permissions. Use human or model-assisted review for explanation quality, but never let a language-model judge be the only authority on whether the data query was correct. Measure trust, not just chat volume High usage can mean people love the agent. It can also mean they are repeatedly trying to get a trustworthy answer. Track a smaller set of operational measures: evidence-receipt open rate on consequential answers; percentage of questions resolved without analyst rework; definition-clarification rate for ambiguous terms; freshness and quality warnings surfaced before an answer is used; disagreements between agent results and certified dashboard results; time from a metric-definition change to an updated evaluation pass. Do not punish clarification. A rising clarification rate early in the rollout may mean the agent is finally exposing ambiguity that dashboards had quietly buried. Press enter or click to view image in full size Run a safe first pilot Choose one team with strong data ownership and a clear recurring decision loop. Product growth, customer success, or revenue operations often work well. Give the agent only curated metrics, read-only tools, and one approved output channel. A practical four-stage sequence looks like this: Shadow mode: replay real historical questions. Compare agent output with analyst-approved answers, but do not expose results broadly. Small cohort: let a few data-literate users ask questions. Require receipts and collect examples of ambiguity. Decision support: add the agent to a recurring review, with a human owner approving any published narrative. Monitored self-service: widen access only after core metrics, evaluation cases, and escalation paths are stable. Keep a simple escalation lane. When the agent cannot map a question to a contract, encounters stale data, or produces a result that conflicts with a certified view, it should stop and hand off the question with the relevant context. “I do not know” is a feature when it comes with a useful next step. Treat source drift as a first-class agent failure A metric contract can be perfectly written and still lead to a bad answer. Imagine that the definition of active users correctly counts unique user_id values. A new guest checkout flow then begins sending null IDs, or a partner begins replaying events. The query remains valid. The number becomes misleading. That is why the contract needs a few quality signals beside the definition: expected freshness, null-rate limits for critical fields, volume-change thresholds, and a link to the latest source tests. The agent does not need to diagnose every anomaly itself. It needs to know when it is unsafe to present a clean answer. For example, a query_metric response might carry a machine-readable warning: { "metric": "active_customers", "value": 48231, "quality": "warning", "warnings": [ "Guest-session user_id null rate is above its agreed threshold", "The event source is 19 hours behind its normal refresh" ], "recommended_action": "Do not compare this period with the prior week until the source check passes." } Make the agent repeat that warning in plain language, rather than burying it in a trace. This separates a useful exploratory answer from a decision-ready answer. It also gives data teams a focused backlog: improve the source contract, not another free-form prompt. Version definitions like code Metrics change for good reasons. Finance may change a revenue-recognition rule. Product may split one event into two. Sales may redefine a qualified lead. A mature agent should not silently use the newest definition for a historical answer if the question is about a prior decision. Version the contract and record the effective date. When a definition changes, run the evaluation bank against both versions, publish a short change note, and decide how historical comparisons should behave. In some cases, restating history is correct. In others, preserving the old definition is necessary to explain what a team knew at the time. The key is to make the choice explicit. A concrete answer flow Suppose a customer-success lead asks, “Why did enterprise retention dip this quarter?” A safe agent flow looks like this: It resolves “retention” to the approved gross-revenue-retention and logo-retention metrics, then asks which one the lead means if both remain plausible. It checks the lead’s access scope and the quarter definition for the relevant business unit. It calls a governed comparison tool rather than assembling arbitrary joins from warehouse metadata. It retrieves the top changing segments and supporting metrics through approved follow-up tools. It writes a summary that distinguishes observed changes from possible causes. It attaches the metric version, time window, filters, freshness status, and result links. Now compare that with a raw text-to-SQL path. The model sees tables named accounts, subscriptions, and invoices. It guesses a retention formula, misses an exclusion for internal accounts, and creates a convincing explanation. The second path may be faster to prototype. The first is faster to trust. This is also where model choice becomes less central. A stronger model can improve planning and explanations. It cannot infer a company’s authoritative retention definition from column names. Better context is not merely extra tokens; it is a constrained agreement about meaning and permission. Common shortcuts that create expensive problems Putting the data dictionary only in the system prompt Prompts are helpful explanations, not a source of truth. They drift, get truncated, and are hard to audit. Point the agent to a versioned metric contract or tool instead. Giving every user the same warehouse credential This defeats the access model that made your warehouse safe. Use pass-through identity or a broker that applies the user’s entitlements at query time. Hiding SQL because nontechnical users do not want it Do not force everyone to read SQL. Do make the underlying result and definitions one click away for the people accountable for decisions. Trying to automate root cause on day one Start with descriptive and comparative questions. Causal claims usually need experiment context, external events, and domain judgment that a metric contract alone cannot provide. The durable advantage is shared meaning AI data agents will make it easier for more people to ask useful questions. They will also reveal where your organization has several answers to the same question. That can be uncomfortable, but it is productive. The teams that earn trust will not be the ones with the most polished analytics chatbot. They will be the ones that make definitions explicit, keep access scoped, test real questions, and attach evidence to every important claim. Build those habits now, and a new model or data-agent platform becomes a swap in the interface layer rather than a new governance emergency. Frequently asked questions What are AI data agent guardrails? They are technical and operational controls that keep an AI analytics agent within approved metric definitions, permissions, data scopes, freshness rules, and evidence requirements. Is a semantic layer required for conversational analytics? You need a canonical governed interface, but it does not have to begin as a large commercial semantic-layer deployment. A small versioned set of metric contracts and restricted tools is a useful first step. How do AI data agents avoid hallucinating metrics? Do not let them generate calculations against arbitrary raw tables. Route requests to approved metrics, validate supported filters, and return the query result and assumptions with the answer. Should an AI analytics agent generate SQL? It can, but raw free-form SQL should be constrained and reviewed in high-risk environments. For common business questions, metric-level tools or compiled queries are safer and easier to evaluate. What should appear in an AI analytics evidence receipt? Include the metric version, source freshness, time period, filters, query or result link, assumptions, and warnings that could change how a reader interprets the answer. How many metrics should an AI data agent pilot include? Start with five to ten important, well-owned metrics. Expand after the team can explain disagreements, maintain evaluation cases, and update contracts when definitions or source data change.