让 LLM 编写告警规则，但绝不让它按下开关

This is the platform behind taabi Nexus, which Taabi Mobility launched this week. The launch video has three real calls it made into trucks. This is the engineering side. Last Wednesday a truck driver heard his dashcam intercom tell him, in Hindi, that his seat belt was off. He said yes, he could hear. It asked if he was wearing it now. He pulled the belt across while answering. Nobody had dialled that call. A fleet manager had typed a sentence into a text box a few days earlier, and that sentence did. I've spent the last few weeks building the thing in between. Dashcams raise alerts: drowsiness, phone use, seat belt, hard braking, about forty kinds. A manager writes what should happen, in English. The platform messages, waits, re-checks, calls the truck, and writes down the follow-ups. Here is what I'd tell someone starting the same build. The model writes a document. Code runs it. My first instinct was to let the model read the sentence and act. That's a chatbot with side effects, and nobody is going to let one ring their drivers. So the LLM's only job became translating the sentence into a rule document with six keys, and a plain engine executes that: { "trigger": { "event_types": ["fatigueWarn"] }, "where": { "field": "speed_kmph", "op": "gt", "value": 60 }, "aggregate": { "window": { "type": "sliding", "size": "PT10M" }, "group_by": ["vehicle_no"], "function": { "name": "count" }, "having": { "op": "gte", "value": 2 } }, "actions": [ { "action": { "type": "notify", "channel": "telegram", "to": { "role": "manager" } } }, { "action": { "type": "call", "to": { "selector": "driver" } }, "wait": "PT10M", "recheck": { "event_types": ["fatigueWarn"], "same": ["vehicle_no"] } } ], "throttle": { "cooldown": "PT1H", "dedup_key": "{vehicle_no}" } } That is "drowsy above 60 km/h twice in ten minutes, tell the manager; still drowsy ten minutes later, call the truck". The schema is a Pydantic v2 model, exported as JSON Schema and as TypeScript types, so the engine, the simulator, the UI's plain-words summary and the evals all point at the same thing. Freezing it early was the best decision in the project, and the one I almost skipped because it felt like premature ceremony. Where the flow stops The authoring agent is a LangGraph graph with two interrupts. If the sentence is missing a threshold or a channel, the graph stops and asks. It doesn't guess. Once a draft validates (schema plus about thirty semantic checks, written in code, with the errors fed back to the model for up to three repairs), it is replayed over the tenant's last seven days and comes back with a number: "would have fired 27 times, 58 held back by the throttle". That number is what turns "sounds right" into "yes". It has also caught a rule that would have fired four thousand times. Then the second interrupt: a person approves, and approval only saves a draft. Activating is a separate button in a separate service behind a role check. The agent has no tool that can press it. The same shape carries into the calls: a call step is placed by the notifier, never by a model, uncertain calls wait in a queue for a manager, and every call ends with follow-ups a person closes. I stopped calling this "human in the loop" in meetings and started saying "this is where the state machine stops", which is what it actually is. Two bugs I'd rather not have found in production "At most one call per vehicle per hour" came out of the composer as a cap per rule, which in my schema means one call per hour for the entire fleet. The schema was fine. The problem was that the summary the manager approves said "at most one call per hour" and let both readings through. Now it says exactly what the schema means, because the summary is the thing people actually read. The other one was latency. A yes-or-no decision inside a live phone call went through the full agent subprocess and took 18 to 30 seconds, which is most of the call. One direct Messages API call with a tight schema took 1.2 seconds. Heavy machinery where you need tools and memory; not inside a call. What real trucks taught me Tests and a simulated intercom get you to the door. On the other side: engine noise under every word the driver says, Hindi speech recognition that is fine on the agent's own lines and shaky on the driver's, and no way for the agent to see that the belt actually went on. So it asks yes-or-no questions, treats an unclear answer as "not confirmed", and re-checks the alert stream after the call instead of trusting the transcript. Budget more time for the microphone than for the model. I did not, and I would now. The boring parts that let me sleep Every table carries tenant_id and Postgres row-level security is forced, so a forgotten WHERE returns nothing rather than everything. The tenant comes from the JWT, never from a request body or the model, and the MCP server the agent reads through takes a per-tenant token where the token is the tenant; there is no argument for the model to fill with someone else's id. Every alert gets a trace_id at ingest that rides in a Kafka header and is bound into every log line downstream; with OpenTelemetry on, it's the trace id too. Someone asks what happened to an alert, I paste one id into Grafana and see it accepted, persisted, matched and delivered across four services. Under load, 96,000 alerts went through with ingest p99 at 94 ms and nothing in the dead-letter queue, and when I killed the persist consumer on purpose an 86,704-message backlog drained in under five minutes with nothing lost. I wrote those numbers into the service contracts so they would be argued with, not remembered. Stack Python 3.12 and FastAPI in a uv workspace, Pydantic v2 for the contracts, Kafka through aiokafka (a compacted topic for rule updates, so an engine restart rebuilds its rules from Kafka alone), Postgres 16 with RLS and Alembic, Redis for dedup and cooldowns. LangGraph for the agents, Claude via the Agent SDK, a custom MCP server of curated read-only tools, Langfuse for traces, prompts and the eval datasets, with pytest golden suites failing the build. OpenTelemetry to Tempo, Prometheus rules with promtool tests, Grafana provisioned from code. React 18, Vite and TanStack on the front, Playwright end to end. Docker Compose for now; the rules compile to an IR so the engine can move to Flink without the rules changing. If you've shipped natural-language rules or agents with a person in the path, I'd genuinely like to know where you put the switch. Mine is two buttons and a queue. Much more to come next.