用 Cypress 测试流式 AI 界面：不必断言每个 token

A streaming AI test becomes brittle the moment it expects this exact sequence: Hel → Hello → Hello, I → Hello, I can help Enter fullscreen mode Exit fullscreen mode Chunk boundaries are transport details. They can change with buffering, provider behavior, network timing, or a client-library upgrade while the visible result remains correct. The UI still has deterministic responsibilities. It must show progress, distinguish a tool call from ordinary text, ignore stale events, support cancellation, and end in a coherent state. Test those contracts instead of every token. Model the stream as events Do not let the component treat an incoming byte stream as an undocumented collection of callbacks. Normalize it first: type AgentEvent = | { type: "started"; runId: string } | { type: "text_delta"; runId: string; text: string } | { type: "tool_started"; runId: string; name: string } | { type: "tool_completed"; runId: string; name: string } | { type: "completed"; runId: string } | { type: "failed"; runId: string; code: string }; Enter fullscreen mode Exit fullscreen mode The renderer can derive a small set of user-visible states: idle → connecting → streaming → using_tool → streaming → complete └──────────────→ failed Enter fullscreen mode Exit fullscreen mode That state machine is the stable testing surface. “Two chunks arrived instead of three” usually is not. Inject a controllable stream in tests For component or end-to-end tests, put the transport behind an interface. In test builds, expose a synthetic event source: declare global { interface Window { agentTestStream?: { emit(event: AgentEvent): void }; } } Enter fullscreen mode Exit fullscreen mode Now Cypress controls causality without calling a model: it("shows a tool phase and completes the answer", () => { cy.visit("/assistant?stream=test"); cy.get("[data-testid=ask]").click(); cy.window().then(({ agentTestStream }) => { agentTestStream!.emit({ type: "started", runId: "run-1" }); agentTestStream!.emit({ type: "text_delta", runId: "run-1", text: "Checking the policy. ", }); agentTestStream!.emit({ type: "tool_started", runId: "run-1", name: "search_policy", }); }); cy.get("[data-testid=status]").should("contain", "Searching policy"); cy.window().then(({ agentTestStream }) => { agentTestStream!.emit({ type: "tool_completed", runId: "run-1", name: "search_policy", }); agentTestStream!.emit({ type: "text_delta", runId: "run-1", text: "Returns are accepted within 30 days.", }); agentTestStream!.emit({ type: "completed", runId: "run-1" }); }); cy.get("[data-testid=answer]") .should("contain", "Checking the policy") .and("contain", "30 days"); cy.get("[data-testid=status]").should("contain", "Complete"); }); Enter fullscreen mode Exit fullscreen mode This verifies progressive rendering and the final semantic landmarks. It does not care how the provider would divide the sentence. Test the failures streaming makes easy to miss The highest-value cases are usually not the happy path. A stale run writes into a newer answer Start run-1, then start run-2. Emit a late delta from run-1 and assert that it is ignored. Every event should carry a run identifier, and the renderer should accept only the active one. Cancellation changes the terminal state After the user clicks Stop, emit another delta. The UI must remain cancelled and must not append the text. Also assert that the transport's abort function was called. A tool fails after text has appeared Partial prose should not leave the interface looking complete. Emit tool_started, then failed, and assert that the error is visible while the incomplete answer is clearly marked. Duplicate events arrive Reconnects can replay events. Give events stable IDs or sequence numbers and assert that a duplicate delta is not rendered twice. The stream ends without a terminal event Close the transport after several deltas but before completed or failed. The UI should move to an explicit interrupted state instead of presenting partial prose as a finished answer. This catches a class of bugs that an HTTP status assertion cannot see. Use network interception at the right layer cy.intercept() is useful for controlling the initial request, authentication errors, HTTP status, and response delay. It is less useful when the test needs precise control over a long series of application-level stream events. An injected adapter keeps the browser behavior real while making the event schedule deterministic. Keep one integration test against the actual streaming endpoint to verify framing and parsing. Keep most UI state tests provider-free and synthetic. That split makes failures easier to classify: protocol problem or rendering problem. At the protocol layer, include fixtures where one logical event is split across network chunks and several events arrive in one chunk. Your parser must reconstruct frames before the UI reducer sees them. The component test can then remain deliberately unaware of TCP, SSE, or fetch buffering. Assert responsibilities, not prose A robust streaming test usually checks: the visible state transition; ordering around tool activity; cancellation and stale-event handling; accessibility announcements; the final structured outcome or a few semantic landmarks; absence of duplicated content. It should rarely check every intermediate string. For accessibility, assert that status changes are announced once and that rapidly arriving deltas do not flood a live region. The visual answer may update continuously while assistive technology receives milestone events such as “searching,” “approval required,” and “complete.” Streaming interfaces are asynchronous state machines wearing a chat UI. Once tests target that state machine, they become both stricter about real bugs and less sensitive to irrelevant token timing. References Cypress network interception Cypress retry-ability