AI 应用中的 Mocking 与服务虚拟化对比

Mocking vs Service Virtualization for AI Applications Sancharini Panda 6 min read · Just now -- Listen Share Press enter or click to view image in full size Service virtualization and mocking both rest on an assumption so fundamental that it rarely gets stated: the dependency being tested returns the same thing for the same input. Payment API receives a charge request, returns a charge object. Database receives a query, returns rows. Cache receives a key lookup, returns the stored value. The assumption holds so reliably for these cases that it became invisible- baked into how both service virtualization and mocking tools were designed, how their documentation explains them, and how developers learn to use them. GPT-4o does not cooperate with this assumption. Neither does Claude. Neither does Gemini. Send the same prompt twice, and you receive two different responses- different phrasing, different structure, occasionally different conclusions. The model is stochastic by design. Randomness is not a bug to eliminate before you can test properly. It is the mechanism the model uses to generate language. This creates a gap in the standard service virtualization vs mocking comparison that neither approach was designed to address. The question for AI applications is not just which of these two approaches to use. It is what both approaches need to do differently when the dependency they replace does not have a single answer for any given input. What the Standard Approaches Miss In the traditional framing, service virtualization and mocking sit at different levels of the testing stack. Mocking replaces a dependency at the code level- a Python patch, a Jest spy, a Go interface substitute. Service virtualization operates at the network level, intercepting calls and returning configured responses regardless of what makes the call. Both approaches share an architecture for defining what the substitute returns: someone specifies the response, and the substitute returns that specification. The specification is the thing. If the specification is accurate, the test is meaningful. If the specification has drifted from current real behavior, the test is misleading. For AI applications, the specification problem runs deeper than drift. The real LLM does not have a single current behavior to drift from. It has a distribution of behaviors- a range of phrasings, structures, and content variations that are all valid outputs for the same input. Specifying one response from that distribution and treating it as the standard against which your code will be tested leaves the rest of the distribution untested. Production draws from the full distribution. Your tests drew from a sample of one. What Changes When the Dependency Is an AI Model The assertion model breaks for semantic content. Traditional mocking asserts on exact values. The response field equals this string. The status code is 200. For structural response properties- is the choices array present, does content contain a non-empty string, is finish_reason a recognized value -- exact assertions work fine on AI API responses. These structural properties are stable across calls. The semantic content is not. Writing a test that asserts response.summary == "The company reported strong Q3 earnings" fails every time the model phrases the summary differently, even when the summary is entirely correct and useful. What actually works is asserting on properties that any correct response should satisfy: shorter than the source, mentions Q3, does not introduce claims absent from the source. Those properties hold across the full range of valid LLM outputs rather than requiring one specific phrasing. Model updates arrive without any structural signal. A REST API that changes its response schema produces observable drift- a field renamed, a type changed, a nested object flattened. A tool comparing the mock against the current API response catches this. AI model updates produce behavioral drift that comparison tools cannot easily surface: the model reasons differently about edge cases, handles ambiguous inputs with different default assumptions, produces outputs with different stylistic characteristics. The endpoint path stays the same. The request and response schemas stay the same. The model behind them has changed in ways that matter for applications depending on specific behavioral patterns. There is no changelog entry for “model now handles ambiguous pronoun references differently in summarization tasks.” Get Sancharini Panda’s stories in your inbox Join Medium for free to get updates from this writer. Subscribe Subscribe Remember me for faster sign in Response field variability compounds. Standard non-deterministic field handling for mocks works by identifying fields that always vary- IDs, timestamps, session tokens, and excluding them from assertions. AI API responses introduce a category that does not fit neatly into either “always the same” or “always different”: semantic content that is meaningfully variable. The summary text changes on every call. Whether it changes in ways that matter for downstream processing depends on the downstream logic, not on whether the field appears to be deterministic. What Actually Works for AI Application Testing Semantic assertion layers. Tests for AI-dependent code need to validate properties of responses rather than specific responses. The assertion layer evaluates whether the response satisfies the intent- correct structure, appropriate length, contains required concepts, avoids prohibited content rather than whether it matches a stored string. These assertions pass for any correct response the model produces, making the test suite stable across natural output variation. class TestSummarizationService: def test_summary_semantic_properties(self, ai_client): source = "Long article about Q3 financial results..." response = ai_client.summarize(source) # Semantic assertions -- stable across LLM response variation assert len(response.summary) 0 Traffic-based mock generation for AI API interactions. For the integration layer between the application and AI service APIs, observation-based approaches produce mock configurations that reflect what the AI service actually returned for specific inputs during a recording session. This does not resolve semantic variability- the recorded response is one sample from the response distribution, but it addresses the behavioral currency problem that model updates create. When the AI service changes its response structure, token usage reporting, finish reason vocabulary, or error format after a model update, re-running the recording session captures the updated behavior without requiring manual identification of what changed. Keploy’s eBPF-based traffic capture applies here without requiring code changes or SDK integration in the application. During a recording session against the staging environment, it captures what specific prompts return from the current model version, including response metadata fields- token counts, finish reasons, model identifiers- that shift across model versions and are easy to miss when manually updating mock configurations. Fields that vary across multiple observations of the same prompt are automatically detected and excluded from assertions. For teams whose applications call multiple AI service providers: generation from one, classification from another, embedding from a third- recording sessions after each provider’s model updates keep all three sets of mock configurations current without per-provider manual maintenance. Model version metadata in virtual service configurations. The version a virtual service configuration was captured against matters for AI APIs in ways it does not for traditional APIs. A configuration captured against one model snapshot may not accurately represent behavior against a subsequent snapshot, even when the API version has not changed. Storing the model identifier alongside the API version in virtual service metadata and surfacing this in CI when the production model version advances creates a re-validation trigger before behavioral drift reaches production. The Comparison That Actually Applies For AI applications, the productive framing is not service virtualization versus mocking in their traditional forms. It is specification-based approaches versus observation-based approaches, applied at whichever level- network or code fits the testing context. Specification-based approaches require someone to define what the AI dependency returns. For static structural properties this is tractable. For semantic response content it is not, because no specification covers the full distribution of valid LLM outputs. Observation-based approaches record what the AI dependency actually returned during real interactions and derive test fixtures from those observations. The observations represent a sample from the response distribution rather than the full distribution, but they are significantly closer to current reality than authored specifications, and the process for keeping them current after model updates is mechanical rather than requiring semantic understanding of what changed. Where the engineering effort is most productively spent in AI application testing follows from this distinction. Not in writing more complete mock specifications for LLM responses. In building semantic assertion layers that evaluate response properties rather than response content, and in establishing observation-based fixture generation that keeps structural configurations current as AI model versions change beneath unchanged API surfaces. Service virtualization vs mocking for AI applications is not a question of which traditional approach wins. It is a question of which adaptations of both approaches address what AI dependencies actually do that traditional dependencies do not.