当免费额度不再便宜：面向 Agent 后端的负载退出测试

When the Free Tier Stops Being Cheap: A Load-Based Exit Test for Agent Backends A team moves its nightly agent job onto a free endpoint. Week one looks perfect. Week four, the queue drains ten minutes late and the on-call engineer has no data. The endpoint did not break. The workload changed shape around it. Free tiers rarely fail loudly; they fail as latency, retries, and quiet queue debt. This post is a stop rule, not a sales pitch. It gives you one probe, one decision table, and a set of exit criteria you can wire into CI. The failure mode is a debt curve, not an outage Free capacity absorbs small workloads well. It degrades along a curve as concurrency and retries stack up. Three effects dominate that curve: Retry amplification. One slow call becomes three requests after timeouts. Cold-path penalty. An idle endpoint pays extra latency on the first call. Queue debt. Slow drains shift work into the next window, then compound. None of these raise an alert by default. You only see them if you measure the tail, not the mean. What a free tier honestly covers MonkeyCode is an open-source coding agent project. The operator states that it provides free model access and a free server option, with a free token allowance listed in the terms in effect on 2026-09-15. Quotas and hardware change often, so read the current terms page before you plan around any number. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Free access answers a cost question. It does not answer a fitness question. The rest of this post measures fitness. Step 1: Probe four numbers You need four measurements before any migration decision: p50 and p95 latency under your real concurrency error and timeout rate after retries cold-start latency after an idle period total drain time for one realistic batch The script below is a template harness. It has not been executed against MonkeyCode for this article. Adapt the body shape to your provider, then run it in your own environment. #!/usr/bin/env python3 """free_tier_probe.py - measure tail latency and error rate on a JSON endpoint.""" import argparse, asyncio, json, os, time import httpx PROMPT = "Reply with exactly one word: pong" async def one_call(client, endpoint, headers, model, timeout_s): body = { "model": model, "messages": [{"role": "user", "content": PROMPT}], "max_tokens": 4, } t0 = time.perf_counter() try: resp = await client.post(endpoint, headers=headers, json=body, timeout=timeout_s) return {"ok": resp.status_code == 200, "status": resp.status_code, "ms": (time.perf_counter() - t0) * 1000} except Exception as exc: return {"ok": False, "status": type(exc).__name__, "ms": (time.perf_counter() - t0) * 1000} def percentile(values, q): if not values: return float("nan") ordered = sorted(values) idx = min(len(ordered) - 1, int(round(q * (len(ordered) - 1)))) return ordered[idx] async def run(endpoint, api_key, model, total, concurrency, timeout_s): headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"} limits = httpx.Limits(max_connections=concurrency) async with httpx.AsyncClient(limits=limits) as client: sem = asyncio.Semaphore(concurrency) async def guarded(): async with sem: return await one_call(client, endpoint, headers, model, timeout_s) started = time.perf_counter() results = await asyncio.gather(*(guarded() for _ in range(total))) wall_s = time.perf_counter() - started return results, wall_s def summarize(results, wall_s, p95_budget_ms, error_budget): lat = [r["ms"] for r in results] errors = [r for r in results if not r["ok"]] report = { "requests": len(results), "wall_seconds": round(wall_s, 2), "p50_ms": round(percentile(lat, 0.50), 1), "p95_ms": round(percentile(lat, 0.95), 1), "max_ms": round(max(lat), 1), "error_rate": round(len(errors) / len(results), 4), } report["verdict"] = "pass" if ( report["p95_ms"] /v1/chat/completions \ --model \ --requests 200 --concurrency 8 \ --p95-budget-ms 6000 --error-budget 0.02 echo $? # 0 = keep it here, 1 = start the exit plan Enter fullscreen mode Exit fullscreen mode The exit code is the point. A probe that only prints a table gets ignored. A probe that fails CI forces a decision. Measure cold start separately. Wait at least thirty minutes idle, then time a single call. Record it next to the warm p50. Step 2: Read the decision table Workload shape Free tier fit Reason Local dev chat, one user Good Low concurrency hides tail latency Idempotent lint or test agent Conditional Safe only with capped retries Nightly batch inside a wide window Conditional Drain time must fit the window Interactive path with a latency SLO No No capacity guarantee to cite 10x burst fan-out No Retries multiply during bursts Contractual uptime obligations No You need a vendor commitment Conditional means the probe decides. "No" means no probe will save it. Step 3: Write exit criteria before you need them Write these down while the system is calm. Numbers below are policy examples, not product facts. Set your own. p95 exceeds three times your interactive budget on two consecutive runs. Error rate stays above two percent after retries are counted. Drain time exceeds the available window by more than twenty percent. Two incidents in one month trace to capacity instead of code. Engineer hours spent diagnosing exceed one month of the paid tier. Criterion five is the honest one. Free compute stops being free when it consumes senior time. Red flags that say stop now Your retry policy has grown from one attempt to three. The same job now needs a second scheduling window. Nobody can state the current concurrency ceiling. Dashboards show mean latency only. The team routes around the endpoint during releases. Any two of these means the exit plan should already exist. Better alternatives when the answer is no Move the workload to a paid tier with a written rate limit. Self-host a small inference server for steady, private traffic. Add a queue with backpressure so bursts shed instead of retry. Split the job: fast path stays free, slow path runs in batch. Degrade the feature deliberately instead of timing out. Most teams need two of these, not all five. Who should not use this approach Teams with uptime clauses in customer contracts. Workloads under data residency or audit constraints. Projects with no metrics pipeline to feed the probe. Jobs that cannot cap concurrency or retries. For everyone else, the probe is cheap and the answer arrives in an hour. The numbers decide Free access is a legitimate place to start an agent backend. It is a poor place to hide an unmeasured one. Run the probe against your free endpoint this week. If it passes, keep the workload there and revisit next quarter. If it fails, you now have a dated report to justify the next step, and a soft place to try the free server option before you commit to anything larger.