Four bugs shipping a live x402 endpoint
Four bugs reached production in a pay-per-call scraping API that settles USDC on
Base. Every one of them passed a green test suite. In each case the thing meant
to catch the bug was more forgiving than production was.
The dependency nobody declared
Every payment through the Coinbase facilitator had been failing for weeks, and
the error said nothing useful: clients got facilitator_unreachable.
The cause was one line that did not exist. cdp_facilitator.py imported PyJWT
lazily, inside the function that signs the bearer token:
def build_bearer_jwt(...):
import jwt # never declared in requirements.txt
Enter fullscreen mode
Exit fullscreen mode
PyJWT was installed on every development machine as somebody else's transitive
dependency. It was not in the container. The import only ran when a real payment
arrived, so nothing failed at startup, no health check noticed, and the test
suite passed because the tests ran where the package happened to exist.
A lazy import is exactly the shape that survives local development and dies in
production. It runs on the one code path nobody exercises before shipping.
The fix is a test, not a discipline:
RUNTIME_IMPORTS = {"jwt": "pyjwt", "cryptography": "cryptography", "stripe": "stripe"}
def test_runtime_imports_are_declared(self):
for module, distribution in RUNTIME_IMPORTS.items():
assert distribution in declared_distributions()
Enter fullscreen mode
Exit fullscreen mode
Worth knowing the limit of that guard: it checks a hardcoded list rather than
scanning source for imports. It protects the dependencies someone remembered to
add to it, which is better than nothing and less than it sounds.
A wallet that can be paid and can never pay
The x402 exact scheme on Base settles with EIP-3009
transferWithAuthorization. The payer signs an authorization off-chain, the
facilitator submits it and pays the gas. That signature is ECDSA, produced by a
private key.
A smart-contract wallet has no private key. It cannot produce that signature, so
it can never be the payer ΓÇö even though it receives USDC perfectly well and
looks identical in a block explorer.
One call tells them apart:
curl -s -X POST https://mainnet.base.org \
-H 'content-type: application/json' \
-d '{"jsonrpc":"2.0","id":1,"method":"eth_getCode","params":["0xYOUR_ADDRESS","latest"]}'
Enter fullscreen mode
Exit fullscreen mode
0x means an EOA, which can sign. Anything longer is a contract, which cannot.
The receiving wallet here returned 48 bytes of code.
Two related things cost real time:
A deposit address is not proof of control. An earlier receiving address
had been pasted from an exchange deposit screen and treated as verified.
On-chain it had never received or sent anything ΓÇö zero transfers, ever. The
test deposit it was supposed to have received did not exist. The rule that
replaced it: send a dollar in, send it back out. If you cannot send it back,
you do not control it.
Contract wallets pay gas in tokens. The one here pays via a paymaster,
charging roughly 1.5 cents in USDC per outbound transfer. Fine for receiving
revenue, expensive to spend from.
The transaction that never committed
This one shipped to production and reported success every time.
transaction = self._client.transaction()
customer_doc = customer_ref.get(transaction=transaction)
...
transaction.update(customer_ref, {"credit_used_micro": new_value})
transaction.set(event_ref, {...})
return DebitResult("ok", remaining)
Enter fullscreen mode
Exit fullscreen mode
Transaction.update() and .set() in google-cloud-firestore append to an
internal write buffer. Those writes are sent by commit(), which is called by
the @firestore.transactional decorator or by using the transaction as a
context manager. This code does neither, so the buffer is discarded when the
object is garbage collected.
Nothing raises. The function returns ok, the request is served, and the
customer's balance never moves. Refunds wrote directly rather than through a
transaction, so a balance could only ever go up.
The reason 678 passing tests said nothing about it is the part worth copying.
The fake Firestore client applied transaction writes immediately:
class FakeTransaction:
def update(self, ref, data):
ref.update(data) # real client buffers; this does not
Enter fullscreen mode
Exit fullscreen mode
The fake modelled the API I assumed rather than the one that exists. A test
double more forgiving than production hides exactly the class of bug it exists
to catch. The suite was not weak on coverage; it was confidently wrong about
the dependency.
Two changes, in order of importance:
The fake now buffers until commit(), and a test asserts writes are
invisible before it. The regression test for the bug is a runner that never
commits, asserting the balance stays put while the call claims success.
Begin, retry and commit moved into a single injected runner, so no individual
method drives a transaction by hand and no future method can forget.
It cost nothing, because no customer had a key yet. It would have hit the first
one.
Completed does not mean paid
The card path had the same shape of error, caught before it shipped.
checkout.session.completed sounds like the fulfilment event. For card payments
it is. For delayed payment methods ΓÇö bank debits and similar ΓÇö the session
completes when the customer finishes the flow, which is before the money
arrives. Crediting on that event alone grants credit for funds that may never
settle.
So the handler gated on payment status:
if session.get("payment_status") not in (None, "paid"):
return None
Enter fullscreen mode
Exit fullscreen mode
Correct, and half a feature. Nothing listened for
checkout.session.async_payment_succeeded, the event that fires when the
delayed payment actually settles. A customer paying by bank debit would have
been charged and never credited ΓÇö the failure mode that generates a support
ticket and a chargeback rather than a quiet loss.
Both events fulfil now, both gated on payment status.
Two smaller things from the same reading of Stripe's own guidance:
Setting the module-level stripe.api_key on each call is deprecated, and in a
concurrent server it is shared mutable state. A StripeClient instance
carries the key instead.
Passing payment_method_types disables dynamic payment methods, which decide
what to show each buyer. Omitting it is the default for a reason.
The webhook is also the most attacker-interesting endpoint in the service: a
public URL that moves money into an account. It verifies the signature before
parsing the body, grants only what the processor reports as paid, and is
idempotent on the event id, because a webhook is retried on any non-2xx. A
verified event that does nothing still returns 200 ΓÇö otherwise it is redelivered
forever.
The pattern
None of these were hard bugs. Each one hid behind something that was supposed to
catch it and was more permissive than production:
Bug
What should have caught it
Why it didn't
PyJWT undeclared
The test suite
Ran where the package happened to exist
Contract wallet as payer
The address looking valid
Only eth_getCode distinguishes them
Transaction never committed
678 passing tests
The fake applied writes the real client buffers
Credit on completed
Reading the event name
The name describes the session, not the money
The cheap checks, in the order I wish I had done them:
Assert your declared dependencies, do not trust your machine. A lazy
import is the one that gets you.
Prove wallet control by moving money out, not in. Anyone can send to an
address nobody holds the key to.
Make test doubles pessimistic. If the real client buffers, the fake
buffers. A double that is kinder than production converts a green suite into
false confidence.
Read the payment provider's own guidance before going live, not after.
Two of these came straight out of it, and it took ten minutes.
The one that generalises furthest is the third. Coverage was never the problem ΓÇö
the money path had tests for concurrency, idempotency, expiry and refunds, and
they all passed. They were all asking a stub that had been written to agree with
me.
The endpoint these came from is a hosted Crawl4AI service that takes x402
payments: page in, markdown or LLM-extracted JSON out. There is a demo that
needs no signup, if you want something to point an agent at.
curl -X POST https://api.doit2winsolutions.co/demo/scrape \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com"}'
Enter fullscreen mode
Exit fullscreen mode
It is new and has no customers yet. I would rather hear that it breaks on your
pages than hear nothing.