Point wrk at an MCP server and you get a clean report: every response a 200. Some of those 200s carry isError: true results, and nothing in the output says what the tool definitions cost the model on every turn. The tool is measuring requests per second. An agent runs a workflow, and what the server costs it is latency per call times calls per task, plus the tokens each response and each definition puts into its context.

You test an MCP server by chatting with it. Ask for the open bugs, get the open bugs, ship it. The model never sends limit: "ten", an empty path or a 200 KB query while you’re watching, so those paths stay dark until a real session hits one. Say it’s the limit. The handler throws, the framework wraps the exception in a 40 KB stack trace, and the model reads all of it, adjusts, and retries with the same bug in a new shape.

You’re about to refactor the billing endpoints. The service has a few dozen tests, mostly happy paths, and nobody trusts them to catch a changed rounding rule or a renamed field. Writing better ones by hand means reading every handler and inventing inputs. Meanwhile production receives thousands of real inputs a minute, and each one comes labelled with the response your current code gives.

Capturing that traffic is the easy part; a proxy, a packet tap or a log line with the body in it will do. The product is everything after. Ten minutes of traffic holds thousands of near-duplicate requests and a handful that exercise something different, and a tool is only useful if it can tell them apart. Cluster by endpoint, request shape and response shape. Pick representatives that cover the status codes and branches. Assert only on fields that are stable. Mock the downstream calls. What comes out is a few dozen characterization tests (Michael Feathers’ name for tests that pin down what code does today).

An API can pass every unit test in its suite and still break every client that calls it, because unit tests check that functions do what the code says they do, not that the API does what the contract says it does. That gap is where most production API incidents actually come from: a field renamed, a status code changed from 200 to 204, a required parameter quietly made optional. Testing an API well means testing at several different levels, because each one catches a different class of failure.

Unit tests verify that individual functions behave correctly in isolation. They run fast, catch regressions close to where they are introduced, and document expected behavior at the code level. They do not verify that the API endpoints exposed to consumers behave according to the documented contract, that changes in one service do not break the consumers that depend on it, or that the system behaves correctly under the conditions that production traffic creates.