Testing code that calls APIs and services with quelltest

quelltest verifies whether your tests would notice a change in your code; it does not know whether the stubs those tests use resemble the real services. This page explains what quelltest does about code that depends on API calls and injected services: it derives stubs from the guards on a call's result, checks your stubs against recorded responses with quell mocks, labels survivors that sit on mocked code, and can run verification against real dependencies. It also says what none of this can do.

What are the two kinds of input?

How does stub synthesis work?

Take a function that receives its dependency as a parameter:

async function loadPlan(userId, fetchUser) {
  if (!userId) throw new Error('userId required');
  const res = await fetchUser(userId);
  if (res.status === 404) return { plan: 'free' };
  if (res.status !== 200) throw new Error('upstream failed');
  return { plan: res.data.items[0].plan };
}

quell generate treats the result of each call to something the caller supplies (a parameter, a method of a parameter such as User.findOne, or global fetch) as a symbolic variable. The guards on that variable are the path conditions. A small constraint solver takes candidate values from the constants in the conditions and their neighbours, picks an assignment that makes every guard false, then varies one field at a time. The expected outcome is that of the first guard in source order that fires: status 404 returns { plan: 'free' }; any other non-200 status must throw upstream failed, never behave like a 404.

Run on this function, the generator wrote (excerpt of the real output, test names and layout as generated):

// Stubs below are derived from the guards in the source, not from the real service. Compare them with recorded examples: quell mocks.

it("handles fetchUser result.status = 404", async () => {
  expect(await loadPlan(true, async () => ({ "status": 404 }))).toEqual({ plan: 'free' });
});

it("rejects fetchUser result.status = 403", async () => {
  await expect(loadPlan(true, async () => ({ "status": 403 }))).rejects.toThrow("upstream failed");
});

For global fetch the generated test replaces globalThis.fetch and always restores it in a finally. Candidate tests still have to pass on the original code and fail on a source mutant before they are kept when you pass --verify. Every generated test with a stub carries a comment saying the stub is derived from the guards, not from the real service.

The solver handles comparisons with literals, typeof, truthiness and ||-joined conditions. Anything else becomes a TODO note naming the unsupported expression. Guards nested in else-branches, loops or other conditions, reassigned parameters and functions that call the network without a stubbable result are refused with a reason.

How do I check my stubs against the real service?

Annotate the stub literal in a test with the route it stands in for, then compare it with recorded responses:

// quell-mock GET /users/:id
const stub = { data: { id: 1, name: 'Ada', plan: 7 } };

npx quell mocks tests --examples recorded.json
echo $?

Stub drift: 2 difference(s) against 1 recorded example(s)
  tests/user.test.js:2 GET /users/:id data.plan — stub has number, the recorded example has string
  tests/user.test.js:2 GET /users/:id data.email — the recorded example has this field, the stub does not
6

The examples file can be:

• a HAR file (method and URL path become the route, the JSON response body is the example),
• a JSON map such as { "GET /users/:id": { "status": 200, "body": { "data": { "id": 1 } } } },
• an OpenAPI document in JSON with response examples.

All three forms were run against the same stub and produced the same verdict. The annotation is // quell-mock METHOD /path, optionally status=200; the first object or array literal after it (within eight lines) is the stub, and it is compared with the response body. Routes accept :id and {id} parameters. The command reports fields only the recording has, fields only the stub invents and type mismatches; exit code 6 means drift, 0 means none. Dynamic values and objects with spreads never count as drift. --json prints { examples, problems }.

This follows the logic of consumer-driven contracts (Pact) and of mocks generated from values observed in production. It needs no provider to take part, which also means it only checks the recording you give it.

What is the mock-boundary label?

When you run verify --handoff, a surviving mutant is marked mockBoundary if its line uses a module the test replaces (jest.mock, vi.mock, sinon.stub, jest.spyOn, t.mock.method). In one report on a route file whose model layer was mocked, 46 of 46 survivors were removed awaits: a mocked model that resolves immediately hides a missing await. The label separates "weak test" from "the mock hides the behaviour". It is derived statically from the test and the source, so it misses doubles passed in by hand.

How do I measure a test against a real database?

verify --command runs your own command for the baseline and every mutant, so a test that talks to Testcontainers or an in-memory server is measured against the real dependency. Races, unique-index errors and transactions only exist there.

npx quell verify tests/orders.int.test.js --source src/orders.js \
  --command "npm run test:integration -- {test}" --infra-exit 70 --timeout 300

Reserve an exit code (here 70) for "the environment failed"; such runs are inconclusive, never kills. It costs time: about 1.4 seconds per mutant with mocks in one report, much more with containers. The command is yours and runs through your shell. Details: how verify works.

Which rules cover the dependency side?

What can it not do?

• It does not record live traffic or verify a provider. Use Pact, Keploy or Schemathesis for that. Importing HAR or Keploy recordings as inputs for generated cases is planned, not built.
• A high mutation score does not mean the mocks are right.
• A stub derived from guards cannot know fields the code never tests, status codes the service really returns, or whether res.data.items exists at all.

What research is this based on?

• AS4UT, an automated stub method for unit testing C/C++ projects: treats each function call as a mock variable and runs concolic testing over the unit, the same idea applied to JavaScript with a much lighter solver. The paper's venue was not confirmed; only the title and abstract were checked.
• Pex and Moles (Tillmann and de Halleux, Microsoft Research, .NET): parameterised unit tests whose inputs a constraint solver computes, with Moles isolating code from the environment through stubs and detours.
• EvoSuite environment and network mocking (Arcuri et al., ASE 2014 and FSE 2015): separating code from its environment raised average line coverage from 29.1% to 50.8% over 110 projects, and from 31.8% to 76.6% on 42 network-heavy classes (Java). The authors say generating the right interactions remains an open problem, which is why quelltest only emits stubs it can derive from the code's own guards and then verifies.
• Mimicking Production Behavior With Generated Mocks (ICSE 2025 journal-first track): mocks generated from values observed in production.
• MOCKMILL, Improving LLM-Driven Test Generation by Learning from Mocking Information (arXiv 2604.19315): mocking information extracted from developer-written tests is used to generate tests for the mocked components.
• Pact documentation, Comparisons: consumer-driven contracts versus stubbing libraries.

The same list, with notes on how far each source was confirmed, is in the package's docs/dependencies.md.