Skip to content
All posts
·8 min read·Shashank Bindal

Testing Code That Calls APIs and Services: What quelltest 0.3.1 Does About It

quelltest 0.3.1 derives stubs from the guards on a call's result, restores fetch after each test, and checks stubs against recordings. And what it cannot know.

TL;DR. quelltest 0.3.1 reads the if checks your code makes on the result of a call (res.status === 404) and builds stubs that exercise each branch, including a fetch stub that is always restored. It can also compare the stubs in your tests with recorded responses and report drift. It cannot know whether a stub resembles the real service, and it says so in every generated test.

All output below comes from quelltest@0.3.1 (published on npm), run on Windows with Node.js 24.11.0.

Why is code that calls services hard to test automatically?

Two kinds of input reach a function. Users give it arguments, request bodies and query strings. Dependencies give it results: the status and shape of what fetchUser(id) returns, what User.findOne(q) resolves to, what fetch(url) brings back.

For the first kind, a tool can read the guards on a parameter and derive boundary cases. The second kind is harder because the right value depends on a service the tool cannot see. Any mutation or test-generation tool measures test strength against whatever the tests mock. quelltest is no exception.

So the 0.3.1 design rule is narrow: only emit a stub that the code's own conditions imply, label it as derived, and then require the resulting test to fail on at least one mutant.

What are path conditions over a dependency result?

Here is a function that takes its HTTP client as a parameter:

export async function loadPlan(userId, fetchUser) {
  if (!userId) throw new Error("userId required");
  const res = await fetchUser(userId);
  if (res.status === 404) return { plan: "free" };
  if (res.status !== 200) throw new Error("upstream failed");
  return { plan: res.data.items[0].plan };
}

quell generate treats the result of fetchUser(...) as an unknown value. The checks on it (res.status === 404, res.status !== 200) are the path conditions. A small solver takes candidate values from the constants in those conditions and their neighbours, then varies one field at a time. The expected outcome is that of the first guard in source order that fires. A status of 500 must produce upstream failed, never the 404 result.

We ran it with writing and verification turned on:

npm install --save-dev quelltest
npx quell generate src/plan.js --write --verify

Wrote 3 tests (3 verified), 2 TODO notes, 1 flagged, 0 already covered
  TODO src/plan.js:6 — no executable test rule for deep-response-chain findings yet
  TODO src/plan.js:1 — no executable test rule for async findings yet

One of the generated files looks like this:

// Stubs below are derived from the guards in the source, not from the real service.
// Compare them with recorded examples: quell mocks.

test("loadPlan: handles fetchUser result.status = 404", async () => {
  assert.deepStrictEqual(await loadPlan(true, async () => ({ "status": 404 })), { plan: "free" });
});

The files also include six rejects tests for statuses such as 403, 405, 199, 201, 1 and 0, and three for a falsy userId. All ten passed under node --test.

Notice what is missing. There is no test for status 200, the happy path. That branch reads res.data.items[0].plan, and the tool cannot invent what a real data.items looks like. It wrote a note instead:

# TODO: risk (deep-response-chain) in `loadPlan`
- Evidence: `res.data.items[0].plan`
- Why no test was generated: no executable test rule for deep-response-chain findings yet
This is a note, not a test: nothing here executes or asserts anything.

How do the fetch stubs work?

When the dependency is the global fetch, the generated test replaces globalThis.fetch and restores it in a finally block. The security gate on generated code allows globalThis.fetch only as an assignment target or as const originalFetch = globalThis.fetch.

export async function isHealthy(url) {
  const res = await fetch(url);
  if (res.status === 503) return false;
  if (res.status !== 200) throw new Error("unexpected status");
  return true;
}

This is one of the generated tests for it:

test("isHealthy: rejects fetch result.status = 502", async () => {
  const originalFetch = globalThis.fetch;
  globalThis.fetch = async () => ({ "status": 502 });
  try {
    await assert.rejects(() => isHealthy("abc"), { name: "Error", message: "unexpected status" });
  } finally {
    globalThis.fetch = originalFetch;
  }
});

The run wrote 2 tests (2 verified), including the 200 and 503 cases. find also flags the call itself: unguarded-network-call fires when a network call has no try/catch, no .catch chain and no known async wrapper, and asks for a timeout or signal.

How does the quell-mock contract check work?

Annotate the stub in your test with the route it stands in for, then give quell mocks a recording: a HAR file, a JSON map or an OpenAPI document in JSON.

test("returns the plan from the first item", async () => {
  // quell-mock GET /users/:id status=200
  const body = { data: { items: [{ plan: "pro" }], name: "Ada", seats: "3" } };
  const result = await loadPlan("u1", async () => ({ status: 200, ...body }));
  assert.deepEqual(result, { plan: "pro" });
});

With a recording that has data.items[0].renews and a numeric seats:

npx quell mocks tests/plan.test.js --examples recorded.json

Stub drift: 3 difference(s) against 1 recorded example(s)
  tests/plan.test.js:7 GET /users/:id data.items[0].renews — the recorded example has this field, the stub does not
  tests/plan.test.js:7 GET /users/:id data.seats — stub has string, the recorded example has number
  tests/plan.test.js:7 GET /users/:id data.name — the stub invents this field, the recorded example does not have it

The command exits with code 6 on drift. The test itself was green. That is the point: a passing test with a stub that disagrees with reality is the failure this check is for.

Two details we learned by running it. The stub literal is compared with the recorded response body, so annotate the body object and not the whole response with its status. And the check only knows the recording you hand it. It does not need the provider to take part, which also means it cannot tell you the recording is typical.

Which research does this follow?

We did not invent the idea. The documentation lists the work behind it:

  • Pex and Moles, from Microsoft Research, paired white-box test generation with isolation through stubs and detours for .NET. The project page says both have been superseded by IntelliTest and Fakes. Project page.
  • Arcuri, Fraser and Galeotti showed for the EvoSuite generator that separating classes from their environment improves generated tests. Their ASE 2014 paper reports coverage gains of up to +80% and +90% in some cases and more than 50% fewer unstable tests across 100 SourceForge projects. Their FSE 2015 paper on network mocking reports line coverage rising from 31.8% to 76.6% on 42 network-heavy Java classes. ASE 2014, FSE 2015 PDF.
  • Tiwari, Monperrus and Baudry generated mocks from values observed in production (the RICK tool), presented at ICSE 2025 as a journal-first paper. Programme entry, arXiv.
  • MOCKMILL extracts mocking information from developer-written tests to steer LLM test generation. The authors call their evidence preliminary. arXiv 2604.19315.
  • Pact is a consumer-driven contract testing tool: the consumer's code generates the contract. quell mocks is a much lighter relative that checks a recording, not a provider. Pact documentation.

Those papers are on Java and .NET. We borrowed the idea and built a much lighter solver for JavaScript. They do not validate quelltest's results.

What are the limits?

  • A stub derived from guards has the shape the code's own conditions imply. It cannot know fields the code never tests, status codes the service really returns, or whether res.data.items exists.
  • The solver handles comparisons with literals, typeof, truthiness and ||-joined conditions. Anything else becomes a TODO note naming the unsupported expression.
  • Guards nested in else-branches or loops, tests of reassigned parameters, and functions that call the network without a stubbable result are refused with a reason.
  • There is no live traffic recording and no provider verification. Use Pact, Keploy or Schemathesis for that. Importing recordings as inputs for generated cases is planned, not built.
  • A high mutation score does not mean the mocks are right. On mocked code, verify --handoff labels survivors mockBoundary so you can tell a weak test from a mock that hides behavior. verify --command runs your own integration command against every mutant, for example against Testcontainers.

See dependencies and limits in the docs, and the changelog.

FAQ

Do the generated stubs replace integration tests?

No. They exercise your code's branches against shapes your code implies. Integration tests check the real service.

Does quell mocks call my API?

No. It reads annotated stubs and a recording file. Nothing is sent anywhere.

What happens if the stub drifts?

quell mocks exits with code 6 and prints each difference with a file and line. Dynamic values and spread objects never count as drift.

Can the fetch stub leak into other tests?

The generated test restores globalThis.fetch in a finally block, so it is restored even when the assertion fails.

Sources