Skip to content
All posts
·6 min read·Shashank Bindal

One Verification Layer for Jest, Vitest, Mocha and node:test

How quelltest detects your runner, what its runner matrix proves, and the exact verify and generate commands for Jest, Vitest, Mocha and node:test.

TL;DR. quelltest does not replace your runner. It runs your own tests, in a subprocess, against mutants of your source. quell generate detects Jest, Vitest, Mocha or node:test from package.json. quell verify does not detect anything: it uses node:test unless you pass --runner. We ran all four and found one more catch, with Vitest, which we describe below along with a workaround.

Captured on Windows with Node.js 24.11.0, the published quelltest@0.3.1, Jest 30.5.2, Mocha 12.0.3 and Vitest 5.0.5.

Do I have to change my test runner to use quelltest?

No. Keep the runner you have. quelltest asks the runner one question per mutant: did this test file pass or fail. It uses that answer to decide whether a test is capable of failing.

It always runs the runner as a subprocess and never in-process, because a module cache inside one process would hide the change you just made to the source.

How does runner detection work?

For generate, the runner comes from the nearest package.json, checked in this order:

  1. vitest in dependencies or devDependencies, or in the test script.
  2. jest, react-scripts, a jest key in package.json, or jest or react-scripts in the test script.
  3. mocha in dependencies or in the test script.
  4. Otherwise node:test.

Pass --runner node|jest|vitest|mocha to override it. The generated file matches the runner: describe/it with expect for Jest, an import ... from 'vitest' header for Vitest, describe/it with node:assert for Mocha, and test from node:test otherwise. node:test files get a .test.cjs or .test.mjs extension to match the module system.

We ran quell generate src/discount.js --write --verify in three tiny projects, one per runner, with no --runner flag. The first lines of each generated file:

jest:   const { applyDiscount } = require("../../src/discount.js");
        describe("applyDiscount", () => {
vitest: import { describe, it, expect } from 'vitest';
        import { applyDiscount } from "../../src/discount.js";
mocha:  const assert = require('node:assert/strict');
        const { applyDiscount } = require("../../src/discount.js");

Each run reported Wrote 1 tests (1 verified), 0 TODO notes, 2 flagged, 0 already covered.

What does the runner matrix prove?

The repository has test/matrix.test.js. It copies a small fixture project (CommonJS or ESM), generates tests for it, verifies them, and runs them under the real runner. We ran it from a copy of the repository's current main branch, because the published package does not include the test/ directory:

node --test test/matrix.test.js

✔ generated CJS tests run and verify under jest (3737.7704ms)
✔ generated CJS tests run and verify under mocha (674.64ms)
✔ generated ESM tests run and verify under vitest (2840.5496ms)
✔ generated CJS tests run and verify under node (659.6604ms)
✔ generated ESM tests run and verify under node (619.6303ms)
ℹ tests 5
ℹ pass 5
ℹ fail 0

That proves generation, the mutation loop and the pass/fail decision work for those five runner and module-style combinations on a tiny fixture. It does not prove they work with your Babel or ts-jest setup, your Vitest config, a monorepo layout or a custom reporter. Treat it as a smoke test for the plumbing.

What are the commands for each runner?

The fixture is the same function in each project, with tests for the discount, the cap, a negative price and a zero price.

node:test

npx quell verify tests/discount.strong.test.js --source src/discount.js

No flag needed. This is the default.

Jest

npx quell verify tests/discount.test.js --source src/discount.js --runner jest

Verified: test killed 7/8 mutants (88%) [copy]
  survived: ?? -> || (line 5)

Mocha

npx quell verify tests/discount.test.js --source src/discount.js --runner mocha

Verified: test killed 7/8 mutants (88%) [copy]
  survived: ?? -> || (line 5)

The survivor in both is the equivalent mutant explained in the verify post.

Vitest

npx quell verify src/discount.test.js --source src/discount.js --runner vitest

Verified: test killed 7/7 mutants (100%) [copy], 1 ignored

That worked because the test sits next to the source. Read the next section before you try Vitest with a separate tests/ directory.

What happens if I forget --runner?

verify runs node --test and the baseline fails. For a Jest file:

Not verified: test fails on the original code (node --test ...): ReferenceError: test is not defined

Mocha gives ReferenceError: describe is not defined and exits with code 4. The tool did not detect the runner for verify. If you want detection, use generate. For verify, pass the flag in your npm script or CI step.

What is the Vitest catch?

In our runs, --runner vitest failed when the test lived outside the source file's directory, for example tests/discount.test.js for src/discount.js:

Not verified: test fails on the original code (node ...\node_modules\vitest\vitest.mjs run ...\tests\discount.test.js): No test files found, exiting with code 1

Running Vitest directly in the same project passed all four tests. The likely cause, from reading the source: the runner's working directory defaults to the source file's directory inside the temporary copy, and Vitest treats the working directory as its root, so a test outside that root is not found. Jest and Mocha were not affected in our runs.

The workaround is --command, which runs your own command in the copy with the test path substituted:

npx quell verify tests/discount.test.js --source src/discount.js \
  --command "npx vitest run --root .. {test}" --timeout 120

Verified: test killed 7/7 mutants (100%) [copy], 1 ignored

We read this as a defect in 0.3.1 rather than intended behavior, because Jest and Mocha accept the same layout. Check the changelog for a fix. We ran this on Windows only.

What are the limits?

  • Only these four runners are supported: node, jest, vitest and mocha. Others, such as Ava or Jasmine, can be driven through --command but are untested.
  • The runner matrix uses one tiny fixture per combination, not real projects.
  • verify mirrors the project into a temporary directory and links node_modules. Very large projects fall back to in-place mutation with a backup file.
  • We did not test TypeScript projects for this post. quelltest runs your runner and not a compiler, so your runner has to be able to execute the .ts files.
  • We have not verified behavior on macOS or Linux for the cases above. The repository's CI runs on Windows and macOS as well, according to its README.

Docs: /docs/npm/verify, /docs/npm/cli and /docs/npm/limits.

FAQ

Which runner does verify use by default?

node:test. It does not read package.json. Pass --runner jest, vitest or mocha.

Does generate pick the right runner automatically?

Yes, from the nearest package.json, in the order vitest, jest, mocha, node. Use --runner to override.

Can I use quelltest with a runner it does not list?

Yes, through verify --command. Your command runs for the baseline and every mutant, and an exit code of 0 means the test passed.

Which Node.js version do I need?

The published 0.3.1 declares Node.js 20 or newer. The next release will require Node.js 22.18+ or 24.11+.

Sources