# The Gate for AI-Written Tests: Introducing quelltest 0.3

> quelltest 0.3 proves a test fails when the code is wrong, flags weakened tests in a diff, and finds untested risky edges. Free, offline, Apache-2.0.

Published: 2026-10-20 | Author: Shashank Bindal | Tags: npm, ai-generated tests, mutation testing, javascript testing, release

Canonical page: https://quelltest.com/blog/introducing-quelltest-0-3-the-gate-for-ai-written-tests

**TL;DR.** A coding agent that writes both the code and the tests can make the suite pass without making it meaningful. quelltest 0.3 for npm adds two checks that do not depend on the agent's honesty: `quell verify` runs a test against mutants of your source and fails it if nothing notices, and `quell gate` fails a change that deleted, skipped or loosened tests. Both are free and offline.

Everything below was run on Windows with Node.js 24.11.0 and the published `quelltest@0.3.1` package from npm.

## What problem does quelltest 0.3 solve?

When an agent finishes a task, it reports that the tests pass. That sentence has two weak points. The tests may never have been able to fail, and the agent may have edited the tests to get to green.

Coverage does not help with either. A line that ran is not a line that was checked. We wrote about that in [100% code coverage is a lie](https://quelltest.com/blog/code-coverage-lie.md).

quelltest 0.3 is our answer for JavaScript and TypeScript. It is three commands you can run locally, in CI, or from inside an agent session.

## What do verify, gate and find do?

| Command | Question it answers |
| --- | --- |
| `quell verify TEST --source FILE` | Would this test fail if the code were wrong? |
| `quell gate --diff REF` | Did this change weaken the tests? |
| `quell find --diff REF --coverage FILE` | Which risky edges in my change have no real test? |

`verify` copies your project to a temporary directory, checks that the test passes on the original source, then runs it again on small mutations of the source: a removed guard, a flipped `<`, a `??` swapped for `||`, a removed `await`. A test that passes on every mutant is not protecting that code. Your real files are never written. The details are in [How quell verify works](https://quelltest.com/blog/does-your-test-fail-when-the-code-is-wrong.md).

`gate` compares the test files in your branch with their version at the merge-base. It reports deleted tests, skipped tests, a new `.only`, removed assertions and equality loosened to a truthiness check. See [Did the AI weaken your tests?](https://quelltest.com/blog/did-the-ai-weaken-your-tests-quell-gate.md).

`find` parses your source with a real parser and lists risk-ranked candidates per function: thrown errors, boundaries, nullish paths, and a rule pack for defects that lack a guard, such as a regular expression built from request data. It follows request data one hop from `req.body`, `req.query` and `req.params` into dangerous sinks.

## Why don't prompts asking agents to be rigorous hold up?

This is our position, not a measurement. An instruction in a prompt is a request. The agent can forget it, misread it, or satisfy its wording while missing its intent. When the same agent writes the code and the tests, an assertion that is inconvenient is also easy to loosen.

A program with an exit code does not negotiate. `quell gate` exits with code 5 when it finds weakened tests. `quell verify` exits with code 4 when it cannot verify a test. A CI job or an agent hook can act on those codes without trusting anyone's summary. We cover the hook setup in [Giving Claude Code, Cursor and other agents a verifier](https://quelltest.com/blog/quelltest-mcp-server-and-stop-hook-for-coding-agents.md).

## What does it look like on a real test?

Take a small function and two tests for it.

```js
export function applyDiscount(price, percent) {
  if (typeof price !== "number" || price < 0) {
    throw new RangeError("price must be a non-negative number");
  }
  const capped = Math.min(percent ?? 0, 50);
  return price - price * (capped / 100);
}
```

The first test checks only the type of the result:

```js
test("returns a number", () => {
  assert.equal(typeof applyDiscount(200, 10), "number");
});
```

It passes, and it has an assertion, so a naive check would accept it. This is what `verify` says with a 60% floor:

```bash
npm install --save-dev quelltest
npx quell verify tests/discount.weak.test.js --source src/discount.js --min-score 60
```

```text
Not verified: mutation score 13% is below the required 60% [copy]
  survived: ?? -> || (line 5)
  survived: guard removed (line 2)
  survived: throw removed (line 3)
  survived: || -> && (line 2)
  survived: < -> <= (line 2)
  survived: - -> + (line 6)
  survived: * -> / (line 6)
```

One mutant out of eight was caught. A fuller test file that checks the discount, the cap, the missing percent, the negative price and the zero boundary gets this (one mutant that cannot change behavior is marked with `quell-ignore-mutant`, which is explained in the post on `verify`):

```text
Verified: test killed 7/7 mutants (100%) [copy], 1 ignored
```

Without `--min-score`, a test is verified as soon as it kills at least one mutant. That is why the weak test above reports `Verified: test killed 1/8 mutants (13%)` and exits 0 by default. We recommend setting a floor; the benchmark below uses 60%.

Now the gate. We made a branch in a throwaway git repository where an agent "fixed" the suite by weakening it:

```bash
npx quell gate --diff main
```

```text
Gate failed: 4 issue(s) in 1 changed test file(s)
  ✖ tests/discount.test.js :: rejects negative prices — test deleted
  ✖ tests/discount.test.js :: accepts a price of exactly zero — test deleted
  ✖ tests/discount.test.js :: applies a percentage discount — assertions weakened (0 -> 1 weak, e.g. equality replaced by ok/truthy/not-null)
  ✖ tests/discount.test.js :: caps the discount at 50 percent — test is now skipped (skip/todo/x-prefixed)
```

`npm test` on that branch printed `pass 2, fail 0, skipped 1`. The suite was green.

## What does the benchmark show?

quelltest ships a benchmark, and `npx quell eval` prints it. This is the actual output from the published 0.3.1 package:

```json
{
  "rules": {
    "total": 18,
    "detected": 11,
    "recall": 0.6111111111111112,
    "detectableRecall": 1,
    "missedDetectable": [],
    "knownMisses": ["B3", "B8", "B10", "B12", "F3", "F5", "F6"],
    "cleanFiles": 7,
    "noise": []
  },
  "verifier": {
    "minScore": 60,
    "strong": 3,
    "strongAccepted": 3,
    "weak": 7,
    "weakRejected": 7,
    "accuracy": 1,
    "wrong": []
  }
}
```

(We reformatted the `knownMisses` array onto one line. The values are unchanged.)

Read it carefully. The rule pack detected 11 of 18 defects that were reported against a real code base. The other seven are documented known misses, mostly wrong business rules that a line-level rule cannot see. On seven files of guarded code it raised no findings. The verifier accepted all three strong tests and rejected all seven weak variants (tautology, literal-only, no assertion, wrong target, happy path only) at the 60% floor.

These fixtures were written by the authors. Treat the numbers as regression guards for the tool, not as an independent benchmark of how it performs on your code.

## How do I try it?

```bash
npm install --save-dev quelltest
npx quell verify tests/pay.test.js --source src/pay.js --min-score 60
npx quell gate --diff origin/main
npx quell find src --diff origin/main --coverage coverage/lcov.info
```

`find` never invents a score. With no coverage evidence it prints this instead of a number:

```text
Score: not scored (no coverage evidence) | 13 candidates
```

Quickstart: [/docs/npm/quickstart](https://quelltest.com/docs/npm/quickstart.md). Installation: [/docs/npm/installation](https://quelltest.com/docs/npm/installation.md). The full command list is in the [CLI reference](https://quelltest.com/docs/npm/cli.md).

## What are the limits?

- `verify` measures whether a test notices changes in your code. It does not know whether your mocks resemble the real services. See [Testing code that calls APIs and services](https://quelltest.com/blog/testing-code-that-calls-apis-and-services-quelltest-0-3-1.md).
- `verify` runs your tests once per mutant, so it is slower than one test run. The default cap is 30 mutants.
- `gate` does not catch a test rewritten from scratch under a new name, weakening done by changing the source, or logic hidden in helper functions that the test calls.
- Findings from `find` are review hints. Line-level rules cannot see wrong business rules.
- Generation handles guards the rules can read statically. Everything else becomes a TODO note, never a placeholder test.
- It is a young tool. Read the [changelog](https://quelltest.com/changelog) and [limits page](https://quelltest.com/docs/npm/limits.md).

## FAQ

### Does quelltest send my code anywhere?

No. It runs locally and sends no code to any service. An optional `--llm` hook runs a module you supply, and that module is your code.

### Which runners and Node versions work?

Jest, Vitest, Mocha and `node:test`. The published 0.3.1 declares Node.js 20 or newer; the next release will require Node.js 22.18+ or 24.11+. See [one verification layer for four runners](https://quelltest.com/blog/one-verification-layer-for-jest-vitest-mocha-and-node-test.md) for the runner details, including one `--runner` caveat.

### Is this a replacement for Stryker?

No. Stryker produces a full-suite mutation score. quelltest checks one test file or one change at a time and aims mutants at risky code. If you need a suite-wide score, use Stryker.

### What licence is it under?

The npm package is Apache-2.0. The Python package on PyPI is a separate product under the MIT licence. [Picking between them](https://quelltest.com/blog/quelltest-on-npm-or-pypi-which-package-to-use.md) has a decision table.

## Sources

- quelltest npm package: [npmjs.com/package/quelltest](https://www.npmjs.com/package/quelltest)
- Source repository, README and benchmark fixtures: [github.com/quelltest/quelltest-npm](https://github.com/quelltest/quelltest-npm)
- Changelog for 0.3.0 and 0.3.1: [CHANGELOG.md](https://github.com/quelltest/quelltest-npm/blob/main/CHANGELOG.md)
- Agents and CI recipes: [docs/agents.md](https://github.com/quelltest/quelltest-npm/blob/main/docs/agents.md)
- Apache License 2.0: [apache.org/licenses/LICENSE-2.0](https://www.apache.org/licenses/LICENSE-2.0)
- Stryker for JavaScript: [stryker-mutator.io](https://stryker-mutator.io/docs/stryker-js/introduction/)