Skip to content

How the Python quelltest pipeline works

The Python package turns specifications that already exist in your code (guard clauses, docstrings, Pydantic models, type annotations, bug reports) into testable requirements, checks which requirements have no test, generates a pytest test for each gap, proves that the test catches a violation, and only then writes it to your test file. Read this page to understand why a test was written, scaffolded or flagged.

What are the stages?

spec readers  ->  requirements  ->  coverage checker  ->  synthesis  ->  verification  ->  writer
  1. Spec readers produce one unified Requirement model.
  2. Coverage checker marks each requirement covered or uncovered.
  3. Synthesis generates a test: a deterministic rule engine first, an LLM only when you opt in.
  4. Verification runs five gates; only tests that pass them continue.
  5. Writer injects the test with libcst, with a backup and a syntax check.

Which spec readers exist?

ReaderReadsUsed by
Code guardsif ... raise, assert, isinstance checks, None checks, auth checks, try/except re-raises, bare except, silent return None, magic values in conditions. Pure AST, no LLM.quell find
DocstringsGoogle, NumPy and RST docstrings: Raises, parameter constraints, enumerated values, Returns. Prose falls back to an LLM only if one is configured.quell ci, quell prove, the Python SDK, the MCP tools
TypesPydantic Field(gt=..., ge=..., lt=..., le=..., min_length=..., max_length=...), Literal[...], Annotated constraints.quell ci, quell prove, the SDK, the MCP tools
PySparkPySpark schemas, when enable_pyspark is set.the SDK
Bug reportsA natural-language bug description.quell reproduce

Every reader returns an empty list on any error and never raises.

How does the coverage checker decide?

It parses your test files with ast and does not run them. For a source file payments.py it looks for tests/test_payments.py, tests/payments_test.py or test_payments.py next to the source, and matches test functions to requirements. When it is uncertain it marks a requirement uncovered: a duplicate test is better than a missed gap.

How is a test generated?

The rule engine handles these kinds with deterministic templates: must_raise, boundary, enum_valid, must_return, bug_repro, not_null, type_check, silent_fail, magic_value and custom. See constraint kinds. Framework route handlers (FastAPI, Flask) are routed to a framework engine that needs an app object in the project. The LLM is only called when the rule engine cannot handle a requirement and you passed --use-llm.

What are the five gates?

GateCheck
1The test parses as valid Python and its imports resolve.
2Originality: it is not a duplicate of an existing test.
3Security: no eval, network or environment mutation.
4The test passes on the original code.
5The test fails when the requirement is violated in the source.

The verifier writes the candidate to a temporary file, runs pytest in a subprocess (never in-process, so module caches cannot hide a change), then injects a violation into the source (for example commenting out the raise, or weakening a threshold), runs the test again and requires it to fail. The source is always restored in a finally block. Details: five-gate pipeline.

How is a test written?

The writer injects the test function into the test file with libcst, never with string concatenation. It backs the file up first, validates that the new module parses before writing, restores the backup on failure, and appends an entry to an audit log (.quell/audit.jsonl by default).

What happens to requirements that fail?

They are not dropped. Each ends up in a bucket (written, scaffolded or flagged) with a reason in quell-report.json. See three-bucket output and the Production Readiness Score.

For how the npm package verifies tests (mutating your source rather than injecting a violation of one requirement), see how verify works.