Test Audit

Three modes, one value bar. Authoring gates every new or changed test at write time. Audit runs a focused sweep of existing tests that re-assert source, duplicate stronger proof, couple to implementation, or keep test-only production seams alive. Campaign prunes one subsystem's whole test surface in parallel lanes. Optimize for confidence, not deletion count — but a test that cannot name the bug it catches is cost, not coverage.

Read .claude/rules/sim-testing.md first: it defines the test layers, file naming, and the mechanics (global mocks, @sim/testing, performance rules).

The three rules

1. Never write unit tests after you write code. A test written to describe code that already exists restates the implementation, passes on the first run, and proves nothing. If the change needs proof, prove it end to end.
2. Prefer E2E tests; use them to verify complex features. Exercise the real boundary — real Postgres/Redis (*.integration.ts), a running app over real HTTP, or the packaged desktop app (Playwright). Every E2E run ends with a verifiable, repeatable artifact: a JSON report of checks with pass/fail and durations, an HTTP status log, a trace, or a screenshot, written to a path the caller controls (<SUITE>_REPORT_PATH) and uploaded by CI on failure. apps/sim/scripts/test-scim-e2e.ts is the reference shape.
3. If you must test a system in isolation, write the failure modes down first, then write the code. List every way the unit can fail (bad input, boundary, concurrency, partial failure, permission denial, resource cap). Each listed mode becomes one test that fails before the code exists. A mode you cannot list is not a test you should write.

Authoring gate

Before adding any test, answer all four. A missing answer means do not add it.

1. What observable behavior, invariant, or independent contract does it protect?
2. What credible regression makes it fail?
3. Why does existing coverage not already catch that failure? Type-check, next build, bun run check:audits, and the integration/E2E suites are coverage too. Each contract has one primary owner at the strongest boundary; another layer needs its own distinct risk. Prefer extending an existing table-driven case over a near-duplicate test.
4. Does it need a production seam (export, flag, wrapper, injection hook) that no production caller needs? If yes, test at the real boundary instead.

Then check it against every junk pattern below. A match fails the gate unless the retention bar names the contract it independently guards. A test that would break under behavior-preserving refactoring asserts implementation, not behavior.

Regression tests must fail on the pre-fix code for the intended reason. Revert each guard of the fix separately and watch the test named for that guard go red, then restore. A regression test that never demonstrably failed proves the mock, not the fix. One regression at the owner boundary covers the bug; do not replay it at every layer it crosses.

Junk patterns

• assertion-free or toBeDefined()-only tests; "renders without crashing";
• restating declarations: block/tool/trigger/provider config (subBlock ids, params, outputs, URL templates, header maps), constant tables, registries, enums, export lists — type-check and check:audits own these;
• route/handler tests that mock every collaborator and assert toHaveBeenCalledWith on the mocks, or re-assert a mock's canned return;
• mocks that implement the asserted behavior, or one mock standing in for different APIs;
• Zod contract tests proving a schema accepts a valid object or rejects an obviously invalid one;
• React tests of text, class names, aria presence, snapshots, "calls onClick";
• hook tests asserting query keys or fetch URLs; store tests of trivial setters;
• tests of test infrastructure (mocks, factories, builders testing themselves);
• source-text or import greps (readFileSync(src) + toContain);
• expected values produced by the helper under test;
• duplicate invocations of the same contract, or provider-local replays of a shared helper;
• fixtures that supply the ordering, receipt, or callback the owner should produce;
• negative controls that pass for an unrelated reason (a different guard short-circuits first);
• names or fixtures that promise more than the input exercises;
• dead production code or exports whose only callers are tests.
• a hand-rolled vi.mock factory for a module vitest.setup.ts or @sim/testing already mocks, or a local copy of a @sim/testing helper (bun run check:test-patterns fails on these).

Retention bar

Keep a test when it independently enforces one of:

• security — authn/authz denial, tenant/workspace isolation, SSRF/URL validation, secret redaction, encryption, signature verification, path traversal, injection, rate limits;
• money and data integrity — billing/usage math, metering, quotas, idempotency, migrations, persistence semantics, concurrency/locking/leases, outbox, retries;
• executor semantics — DAG traversal, loops/parallels, conditions/routers, reference resolution, streaming, pause/resume, run-from-block, cancellation;
• real algorithms with edge cases — chunkers, parsers, query builders, cron, diff/merge, pagination, encoding, date math, ranking;
• cross-process wire contracts — realtime protocol, desktop bridge/IPC, CLI/SDK wire, provider webhooks, MCP — that type-check cannot see;
• a regression with a credible repeat, shown red on the pre-fix code.

Also keep call ordering when order is observable, and a source inspection when it is the cheapest independent guard of a user-facing byte, key, or path. A retained test that fails on the baseline is a possible product bug: reproduce it and fix the owner rather than deleting it. Static or slow is not a deletion reason.

Audit mode

Keep discovery read-only and report evidence before editing. Before judging a candidate, read the complete test and its production owner, callers, sibling implementations, overlapping tests, CI routing (CI discovers *.integration.ts by glob; .github/workflows/*.yml names a few scripts and files by path), and relevant history (git log --format='%h %s' -5 -- <file>).

Record for every deletion candidate: the test and location; the failure it can actually detect; non-test callers of the seam it covers; the stronger remaining proof (or why none is needed); the production or test-support code its deletion unlocks; and the focused validation command.

Edit shape. One coherent owner-boundary batch per PR. When pruning inside a file, also delete now-unused imports, mocks, fixtures, and helpers. Delete test-only exports and dead production paths instead of preserving aliases (rg -n '<name>' --glob '!**/*.test.*' must show no other reference, including string and dynamic-import references; never delete route files, registry entries, or generated files). Prefer net-negative production LOC. Do not add replacement tests that restate the same implementation.

Campaign mode

For a whole subsystem or the whole repo:

1. Partition test files into lanes of ~150–350 files by owning directory, and list the protected set (every *.integration.ts, *.live.test.ts, __integration__/**, apps/desktop/e2e/**, and every path named in .github/workflows/*.yml).
2. Give each lane its own git worktree and branch (git worktree add -b <branch> <path> <base>, then bun install --frozen-lockfile inside it). Lanes never share a checkout, never symlink node_modules, and never use git stash — the stash is shared across worktrees.
3. Each lane commits once and writes a report: counts, categories removed with examples, notable keeps and why, production seams removed with grep evidence, and the commands it ran.
4. Merge lane branches, then sweep orphaned shared test support (packages/testing/**, fixtures, helpers) that no remaining test imports.

Validation

Never edit source or tests while Vitest is running in the same checkout.

1. Run the touched and sibling test files (.claude/rules/sim-testing.md → Running).
2. If production code changed: bun run type-check in that workspace.
3. bun run check:audits from the repo root (some audits list test files by path).
4. bun run lint, then git diff --check.
5. Report git diff --shortstat with production and test changes counted separately.

Handoff

Report the categories removed, production simplifications, retained false positives and why they stay, the validation actually run, production vs test LOC, and named follow-ups.