Testing agents

Compatibility: Pactmark 0.1.x. @pactmark/testing is a development dependency and is never part of a production export.

What the testing package gives you

FakeClock

Deterministic time, so expiry and timeout tests are not flaky.

SequenceIdGenerator

Predictable ids, so snapshots are stable.

FakeModelDriver

Scripted turns — tool calls, finals, errors — with no provider key.

FakeTool / FakeToolExecutor

Tools that record how they were called, including that they were not called.

CrashInjector

crashAtEveryBoundary turns crash-safety into an enumerated matrix, not a hope.

ScenarioBuilder

Compose a deterministic scenario once and reuse it across suites.

Store contract suites

Run the same contract tests against memory and Postgres implementations.

Contract fixtures

Stable work orders, events, artifacts and protected values for adapter tests.

The coverage list

A green happy-path test tells you very little. These are the scenarios that matter:

Success and artifact production

The run completes, the artifact exists at the expected digest, required verifiers passed, and the evidence record validates.

Malformed input

External input that fails its schema produces KAF_SCHEMA_INVALID and no model call.

Policy denial

A denied risk class and a missing grant both refuse — and the tool executor records zero calls.

Grant and approval replay

The same command replayed returns the same semantic result. A one-use proof cannot be spent twice.

Cancellation

Cancelling mid-run reaches the tool’s AbortSignal, and the run reaches cancelled rather than hanging.

Budget exhaustion

Turns, tool calls and active execution milliseconds each terminate the run at their own limit.

Concurrent commands

Two commands racing on the same run: one wins, the other conflicts rather than interleaving.

Lease loss

A worker that loses its lease cannot commit. Assert on the fencing token, not on a log line.

Crash boundaries

Inject a crash at every boundary and assert the run is resumable and no effect was duplicated.

Uncertain effects

An uncertain outcome parks. Assert that the external target was called once, not that no error was thrown.

Registration drift

Change a tool schema without changing its version; assert the run suspends instead of resuming.

Cross-tenant access

Every store method, with another tenant’s authority. Assert refusal and non-disclosure.

Assert on effects, not on errors

This is the most valuable habit in this whole page:

// Weak — passes even if the refund was sent and then something else threw.
await expect(run).rejects.toThrow();

// Strong — states the property you actually care about.
expect(paymentGateway.calls).toHaveLength(0);
expect(events.filter((e) => e.eventType === "EffectDispatched")).toHaveLength(0);

A zero-dispatch counter is a security assertion. A thrown error is a symptom.

Crash matrices

import { CrashInjector, crashAtEveryBoundary } from "@pactmark/testing";

for (const plan of crashAtEveryBoundary()) {
  it(`resumes after a crash at ${plan.boundary}`, async () => {
    const injector = new CrashInjector(plan);
    // run until the injected crash, then resume in a fresh runtime instance
    // assert: the run completes, and the external target was called exactly once
  });
}

Two-process acknowledged-effect proofs are the strongest evidence you can produce for the uncertain-effect design. They are also the tests most likely to find a real bug in your own tool implementations.

Test the published bytes, not the workspace

Workspace links alone cannot prove package metadata, exports, type declarations or initializer developer experience. Consume packed tarballs in independent fixtures:

pnpm test:loopback-registry

That harness packs all candidates, publishes them dependency-first to an ephemeral loopback registry, executes the initializer, and leaves global npm configuration unchanged.

The repository runs independent NodeNext, Bundler, Yarn, Bun and loopback-registry fixtures for exactly this reason. “It works in the monorepo” and “it works when installed” are different claims.

Running the suites

pnpm verify

npx pactmark test --json

npx pactmark eval --json

Examples

Every example ships with the scenario tests described here.

Threat model

Each high and critical row names the executable evidence that backs it.

Getting started

Core concepts

Guides

Deployment

Production

Security

What the testing package gives you

FakeClock

SequenceIdGenerator

FakeModelDriver

FakeTool / FakeToolExecutor

CrashInjector

ScenarioBuilder

Store contract suites

Contract fixtures

The coverage list

Assert on effects, not on errors

Crash matrices

Test the published bytes, not the workspace

Running the suites

Examples

Threat model

​What the testing package gives you

FakeClock

SequenceIdGenerator

FakeModelDriver

FakeTool / FakeToolExecutor

CrashInjector

ScenarioBuilder

Store contract suites

Contract fixtures

​The coverage list

​Assert on effects, not on errors

​Crash matrices

​Test the published bytes, not the workspace

​Running the suites

Examples

Threat model

What the testing package gives you

The coverage list

Assert on effects, not on errors

Crash matrices

Test the published bytes, not the workspace

Running the suites