Skip to main content
ProdBreak is sometimes described as “behaves like the real API.” The real contract is more precise than that — and knowing exactly where the line sits is what lets you trust your green tests.

​
The fidelity contract

LayerGuaranteeWhose job
Faithful data — lifecycle transitions, round-trip of your inputs, server-derived fields, idempotency, aggregatesGuaranteed by construction. The pack is the single source of truth; you can’t fake it, and neither can a test.ProdBreak + the pack
Shape of out-of-control values — the structure of fields the API gets from outside itself (schema, format, enum validity)Guaranteed. Validated against the pack’s declared schemas; contract tests catch shape drift.ProdBreak + the pack
Coherence of values you script — do your pinned values + forced faults + branch choices add up to a state real prod could actually produce?Not guaranteed — your responsibility. ProdBreak plays back what you script; it does not invent causal links between independent values you set.You
The first two are the moat. The third is the deliberate boundary.

​
The honest boundary, with an example

ProdBreak never invents causality. If you script a failed pay_rent run and a balance that shows the money moved, it will faithfully serve you that contradiction — because in a passthrough like Deck, both are values you control and there’s no faithful link between them to enforce. 
await sandbox.faults.arm("task_runs", "create", { status: 500, code: "agent_failed" });
await sandbox.exogenous.pin({ operation: "fetchBalance" }, "output.balance", 0); // "after payment"
// the run failed, but you told us the balance dropped — both are served. That's on you.
This isn’t a bug; it’s the design. Enforcing “your scripted values must describe a possible prod state” would require the very effect-puppeteering we deliberately rejected. We guarantee faithful data and the shape of out-of-control values — not that arbitrary combinations of values you script are mutually consistent.
One more boundary worth knowing: seed writes a bare leaf and won’t fabricate the surrounding history (children, event trails, aggregates). That’s faithful-by-omission, not faithful-by-invention. Use it for bare-leaf assertions; a faithful replay seeder that runs the real lifecycle is on the roadmap. See Seed history.

​
How we guard the line: contract tests

Fidelity isn’t a claim we make once — it’s continuously checked against the real vendor API.

Shape drift (auto-caught)

Contract tests hit the live API, capture response shapes, and assert the pack matches — keys, types, envelope, pagination, error shape. Run on a schedule, they’re a drift detector, not a one-shot.

Behavioral drift (hand-watched)

Some changes are invisible to a spec diff: a default flips, an enum gains a value, a lifecycle transition is added. These hit the hand-modeled layer. We pin each pack to a modeled API version and keep a per-pack behavioral-assumptions list for the un-detectable half.

​
Cross-channel consistency

A shape test confirms balance exists and is a number. It can’t confirm the webhook and the GET resolve to the same value. So packs add a dedicated probe — trigger a real run, capture the webhook, GET the resource, assert the values match — which both discovers and guards the source-of-truth wiring. It’s the test that catches a vendor silently splitting one fact into two backing sources.

​
Bottom line

ProdBreak is shape-faithful and state-faithful, continuously verified against the real API. It does not cross-check the values you invent for coherence. Script states that could really happen, and your green tests mean what you want them to mean.