--- title: "Testing kit" description: "Test against in-memory fakes of every interface, assert what your code sent, control time and inject provider outages, with no provider account." section: "interfaces" status: "Building for 0.4 · ships as a stable API" spec: "testing-kit" canonical: "https://mainsdk.dev/docs/interfaces/testing" updated: "2026-10-28" --- # Testing kit > Test against in-memory fakes of every interface, assert what your code sent, control time and inject provider outages, with no provider account. Status: Building for 0.4 · ships as a stable API · Canonical: https://mainsdk.dev/docs/interfaces/testing · Updated: 2026-10-28 `@mainsdk/testing` lets you test code that uses MainSDK with no provider account or network. `createTestApp()` builds your app from `main.yaml` on in-memory drivers and a clock you control. It makes that app the one behind `import { files } from "mainsdk"`, so your route code runs unchanged, and a network never leaves the process. Reach for it in every unit and integration test, and in CI. > **Note:** Built today: `createTestApp()`, the test clock, the faked network, tenant helpers (`asTenant`, `twoTenants`), helpers for api keys, plans, limits, protected files, queries, resources and sign-in, the email mailbox and notify channels (`mailbox(app).toHaveSent()`, `channels(app)`, `fakeMessagingServer()`; see [Email](https://mainsdk.dev/docs/interfaces/email)), and the `fakeAiDriver()` and `testCards()` drivers. Also built: `expect(app.pay).toHaveCharged()`, `chaos.on()` drills, `policyTest()`, the driver kit with `mainsdk driver test`, the compatibility matrix and the parity vectors (`mainsdk conformance`). See [Planned](#planned) for what is left. ## At a glance - Status: Building for 0.4 · ships as a stable API - Phase: 1 (Core, public beta) - Milestone: 0.4 Hardening - Plan: free in your app (Apache-2.0) - Spec: https://github.com/hubxero/mainsdk/blob/develop/docs/specs/testing-kit.md - Category: Developer experience Calls: - `createTestApp({ main })` - `createSim({ apps, start, seed })` - `sim.run(duration, { step })` - `sim.inject.cpu(app, curve)` - `sim.expect.signal(app, name)` - `sim.report({ golden })` - `app.fixtures(opts?)` - `mailbox(app).toHaveSent()` - `clock.advance()` - `chaos.on()` - `expect(app.pay).toHaveCharged()` - `assertFakesCover(ids)` - `runDriverKit(driver)` - `policyTest()` - `createApp(options)` - `configure(app)` - `app()` - `fakeAiDriver()` - `testCards()` - `new App(opts?)` In main.yaml: ```yaml ``` Guarantees, whatever the driver: - A fake for every interface, generated from its contract ## Drivers Pick one in main.yaml with `testing.driver`. | Driver | Id in main.yaml | Default | Note | | --- | --- | --- | --- | | The local drivers with recording and chaos | `local` | yes | | ## Codes No check or runtime codes are assigned to this interface yet. ## Features | Feature | Phase | Milestone | Spec ticket | | --- | --- | --- | --- | | In-memory fakes for every interface | 1 | 0.4 | `testing-kit#fakes-core` | | Assertions and a controllable clock | 1 | 0.4 | `testing-kit#assertions-time` | | Policy test helpers | 1 | 0.4 | `testing-kit#policy-helpers` | | Chaos drills | 1 | 0.4 | `testing-kit#chaos-drills` | | Conformance suite | 1 | 0.4 | `testing-kit#conformance-suite` | | Parity vectors for the other language SDKs | 1 | 0.4 | `testing-kit#parity-vectors` | | Driver kit | 1 | 0.4 | `testing-kit#driver-kit` | ## MainSDK Cloud Everything on this page is free in your app. ## Quick start The kit runs from the repository; it is not published to npm yet. Import it from your workspace. ```ts title="app/rates.test.ts" import { http } from "mainsdk"; import { createTestApp } from "@mainsdk/testing"; const app = createTestApp({ yaml: "schema: 1\nnetwork:\n allow: [api.partner.com]", respond: () => Response.json({ rate: 1.1 }), // answers every call the allow list lets through }); const res = await http.fetch("https://api.partner.com/v1/rates"); console.log(await res.json(), app.sent.length); // { rate: 1.1 } 1 ``` An undeclared host is refused with `MAIN-NET-0002`, exactly as in production. ## Configuration The kit has no `main.yaml` block. It reads your config and puts every interface on a fake. | Option | What it does | | --- | --- | | `main` | Path to `main.yaml`. Overlays are not applied, so tests see the base config | | `yaml` or `config` | The config as text or an object, instead of a file. With none, you get `schema: 1` | | `respond` | Answers outbound calls that `network.allow` lets through; the default is 204 | | `dns` | Fake addresses by host, for the network layer's DNS checks | | `secrets` | Secret values by name | | `clock` | Your own `createClock()` | | `resources`, `metrics`, `signals`, `exit` | Fake sources for health and resources, a metrics sink, shutdown signals and what ends the process | | `auth` | What sign-in calls out to, such as `fakeWhatsApp()` | | `dir`, `configDriver`, `broadcastBus`, `fsWatch` | For live-config tests that link several apps | | `realTimers` | Use real timers for tests that talk to a real socket | ## What you get - `createTestApp()`: your app built from `main.yaml` with every interface on an in-memory driver, behind the module-level `mainsdk` imports. - A test clock (`clock.advance`) and a faked network that refuses undeclared hosts as production does. - Tenant helpers (`asTenant`, `twoTenants`), a mailbox and notify channels, and fakes for AI, payments, sign-in, resources and metrics. - Assertions for what your code sent, such as `app.sent`, `mailbox(app).toHaveSent()` and `expect(app.pay).toHaveCharged()`. - `chaos.on()` drills, `policyTest()`, `createSim` for a simulated afternoon, `app.fixtures()`, the driver kit and the parity vectors. ## Examples ### Control time `app.clock.advance()` moves time with no real waiting, for expiring links, TTLs and retries. `clock.tick()` also lets the work each timer starts finish. ```ts title="app/cache.test.ts" import { kv } from "mainsdk"; import { createTestApp } from "@mainsdk/testing"; const app = createTestApp(); await kv.set("k", 1, { ttl: "1h" }); app.clock.advance("2h"); console.log(await kv.get("k")); // null ``` ### Upload without a browser ```ts title="app/avatar.test.ts" import { files } from "mainsdk"; import { createTestApp } from "@mainsdk/testing"; const app = createTestApp(); const up = await files.uploadUrl({ owner: "u1", types: ["image/png"] }); await app.files.put(up, new Uint8Array([0x89, 0x50, 0x4e, 0x47]), { type: "image/png" }); const ref = await files.complete(up.ref.id, { owner: "u1" }); // ref.status is "ready" ``` ### Run as a tenant, and prove two tenants stay apart `app.asTenant({ id, plan?, roles?, user? })` runs code as a member of a tenant, making it on first use (ids look like `tnt_acme`). `twoTenants(app, { seed, routes })` seeds a unique marker as one tenant, reads your routes as another and fails naming the route and interface that leaked it. ```ts title="app/tenancy.test.ts" import { createTestApp, twoTenants } from "@mainsdk/testing"; const app = createTestApp({ yaml: "schema: 1\ntenancy:\n resolve: [session]\n" }); const notes = new Map(); await twoTenants(app, { seed: async (marker) => { notes.set(app.tenancy.prefix("note"), marker); }, routes: { "GET /note": { interface: "kv", read: async () => notes.get(app.tenancy.prefix("note")) ?? null }, }, }); ``` ### Other helpers | Helper | What it does | | --- | --- | | `app.apikeys.issue({ tenant, scopes })`, `withApiKey(req, key)` | Issue a key without a caller, and send it as a bearer token (needs an `apikeys` section) | | `app.entitlements.setPlan()`, `setUsage()` | Put a tenant on a plan and set the usage a limit reads (needs `plans`) | | `app.limits.exhaust(policy, key)`, `storeDown(true)` | Use a policy up, or make the shared store fail | | `app.protect.signals()`, `party()`, `chain()` | Control the signals and parties of the `local` protect driver | | `expectQueries(handler, { repeat? })` | Fail on an N+1 query pattern with `MAIN-DATA-0054` | | `fakeResources()`, `fakeSignals()`, `fakeMetrics()` | Fake sources for [status and resources](https://mainsdk.dev/docs/interfaces/status-page) tests | | `createFakeOidcProvider()`, `createFakeAuthenticator()`, `fakeWhatsApp()` | Fake sign-in providers | | `fakeAiDriver()`, `testCards()` (from `mainsdk`) | A model that answers from a script, and a payment driver with test card numbers; neither can run in production (`MAIN-CHECK-0026`) | ## Assertions and policy tests `mailbox(app).toHaveSent({ to, template, times })` fails with the messages that were sent when none match. `watchCharges(app.pay)` records the checkouts made through the fake, and `expect(app.pay).toHaveCharged({ amount: 4900, currency: "INR", times: 1 })` asserts on them in minor units. `policyTest(source)` asks a policy set for a decision and checks the answer and the rule behind it. The source is a directory of Cedar files, or an engine with a `decide`, such as `app.access`. ```ts import { createTestApp, policyTest } from "@mainsdk/testing"; const app = createTestApp(); policyTest(app.access) .expect({ subject: { id: "u_2", roles: ["member"] }, action: "orders.update", resource: { type: "order", owner: "u_1" } }) .toDecide("deny") .byRule("orders.owner_only"); ``` `assertFakesCover(ids)` fails with `MAIN-TESTING-0001` when an interface in the catalog has no fake in the kit, and `fakeCoverage(ids)` lists the resolved, planned and missing ones without failing. ## Chaos drills A drill sits on the fake network your app calls out through, so the real drivers run behind it and the contract's own timeouts, retries and fallbacks decide what happens. The test clock runs the waits, so a 3 second delay takes no real time. ```ts import { chaos, createTestApp } from "@mainsdk/testing"; const app = createTestApp({ main: "./main.yaml" }); // email: { driver: resend, fallback: [ses] } chaos.on("email:resend", { outage: true }); // every call fails: the send goes through ses chaos.on("email:resend", { latency: "3s" }); // times out per network.http.timeout, then fails over chaos.on("email:resend", { respond: "malformed" }); // raises a typed error: MAIN-EMAIL-0004, not retryable chaos.off(); // end every drill ``` A target is a driver (`email:resend`), a whole interface (`email`) or a host (`api.resend.com`). `respond` also takes `"error"` (a 500) or a status number. A drill that fails a call before it leaves is not in `app.sent`. An unknown target throws `MAIN-TESTING-0002`. ## Test a driver `mainsdk driver test [--interface email@1.4] [--json]` runs the same suite that `runDriverKit(driver)` from `@mainsdk/testing` runs in code, and returns a report of passed, failed and skipped cases. The suite is generated for the interface version a driver claims and runs against a local harness that answers fake provider traffic. It is the suite the first-party drivers pass, so there is one bar. A run prints a report like this: ```text email@1.4 conformance: 8 passed, 0 failed, 2 skipped · capability profile: attachments=no · idempotency=sdk ``` A driver is made with `defineDriver` and carries a `manifest` of its regions, data labels and hosts. A case the capability profile switches off is reported as skipped with the profile key, never as passed. A call to a host the manifest does not declare fails the run with `MAIN-NET-0002`, even when the driver caught the error. A driver with no manifest, or one that is not a `defineDriver` module, fails with `MAIN-TESTING-0003`. The suite for each interface version is generated from the contract, so it covers every operation and guarantee. `mainsdk driver matrix [--out file]` prints the compatibility matrix of the first-party drivers: each driver, interface version and pass or fail. ## Parity vectors `mainsdk conformance export [--out conformance/] [--check]` writes language-neutral JSON from the TypeScript reference: `contract/main-yaml` (every top-level key, set right and wrong, with the nested keys it covers), `contract/errors` (every registry code with its problem body and status) and `contract/wire` (licence tokens, build hashes, telemetry records), and a `manifest.json` with a digest. The same export twice gives the same files. `--check` fails and names every file that differs from a fresh export. `mainsdk conformance run --impl "" [--only contract] [--json]` plays the vectors to a runner in another language. The runner reads one JSON case per line on stdin and prints one `{"id","result"}` line per case it answers on stdout. A case it does not answer, or answers with `{"id","skipped":true}`, is reported as skipped, never as failed. A line that is not JSON raises `MAIN-TESTING-0004`. ## Fixtures, seed data and previews `await app.fixtures({ seed: 7, tenants: 2, models: { orders: 3 } })` fills the test app's in-memory stores with rows that pass your schemas and returns `{ tenants, users, rows, digest, as }`. The same seed number and counts give the same rows and the same `digest` as `mainsdk seed`. Pass `data: { orders: [{ status: "refunded" }] }` for rows you want by hand; every field you leave out is generated, and a row that fails its schema raises `MAIN-SCHEMA-0001` with the path (`["orders", 0, "status"]`) and no value. `await fx.as(fx.users[0], fn)` runs `fn` in that user's tenant with a session for that user. Generated emails use `example.com`, `example.org`, `example.net` or `.test`, phone numbers come from reserved ranges, and names, addresses and currencies follow the tenant's region. Fields labelled `child`, `phi` or `sensitive` get obviously synthetic values. `mainsdk seed [--number n] [--tenants n] [--reset] [--no-ai] [--init-local] [--env name] [--json]` writes the same rows to your local database. List the modules that export your models under `seed.schemas`. It refuses production before it opens a connection (`MAIN-SEED-0001`) and any database that is not local, a preview or marked seedable (`MAIN-SEED-0002`). `--reset` deletes only the rows of tenants the seed made. A field it cannot generate raises `MAIN-SEED-0003` and names the `seed.models` key to set. Set `seed.ai` to have free-text fields written by your ai driver, cached under `.mainsdk/seed-cache/` and held to `seed.ai.limit`; past it, or when the driver fails, built-in text is used and `MAIN-SEED-0004` is logged once. `mainsdk preview up|down|list|prune` gives each git branch its own database: a copy of the sqlite file for `preview.driver: local`, a branch from `preview.from` for `neon` or `supabase`. `up` seeds it, keeps the connection as the secret `PREVIEW_DB_` and prints only that name, and returns the existing preview when you run it again. `down` deletes only a database `up` made (`MAIN-PREVIEW-0003`). `prune` deletes previews whose branch is gone or whose `preview.ttl` has passed. Every create and delete is written to the audit log. A db driver that cannot branch raises `MAIN-PREVIEW-0001`, and a provider failure deletes whatever it made and raises `MAIN-PREVIEW-0002`. `mainsdk check` reports `seed` or `preview` set for production, and app code that imports `mainsdk/seed` outside tests and scripts (`MAIN-SEED-0010`), and a literal under `seed.models` that looks like real personal data (`MAIN-SEED-0011`). ## Reference | Call | What it does | | --- | --- | | `clock.advance(by)` | Moves the test clock forward by a duration such as `"2h"` or by milliseconds, firing the timers that fall due. It does not wait for the work a timer starts; use `clock.tick()` for that. | | `new App(opts?)` | The class `createTestApp()` builds on: one running app from a `main.yaml`. Use `createTestApp()` in tests, and `new App()` only to wire a custom app. | ## Simulate an afternoon `createSim` builds several apps on one fake clock and one seed, injects load, outages and key events through the SDK's own fakes, and returns a timeline that is the same on every run (spec intelligence-sim). ```ts import { createSim, ramp, step } from "@mainsdk/testing"; const sim = createSim({ apps: { web: { main: "./main.yaml" } }, start: "2026-10-27T00:00:00Z", seed: 42 }); sim.inject.loop("web", step({ from: 20, to: 300, at: "10m" })); // event-loop p99 in ms sim.inject.cpu("web", ramp({ from: 0.2, to: 0.95, over: "10m" })); // share of the allowed cores sim.inject.traffic("web", { rps: sim.curve.noise(sim.curve.constant(5), { sd: 1 }), errors: sim.curve.constant(0.02) }); sim.inject.dependency("web", "db", { down: "2m", at: "1m" }); await sim.run("20m", { step: "1s" }); // 24 hours takes seconds sim.expect.signal("web", "pressure.high").within("12m"); sim.report({ golden: "fixtures/sim/pressure.json" }); // UPDATE_GOLDEN=1 rewrites it ``` - Curves are `constant`, `ramp`, `step`, `spike`, `sine`, `noise` and `replay` (a CSV of seconds and values). A curve reads milliseconds since its injection began. - A sim never emits a built-in signal: `pressure.high` in the timeline comes from the resources module. The app's `main.yaml` needs a `signals`, `norms` or `playbooks` section for the signal layer to run. - `inject.attack` and `inject.key` throw `MAIN-SIM-0001` naming the spec that has no code yet (security-events, secret-rotation). `MAIN-SIM-0002` is a bad input. - `host: "sockets"` runs the real host bus over the fake unix sockets in a temporary folder that `sim.stop()` removes (`sim.home`, `sim.net`). `link: true` on an app seals it and opens its app link to the other linked apps, so `neighbours.ask` works between them. `connectOps` is an MCP client for an app's ops socket. - `sim.timeline()`, `sim.report()`, `sim.approve(runId)` and `sim.decline(runId)` read and drive what the apps decided. `pnpm e2e --only intelligence` plays the connected-intelligence scenarios on the probe apps. ## Planned These are in spec `testing-kit` and not built. Until they are, assert on what the interface recorded: `app.sent`, `app.log.auditRecords`, `fakeAiDriver().requests`. - Assertions for `webhooks`, `jobs` and `files` beside `toHaveSent` and `toHaveCharged`. - The conformance suite for interfaces other than `email@1` and `files@1`, and `behaviour/`, `http/` and `check/` parity vectors (the seal wire format too). - The testing kit in Rust, Python and Go: each language SDK ships its own (milestone 1.5). ## Use a different app The module-level interfaces use one app built from `main.yaml` on first use. `app()` returns it, `createApp(options)` builds another and `configure(app)` makes it current. `createTestApp()` does this for you. ```ts title="test/setup.ts" import { app, configure, createApp } from "mainsdk"; configure(createApp({ yaml: "schema: 1" })); const current = app(); ``` ## Errors and checks The kit has no check codes of its own; seed and preview add `MAIN-SEED-0010` and `0011` to `mainsdk check`. It raises `MAIN-TESTING-0001` to `0005` (a missing fake, a bad chaos target, a driver that cannot be tested, a broken runner protocol, a kit or schema that is not installed) and surfaces the codes of the interfaces under test, such as `MAIN-NET-0002`, `MAIN-DATA-0054` and `MAIN-TENANT-0005`. See [Errors](https://mainsdk.dev/docs/reference/errors). ## Swapping drivers Your tests do not change when your driver does: the fakes implement the interface contract, not a provider. Some interfaces ship a conformance helper for drivers, such as `protectConformance` for `protect`; `mainsdk driver test` runs the generated suite for `email@1` and `files@1`. ## Testing The kit is what you test with, so the Quick start is the pattern: call `createTestApp()` with your `main.yaml` or an inline `yaml`, run your code, then assert on `app.sent`, `app.log.auditRecords` or the helper for the interface. Control time with `clock.advance` instead of waiting. See [Test your app](https://mainsdk.dev/docs/guides/testing). ## Security and privacy - Fakes need no network and no provider keys, so no secret enters your test environment. - The kit is free and open source, like every in-app capability. ## FAQ **Do fakes behave like production?** They run the same interface code with in-memory drivers, so the same checks apply. Provider-specific behaviour is what driver conformance tests check. **Does the app need a real database?** No. `createTestApp()` uses in-memory stores. ## Next - [Testing guide](https://mainsdk.dev/docs/guides/testing): set up tests across a whole app. - [Actions](https://mainsdk.dev/docs/interfaces/action): test an action through its audit event.