Performance budgets

MainSDK sits on every call your app makes, so its speed is a published, tested number, not a claim. Four budgets hold in normal mode and in degraded mode, and a release that breaks one fails (ADR-0003).

The four budgets

p50 and p99 are both recorded. Every number is measured on the pure TypeScript core (ADR-0066): the open harness and the budget gate come from spec fail-static and the browser size gate from spec portable-core, both at 0.4.

Normal and degraded mode

flowchart LR
  subgraph normal["Normal mode (0.4)"]
    n1["decide"]
    n2["overhead"]
    n3["browser size"]
    n4["cold start"]
  end
  subgraph degraded["Degraded mode (0.4)"]
    d0["Cloud blocked +<br/>TUF cache stale +<br/>telemetry buffer full"]
    d1["100,000 decisions: p99 under 1 ms"]
    d2["10,000 sends on a 50 ms fake: under 2 ms added"]
    d3["Cold start under 20 ms,<br/>no wait on TUF or Cloud"]
    d0 --> d1 & d2 & d3
  end
  normal --> gate{"tools/check stage"}
  degraded --> gate
  gate -- regression --> fail["Fail: names budget,<br/>measured value, baseline"]
  gate -- pass --> rel["Results file attached to the release"]

The degraded scenarios prove the budgets hold when our services are down. With every MainSDK host blocked, a sample Next.js app serves 1,000 requests with the same responses and p99 latency within 5% of the unblocked run. With the Cloud unreachable, writing 50,000 log records never takes longer than 0.1 ms per write at p99.

How a regression is caught

$ tools/check stage
budget exceeded: decide p99

• The harness runs in CI on every release candidate.
• A regression past any limit, in normal or degraded mode, fails tools/check stage and the release.
• The browser size check fails the build at 150 KB or more, and reports the Cedar engine chunk on its own line.
• Every release carries a results file with all four budgets, normal and degraded, and a link to the harness.

Reproduce a number yourself

The benchmark method and harness are open source. Results are written as JSON.

pnpm --filter mainsdk bench -- --suite decide --json
pnpm --filter mainsdk bench -- --suite overhead,coldstart --json
pnpm --filter mainsdk size

One engine or one driver

pnpm --filter @mainsdk/testing bench --engine cedar --json
pnpm --filter @mainsdk/testing bench --driver resend --json --region eu

Each run prints one BenchResult: the subject and its version, the region (us, eu or in), the hardware line, the release, p50 and p99 in milliseconds, throughput, memory, cold start and, for a driver, overheadP99Ms. The settings are fixed so a number can be reproduced: the engine decides over a reference set of 200 policies and 10,000 seeded inputs after a 2,000-call warm-up; a driver is called 10,000 times with 200 calls in flight against a recorded provider stub that answers after 50 ms, and the overhead is the p99 of email.send minus the p99 of posting the same body straight to the stub in the same run. The hardware line comes from the machine that ran it (BENCH_HARDWARE overrides it for a named reference machine), and BENCH_REGION and BENCH_RELEASE set the region and release.

On every release the pipeline runs every engine and driver from each of the three regions on a fixed reference machine, fails when any cell is missing, and fails the budget job when a result breaks a budget, with a line such as policy.decide p99 1.4 ms > 1 ms. The public performance pages and the status page are built from those results and from health checks, and the status page is hosted apart from MainSDK Cloud.

Adapters and drivers

Framework adapters (Next.js, Hono, Express, Node workers) stay within the same overhead and cold-start budgets. A driver's own provider latency is not MainSDK overhead; the overhead budget measures only what we add on top.

Later

Next

• Core and bindings: what the decide and cold-start numbers measure.
• Request lifecycle: the path the overhead budget covers.
• Fail static: why the budgets must hold when our services are down.