Events and background jobs

events and jobs move work out of the request: publish CloudEvents, queue jobs that retry, survive restarts and dead-letter, and run every side effect once per key.

jobs:
  driver: db
  dead-letter: true
idempotency: { store: db, ttl: 24h }
events:
  broadcast: { driver: redis, redis: { url: secret:REDIS_URL }, max-payload: 64KB }
  schemas:
    order.shipped: { version: 1, compat: backward, fields: { orderId: string, carrier: "string?" } }

import { events, jobs } from "mainsdk";

await events.publish("order.shipped", { orderId: order.id });
await jobs.schedule("reports.nightly", "0 3 * * *", { region: "in" });

import { describe, it } from "node:test";
import assert from "node:assert/strict";
import { createTestApp } from "@mainsdk/testing";

async function pass(app: ReturnType<typeof createTestApp>, ms: number) {
  for (let t = 0; t < ms; t += 500) await app.clock.tick(500);
}

describe("report export", () => {
  it("webhooks-jobs AC5 retries a job that always throws, then dead-letters it", async () => {
    const app = createTestApp();
    let tries = 0;
    app.jobs.on("reports.export", () => {
      tries++;
      throw new Error("always fails");
    });
    const { id } = await app.jobs.run("reports.export", { month: "2026-10" }, { retries: 3 });
    await pass(app, 30_000);
    assert.equal(tries, 4);
    const dead = await app.jobs.dead.list();
    assert.equal(dead.items[0]?.id, id);
    assert.equal(dead.items[0]?.error?.code, "MAIN-JOB-0002");
  });

  it("webhooks-jobs AC6 applies the side effect once when an attempt is retried", async () => {
    const app = createTestApp();
    let sent = 0;
    let attempts = 0;
    app.jobs.on("receipt.send", async () => {
      attempts++;
      await app.idempotency.once("email.send", { to: "a@pinecart.dev" }, () => void sent++);
      if (attempts === 1) throw new Error("crashed before the acknowledgement");
    });
    await app.jobs.run("receipt.send", { orderId: "o1" });
    await pass(app, 10_000);
    assert.equal(attempts, 2);
    assert.equal(sent, 1);
  });
});

jobs:
  driver: db

Quick start

jobs:
  driver: db          # the app's own database; `local` for development
  dead-letter: true

import { jobs } from "mainsdk";

declare function exportReport(input: { userId: string; month: string }, attempt: number): Promise<void>;

jobs.on("reports.export", async (input: { userId: string; month: string }, ctx) => {
  await exportReport(input, ctx.attempt);   // your code; a throw fails the attempt and it retries
});

await jobs.run("reports.export", { userId: "u_1", month: "2026-10" });

jobs.run() writes one record and returns { id, deduped }. It never waits for the worker. Register handlers in a module your app imports at start, because only an instance that has called jobs.on() for a name runs that job.

Configuration

jobs:
  driver: db               # db (the app's database) | local (in process)
  dead-letter: true        # keep jobs that exhausted their retries in the dead-letter queue
idempotency:
  store: db                # db | local; defaults to the jobs driver
  ttl: 24h                 # how long a key is remembered
events:
  broadcast: { driver: redis, redis: { url: secret:REDIS_URL }, max-payload: 64KB }   # memory | redis

jobs.concurrency, jobs.lease, jobs.poll and jobs.keep are tuning keys the runtime reads. mainsdk check does not list them as main.yaml keys yet and reports them as MAIN-CONFIG-0030 (low) until the schema does.

Per-job settings go on the call: jobs.run(name, input, { idempotencyKey, retries, backoff, delay, at }).

A job input must be JSON and at most 256 KB. A job name is lowercase letters, digits, dots, dashes and colons.

Overlays by environment

mainsdk check --env prod fails with MAIN-CHECK-0026 if jobs.driver is local. Keep it in a development overlay:

jobs:
  driver: local   # in-process queue

What you get

events.publish() sends a CloudEvents message to every events.on() handler. jobs.run() queues work for a handler you register with jobs.on(); the queue retries it with backoff and moves it to a dead-letter queue after the last retry. jobs.schedule() runs a job on a cron schedule. Every call that changes the world takes an idempotency key, so a retry or a crashed worker never charges, sends or writes twice. Reach for it when a request does something slow, or when several parts of your app care that something happened.

Idempotency

Every call that changes the world takes an idempotency key. By default it is made from the action and its inputs, so you only pass your own when the natural key is clearer, such as export:u_1:2026-10. One record serves jobs, events, webhooks and workflows. Every key includes the tenant, so one tenant's key never returns another tenant's result. A driver without native idempotency gets it from this record, so a duplicate key stays a duplicate when you swap drivers.

Instance broadcast

When one instance of your app changes something the others keep in memory, tell them. events.broadcast() sends a small message to every running instance of the same app, and events.onBroadcast() listens for it. Use it for cache invalidation and "config changed" nudges.

import { events, instance } from "mainsdk";

declare const productCache: Map<string, unknown>;   // your own in-process cache

// on every instance, including the sender
const stop = events.onBroadcast("cache.invalidate", (payload, meta) => {
  if (meta.from === instance.id) return;          // skip the message this instance sent
  productCache.delete((payload as { key: string }).key);
});

// on the instance that changed the product
const { reached, dropped } = await events.broadcast("cache.invalidate", { key: "product:42" });

events:
  broadcast: { driver: redis, redis: { url: secret:REDIS_URL }, max-payload: 64KB }   # memory | redis

Broadcast is best effort and at most once. An instance that is down when a message is sent never receives it, and nothing is stored or retried. Use events.publish() or jobs for work that must happen.

• events.broadcast() resolves with { id, reached, dropped } once the driver accepts the message or 250 ms pass. It never throws because the driver is down; it resolves with dropped: true and logs once per outage.
• Topics match ^[a-z0-9][a-z0-9._-]{0,127}$. The payload must be JSON and at most max-payload (64KB by default, 1MB at most).
• A handler that throws is logged, and the other handlers still run. meta holds { id, topic, from, sentAt }.
• memory delivers within one process, for tests and mainsdk dev. In production, redis is required; mainsdk check --env prod fails otherwise (MAIN-EVENT-0011).

Reference

The classes and helpers behind events and jobs. mainsdk builds the classes for you from main.yaml, so most apps only call events.* and jobs.*.

import { nextCron, parseCron } from "mainsdk/jobs";

const next = nextCron(parseCron("0 3 * * *"), Date.now());   // the next 03:00 UTC

Examples

Publish an event and react to it

Events use the CloudEvents 1.0 envelope, with your app's source and the tenant as tenantid. events.on() takes an exact type, a prefix such as order.*, or *. Handlers run outside the publisher's call, and one that throws is logged without stopping the others.

"use server";
import { auth, events, models, s } from "mainsdk";

const { orders } = models({
  orders: s.object({ item: s.string(), total: s.int(), status: s.string().optional() }),
});   // declared once in lib/orders.ts, shown inline so the example stands alone

export async function markShipped(req: Request, orderId: string) {
  const session = await auth.require(req);
  await orders.for(session.user).update(orderId, { status: "shipped" });
  await events.publish(
    "order.shipped",
    { orderId, userId: session.user },
    { idempotencyKey: `shipped:${orderId}` },
  );
}

import { events } from "mainsdk";

declare function sendShippedNotice(userId: string, orderId: string): Promise<void>;

events.on("order.shipped", async (event) => {
  const { orderId, userId } = event.data as { orderId: string; userId: string };
  await sendShippedNotice(userId, orderId);   // your code, ideally a call that takes an idempotency key
});

Event types are lowercase words joined by dots; anything else fails with MAIN-EVENT-0001. The same idempotency key, or the same type and data when you pass none, publishes once and returns the first event.

Run a job with retries

import { auth, jobs } from "mainsdk";

export async function POST(req: Request) {
  const session = await auth.require(req);
  const { month } = (await req.json()) as { month: string };
  const { id, deduped } = await jobs.run(
    "reports.export",
    { userId: session.user, month },
    { idempotencyKey: `export:${session.user}:${month}`, retries: 3, backoff: "exponential" },
  );
  return Response.json({ id, deduped }, { status: 202 });
}

A job that always throws with retries: 3 is tried 4 times with growing delays, then moves to the dead-letter queue. Read a job with jobs.get(id); it returns { id, name, state, attempts, maxAttempts, runAt, traceId, input, error? }, where state is queued, running, done, dead or failed.

Apply a side effect once

A handler runs inside a context key for its job. Calls it makes through MainSDK use that key, so an attempt that runs again finds the first attempt's record and does not repeat the effect. Wrap your own side effects the same way with idempotency.once().

import { idempotency, jobs } from "mainsdk";

declare function sendReceipt(input: { orderId: string; to: string }): Promise<void>;

jobs.on("receipt.send", async (input: { orderId: string; to: string }) => {
  await idempotency.once("email.send", { to: input.to, orderId: input.orderId }, () =>
    sendReceipt(input),   // your code; runs once per key even if a retry reaches this line again
  );
});

idempotency.once(action, input, fn, { key?, ttlMs? }) returns the first result for the key. A repeat that arrives while the first is running waits for it. If fn throws the record is removed, so a retry runs again. idempotency.key(action, input) returns the default key, a stable hash.

Schedule a nightly job

import { jobs } from "mainsdk";

jobs.schedule("reports.nightly", "0 3 * * *", { region: "in" });   // 03:00 UTC every day

The schedule is a five-field cron expression in UTC with numbers, *, ranges, lists and steps; names such as mon are not accepted, and a bad expression fails with MAIN-JOB-0004. Every instance may call schedule; each firing uses one idempotency key, so the job is queued once however many instances run. It returns a function that stops the schedule.

Dead letters

A job that fails its last attempt moves to the dead-letter queue instead of disappearing, with the error code and message of the last failure.

import { jobs } from "mainsdk";

const { items } = await jobs.dead.list({ limit: 20 });          // oldest first, current tenant
for (const job of items) await jobs.dead.retry(job.id);          // a fresh set of attempts

jobs.dead.retry(id) puts the job back in the queue; an id that is not a dead job in the current tenant fails with MAIN-JOB-0005.

Errors and checks

Errors are MainError values that serialise as RFC 9457 problem details with a code, a fix and a docs link.

A job that fails every retry is not an error to your caller: it moves to the dead-letter queue.

Swapping drivers

Change one line. Your events.publish(), events.on(), jobs.on(), jobs.run() and jobs.schedule() calls stay the same.

jobs:
  driver: db   # was local

The idempotency record is MainSDK's own, so a duplicate key stays a duplicate across the swap. What happens to jobs already queued on the old driver is not decided yet, so drain the queue before you swap.

Testing

createTestApp runs the queue in memory on a fake clock. Step the clock in small pieces so the timer each retry arms fires in order.

import { describe, it } from "node:test";
import assert from "node:assert/strict";
import { createTestApp } from "@mainsdk/testing";

async function pass(app: ReturnType<typeof createTestApp>, ms: number) {
  for (let t = 0; t < ms; t += 500) await app.clock.tick(500);
}

describe("report export", () => {
  it("webhooks-jobs AC5 retries a job that always throws, then dead-letters it", async () => {
    const app = createTestApp();
    let tries = 0;
    app.jobs.on("reports.export", () => {
      tries++;
      throw new Error("always fails");
    });
    const { id } = await app.jobs.run("reports.export", { month: "2026-10" }, { retries: 3 });
    await pass(app, 30_000);
    assert.equal(tries, 4);
    const dead = await app.jobs.dead.list();
    assert.equal(dead.items[0]?.id, id);
    assert.equal(dead.items[0]?.error?.code, "MAIN-JOB-0002");
  });

  it("webhooks-jobs AC6 applies the side effect once when an attempt is retried", async () => {
    const app = createTestApp();
    let sent = 0;
    let attempts = 0;
    app.jobs.on("receipt.send", async () => {
      attempts++;
      await app.idempotency.once("email.send", { to: "a@pinecart.dev" }, () => void sent++);
      if (attempts === 1) throw new Error("crashed before the acknowledgement");
    });
    await app.jobs.run("receipt.send", { orderId: "o1" });
    await pass(app, 10_000);
    assert.equal(attempts, 2);
    assert.equal(sent, 1);
  });
});

jobs.idle() (on app.jobs) waits until nothing is running or due, which helps in a graceful shutdown.

Security and privacy

• Jobs carry their producer's trace and tenant, so audit and logs follow the work into the background. A job runs inside its tenant.
• Delivery is at least once with idempotent effects; exactly-once delivery across providers is not promised.
• Job inputs are stored in the queue, which is your database with the db driver; keep personal data out of inputs where an ID will do.
• While the instance is under resource pressure, the worker stops claiming new attempts except for jobs named in resources.critical.

FAQ

Events or jobs? Publish an event when something happened and any number of parts of your app may react. Run a job when you want one specific piece of work done, with retries.

What if I don't pass an idempotency key? One is made from the action and its inputs. Pass your own when the natural key is clearer.

Do I need a separate worker process? No. Every instance that registers handlers with jobs.on() works the queue. With the db driver they share it and a crashed instance's job is taken over after lease.

Are Inngest, Vercel Queues, Trigger.dev and Amazon SQS supported? Not yet. Inngest and Vercel Queues are planned; the spec leaves Trigger.dev and Amazon SQS to the driver kit, later. A Node worker adapter is planned too.

What is not decided yet? What happens to jobs already queued on the old driver when you swap.

Next

• Webhooks: hand slow webhook work to a job.
• Durable workflows: several steps, waits and approvals in one function.
• Run background jobs: the same calls as a walkthrough.
• Logs, audit and telemetry: follow a job through its trace.