---
title: "Events and background jobs"
description: "Publish CloudEvents, run background jobs that retry with backoff and land in a dead-letter queue, and apply every side effect once per idempotency key."
section: "interfaces"
status: "Building for 0.3 · ships as a stable API"
spec: "webhooks-jobs"
canonical: "https://mainsdk.dev/docs/interfaces/jobs"
updated: "2026-10-28"
---

# Events and background jobs

> Publish CloudEvents, run background jobs that retry with backoff and land in a dead-letter queue, and apply every side effect once per idempotency key.

Status: Building for 0.3 · ships as a stable API · Canonical: https://mainsdk.dev/docs/interfaces/jobs · Updated: 2026-10-28

`events` and `jobs` move work out of the request: publish CloudEvents, queue jobs that retry, survive restarts and dead-letter, and run every side effect once per key.

## At a glance

- Status: Building for 0.3 · ships as a stable API
- Phase: 1 (Core, public beta)
- Milestone: 0.3 Every interface
- Plan: free in your app (Apache-2.0)
- Spec: https://github.com/hubxero/mainsdk/blob/develop/docs/specs/webhooks-jobs.md
- Category: Events, jobs and webhooks

Calls:

- `events.publish()`
- `events.on()`
- `events.broadcast(topic, payload)`
- `events.onBroadcast(topic, fn)`
- `jobs.run()`
- `jobs.schedule()`
- `new Events(opts)`
- `typeMatches(pattern, type)`
- `new Jobs(opts)`
- `new WorkGate(pressure?)`
- `nextCron(c, after)`
- `parseCron(expr)`
- `jobs.on(name, handler)`
- `jobs.get(id)`
- `jobs.dead.list({ cursor?, limit? })`
- `jobs.dead.retry(id)`
- `jobs.idle()`

In main.yaml:

```yaml
jobs:
  driver: db
  dead-letter: true
idempotency: { store: db, ttl: 24h }
events:
  broadcast: { driver: redis, redis: { url: secret:REDIS_URL }, max-payload: 64KB }
  schemas:
    order.shipped: { version: 1, compat: backward, fields: { orderId: string, carrier: "string?" } }
```

Guarantees, whatever the driver:

- CloudEvents envelope with versioned schemas
- Retries with backoff and a dead-letter queue
- A duplicate key never runs the side effect twice
- Broadcast is best effort between instances of one app, and never blocks a request


## Drivers

Pick one in main.yaml with `jobs.driver`.

| Driver | Id in main.yaml | Default | Note |
| --- | --- | --- | --- |
| The app's database | `db` | yes |  |
| In-process local queue | `local` |  |  |
| Inngest | `inngest` |  |  |
| Vercel Queues | `vercel-queues` |  |  |
| Trigger.dev | `trigger-dev` |  | planned driver |
| Amazon SQS | `sqs` |  | planned driver |

## Codes

| Code | Where | Meaning |
| --- | --- | --- |
| `MAIN-EVENT-0004` | check | A breaking event schema change under the same version |
| `MAIN-EVENT-0010` | check | A broadcast payload is larger than `max-payload` (runtime), or `max-payload` is over 1MB (check) |
| `MAIN-EVENT-0011` | check | `events.broadcast` uses the `local` driver in production, so other instances never receive it |
| `MAIN-EVENT-0012` | runtime | A broadcast topic doesn't match `^[a-z0-9][a-z0-9._-]{0,127}$`, or the payload isn't JSON |
| `MAIN-EVENT-0013` | runtime | A broadcast was dropped because the driver is unreachable; the call resolves with `dropped: true` and logs once per outage |
| `MAIN-EVENT-0001` | runtime |  |
| `MAIN-EVENT-0014` | runtime |  |
| `MAIN-IDEM-0001` | runtime |  |
| `MAIN-IDEM-0002` | runtime |  |
| `MAIN-JOB-0001` | runtime |  |
| `MAIN-JOB-0002` | runtime |  |
| `MAIN-JOB-0003` | runtime |  |
| `MAIN-JOB-0004` | runtime |  |
| `MAIN-JOB-0005` | runtime |  |
| `MAIN-JOB-0006` | runtime |  |

## Features

| Feature | Phase | Milestone | Spec ticket |
| --- | --- | --- | --- |
| events@1 and jobs@1 contracts | 1 | 0.3 | `webhooks-jobs#contract` |
| Shared idempotency record | 1 | 0.3 | `webhooks-jobs#idempotency-store` |
| Publish and subscribe with a schema registry | 1 | 0.3 | `webhooks-jobs#events` |
| Run and schedule with retries and dead letters | 1 | 0.3 | `webhooks-jobs#jobs` |
| Inngest, Vercel Queues and local queue drivers | 1 | 0.3 | `webhooks-jobs#drivers-jobs` |
| Node worker adapter | 1 | 0.2 | `adapters#worker-adapter` |
| Instance broadcast on the memory driver, with topic and payload limits | 1 | 0.3 | `webhooks-jobs#broadcast-api` |
| Instance broadcast on the redis driver | 1 | 0.3 | `webhooks-jobs#broadcast-redis` |
| Production check for the memory broadcast driver | 1 | 0.3 | `webhooks-jobs#broadcast-prod-check` |
| Schedules on serverless platforms through the signed tick route | 1 | 0.4 | `deploy#tick` |

## MainSDK Cloud

Everything on this page is free in your app.


## Quick start

```yaml title="main.yaml"
jobs:
  driver: db          # the app's own database; `local` for development
  dead-letter: true
```

```ts title="app/jobs/reports.ts"
import { jobs } from "mainsdk";

declare function exportReport(input: { userId: string; month: string }, attempt: number): Promise<void>;

jobs.on("reports.export", async (input: { userId: string; month: string }, ctx) => {
  await exportReport(input, ctx.attempt);   // your code; a throw fails the attempt and it retries
});

await jobs.run("reports.export", { userId: "u_1", month: "2026-10" });
```

`jobs.run()` writes one record and returns `{ id, deduped }`. It never waits for the worker. Register handlers in a module your app imports at start, because only an instance that has called `jobs.on()` for a name runs that job.

## Configuration

```yaml title="main.yaml"
jobs:
  driver: db               # db (the app's database) | local (in process)
  dead-letter: true        # keep jobs that exhausted their retries in the dead-letter queue
idempotency:
  store: db                # db | local; defaults to the jobs driver
  ttl: 24h                 # how long a key is remembered
events:
  broadcast: { driver: redis, redis: { url: secret:REDIS_URL }, max-payload: 64KB }   # memory | redis
```

| Key | Type | What it does |
| --- | --- | --- |
| `jobs.driver` | string | Where the queue lives. `db` uses your app's database and is shared by every instance. `local` keeps it in process. Without a `db` section the default is `local` |
| `jobs.dead-letter` | boolean | Keep a job that failed its last attempt as `dead` (default `true`). When `false` it is kept as `failed` |
| `jobs.concurrency` | integer | Attempts one process runs at once (default 4) |
| `jobs.lease` | duration | How long a running attempt is held before another instance takes it over (default 30s) |
| `jobs.poll` | duration | Poll interval on a shared queue (default 1s) |
| `jobs.keep` | duration | How long finished jobs are kept (default 24h); dead jobs are kept 30 days |
| `idempotency.store` | string | Where the idempotency record shared with `webhooks`, `workflows`, `pay`, `email` and `files` lives |
| `idempotency.ttl` | duration | How long a key is kept (default 24h) |

`jobs.concurrency`, `jobs.lease`, `jobs.poll` and `jobs.keep` are tuning keys the runtime reads. `mainsdk check` does not list them as main.yaml keys yet and reports them as `MAIN-CONFIG-0030` (low) until the schema does.

Per-job settings go on the call: `jobs.run(name, input, { idempotencyKey, retries, backoff, delay, at })`.

| Option | Default | What it does |
| --- | --- | --- |
| `idempotencyKey` | hash of name and input | The same key returns the first job instead of queueing another |
| `retries` | `5` | Retries after the first attempt, 0 to 50. `retries: 3` means 4 attempts |
| `backoff` | `"exponential"` | `"exponential"` waits about 1s, 2s, 4s and so on, capped at 5 minutes, with 20% jitter. `"fixed"` waits 1s |
| `delay` | none | Wait this long before the first attempt, such as `"10m"` |
| `at` | none | Run at this time: epoch milliseconds, a `Date` or an RFC 3339 string |

A job input must be JSON and at most 256 KB. A job name is lowercase letters, digits, dots, dashes and colons.

### Overlays by environment

`mainsdk check --env prod` fails with `MAIN-CHECK-0026` if `jobs.driver` is `local`. Keep it in a development overlay:

```yaml title="main.env.dev.yaml"
jobs:
  driver: local   # in-process queue
```

> **Note:** `inngest` and `vercel-queues` are planned job drivers (spec `webhooks-jobs`). Today `db` and `local` are built.

## What you get

`events.publish()` sends a CloudEvents message to every `events.on()` handler. `jobs.run()` queues work for a handler you register with `jobs.on()`; the queue retries it with backoff and moves it to a dead-letter queue after the last retry. `jobs.schedule()` runs a job on a cron schedule. Every call that changes the world takes an idempotency key, so a retry or a crashed worker never charges, sends or writes twice. Reach for it when a request does something slow, or when several parts of your app care that something happened.

### Idempotency

Every call that changes the world takes an idempotency key. By default it is made from the action and its inputs, so you only pass your own when the natural key is clearer, such as `export:u_1:2026-10`. One record serves `jobs`, `events`, `webhooks` and `workflows`. Every key includes the tenant, so one tenant's key never returns another tenant's result. A driver without native idempotency gets it from this record, so a duplicate key stays a duplicate when you swap drivers.

### Instance broadcast

When one instance of your app changes something the others keep in memory, tell them. `events.broadcast()` sends a small message to every running instance of the same app, and `events.onBroadcast()` listens for it. Use it for cache invalidation and "config changed" nudges.

```ts title="app/lib/cache.ts"
import { events, instance } from "mainsdk";

declare const productCache: Map<string, unknown>;   // your own in-process cache

// on every instance, including the sender
const stop = events.onBroadcast("cache.invalidate", (payload, meta) => {
  if (meta.from === instance.id) return;          // skip the message this instance sent
  productCache.delete((payload as { key: string }).key);
});

// on the instance that changed the product
const { reached, dropped } = await events.broadcast("cache.invalidate", { key: "product:42" });
```

```yaml title="main.yaml"
events:
  broadcast: { driver: redis, redis: { url: secret:REDIS_URL }, max-payload: 64KB }   # memory | redis
```

Broadcast is best effort and at most once. An instance that is down when a message is sent never receives it, and nothing is stored or retried. Use `events.publish()` or `jobs` for work that must happen.

- `events.broadcast()` resolves with `{ id, reached, dropped }` once the driver accepts the message or 250 ms pass. It never throws because the driver is down; it resolves with `dropped: true` and logs once per outage.
- Topics match `^[a-z0-9][a-z0-9._-]{0,127}$`. The payload must be JSON and at most `max-payload` (64KB by default, 1MB at most).
- A handler that throws is logged, and the other handlers still run. `meta` holds `{ id, topic, from, sentAt }`.
- `memory` delivers within one process, for tests and `mainsdk dev`. In production, `redis` is required; `mainsdk check --env prod` fails otherwise (`MAIN-EVENT-0011`).

> **Note:** Spec `webhooks-jobs` renames the in-process driver from `memory` to `local` (ADR-0039). The code still accepts `memory`, so write `memory` today.

### Reference

The classes and helpers behind `events` and `jobs`. `mainsdk` builds the classes for you from `main.yaml`, so most apps only call `events.*` and `jobs.*`.

| Call | What it does |
| --- | --- |
| `new Events(opts)` | The class behind `events.publish`, `events.on` and the inherited `events.broadcast` and `events.onBroadcast`. `opts` takes the tenancy, an idempotency store, a log, the CloudEvents `source` that names the app, and the broadcast driver, instance id, timers, clock and `maxPayload` |
| `typeMatches(pattern, type)` | True when a handler's pattern matches an event type: the exact type, `*` for all, or a prefix ending in `.*` such as `order.*` |
| `new Jobs(opts)` | The background queue behind `jobs.*`: enqueue, retry with backoff, dead-letter and schedule. `opts` takes the store, a `local` or `db` driver, the tenancy, log, timers, clock, an idempotency store and a `WorkGate`. It runs 4 attempts at once, holds a running attempt for 30 seconds before another worker may take it over, keeps finished jobs for 24 hours and dead jobs for 30 days unless you set otherwise |
| `new WorkGate(pressure?)` | Decides whether a worker may claim a unit of background work now. With no pressure source it always says yes. Under resource pressure it admits only what the watcher allows (jobs named in `resources.critical`); work that has waited past `max-pause` runs alone, one unit at a time. `admit(kind, name, dueAt, key)` asks, `release(key)` frees the single slot and `onChange(fn)` calls `fn` when the pressure state changes |
| `parseCron(expr)` | Read a five-field cron expression (minute, hour, day of month, month, day of week) in UTC into the values each field allows. It throws `MAIN-JOB-0004` for anything else |
| `nextCron(c, after)` | The first fire time of a parsed expression strictly after `after`, in epoch milliseconds, or `null` when none falls within five years |

```ts title="lib/next-run.ts"
import { nextCron, parseCron } from "mainsdk/jobs";

const next = nextCron(parseCron("0 3 * * *"), Date.now());   // the next 03:00 UTC
```

## Examples

### Publish an event and react to it

Events use the CloudEvents 1.0 envelope, with your app's source and the tenant as `tenantid`. `events.on()` takes an exact type, a prefix such as `order.*`, or `*`. Handlers run outside the publisher's call, and one that throws is logged without stopping the others.

```ts title="app/orders/ship/actions.ts"
"use server";
import { auth, events, models, s } from "mainsdk";

const { orders } = models({
  orders: s.object({ item: s.string(), total: s.int(), status: s.string().optional() }),
});   // declared once in lib/orders.ts, shown inline so the example stands alone

export async function markShipped(req: Request, orderId: string) {
  const session = await auth.require(req);
  await orders.for(session.user).update(orderId, { status: "shipped" });
  await events.publish(
    "order.shipped",
    { orderId, userId: session.user },
    { idempotencyKey: `shipped:${orderId}` },
  );
}
```

```ts title="app/events/order-shipped.ts"
import { events } from "mainsdk";

declare function sendShippedNotice(userId: string, orderId: string): Promise<void>;

events.on("order.shipped", async (event) => {
  const { orderId, userId } = event.data as { orderId: string; userId: string };
  await sendShippedNotice(userId, orderId);   // your code, ideally a call that takes an idempotency key
});
```

Event types are lowercase words joined by dots; anything else fails with `MAIN-EVENT-0001`. The same idempotency key, or the same type and data when you pass none, publishes once and returns the first event.

> **Note:** `events.on()` handlers run in process and are not durable: if the process stops mid-handler, nothing retries it. Put work that must happen in a job, for example by calling `jobs.run()` from the handler.

### Run a job with retries

```ts title="app/api/reports/route.ts"
import { auth, jobs } from "mainsdk";

export async function POST(req: Request) {
  const session = await auth.require(req);
  const { month } = (await req.json()) as { month: string };
  const { id, deduped } = await jobs.run(
    "reports.export",
    { userId: session.user, month },
    { idempotencyKey: `export:${session.user}:${month}`, retries: 3, backoff: "exponential" },
  );
  return Response.json({ id, deduped }, { status: 202 });
}
```

A job that always throws with `retries: 3` is tried 4 times with growing delays, then moves to the dead-letter queue. Read a job with `jobs.get(id)`; it returns `{ id, name, state, attempts, maxAttempts, runAt, traceId, input, error? }`, where `state` is `queued`, `running`, `done`, `dead` or `failed`.

### Apply a side effect once

A handler runs inside a context key for its job. Calls it makes through MainSDK use that key, so an attempt that runs again finds the first attempt's record and does not repeat the effect. Wrap your own side effects the same way with `idempotency.once()`.

```ts title="app/jobs/receipt.ts"
import { idempotency, jobs } from "mainsdk";

declare function sendReceipt(input: { orderId: string; to: string }): Promise<void>;

jobs.on("receipt.send", async (input: { orderId: string; to: string }) => {
  await idempotency.once("email.send", { to: input.to, orderId: input.orderId }, () =>
    sendReceipt(input),   // your code; runs once per key even if a retry reaches this line again
  );
});
```

`idempotency.once(action, input, fn, { key?, ttlMs? })` returns the first result for the key. A repeat that arrives while the first is running waits for it. If `fn` throws the record is removed, so a retry runs again. `idempotency.key(action, input)` returns the default key, a stable hash.

### Schedule a nightly job

```ts title="app/jobs/schedules.ts"
import { jobs } from "mainsdk";

jobs.schedule("reports.nightly", "0 3 * * *", { region: "in" });   // 03:00 UTC every day
```

The schedule is a five-field cron expression in UTC with numbers, `*`, ranges, lists and steps; names such as `mon` are not accepted, and a bad expression fails with `MAIN-JOB-0004`. Every instance may call `schedule`; each firing uses one idempotency key, so the job is queued once however many instances run. It returns a function that stops the schedule.

### Dead letters

A job that fails its last attempt moves to the dead-letter queue instead of disappearing, with the error code and message of the last failure.

```ts title="app/admin/dead-jobs.ts"
import { jobs } from "mainsdk";

const { items } = await jobs.dead.list({ limit: 20 });          // oldest first, current tenant
for (const job of items) await jobs.dead.retry(job.id);          // a fresh set of attempts
```

`jobs.dead.retry(id)` puts the job back in the queue; an id that is not a dead job in the current tenant fails with `MAIN-JOB-0005`.

## Errors and checks

Errors are `MainError` values that serialise as RFC 9457 problem details with a `code`, a `fix` and a docs link.

| Code | When | Fix |
| --- | --- | --- |
| `MAIN-JOB-0001` | A job name, input, `retries`, `delay` or `at` is invalid, or the input is not JSON or is over 256 KB (400) | Use a lowercase name, JSON input, `retries` from 0 to 50, and store large data and pass its id |
| `MAIN-JOB-0002` | A job attempt failed (logged), or it failed its last attempt and moved to the dead-letter queue | Read the job with `jobs.get(id)`, fix the cause, then `jobs.dead.retry(id)` |
| `MAIN-JOB-0003` | A job ran with no handler registered for its name (the attempt fails and retries) | Call `jobs.on(name, handler)` in a module the app imports at start |
| `MAIN-JOB-0004` | A cron expression is not valid | Write five fields, such as `0 3 * * *` |
| `MAIN-JOB-0005` | No dead job with that id in the current tenant (404) | List the queue with `jobs.dead.list()` |
| `MAIN-JOB-0006` | The job queue could not be reached (503, retryable) | Check the database the jobs driver uses, then retry |
| `MAIN-IDEM-0001` | Another caller is still running the same idempotency key after about 30 seconds (409, retryable) | Retry shortly |
| `MAIN-EVENT-0001` | An event type is not lowercase words joined by dots, or the data is not JSON (400) | Use a type such as `order.shipped` and JSON data |
| `MAIN-CHECK-0026` | `jobs` resolves to the local queue in production (check) | Set a production driver in `main.env.prod.yaml` |
| `MAIN-EVENT-0010` | A broadcast payload is larger than `max-payload` (runtime), or `max-payload` is over 1MB (check) | Send less data, or keep large data in `events.publish` or `jobs` |
| `MAIN-EVENT-0011` | `events.broadcast` uses the `local` driver in production, so other instances never receive it (check, runtime) | Set `events.broadcast.driver: redis` in `main.env.prod.yaml` |
| `MAIN-EVENT-0012` | A broadcast topic doesn't match the topic pattern, or the payload isn't JSON (runtime) | Use a lowercase topic and send JSON |
| `MAIN-EVENT-0013` | A broadcast was dropped because the driver is unreachable (runtime, once per outage) | Restore the driver; broadcast is best effort, so keep anything that must happen on `events.publish` or `jobs` |

A job that fails every retry is not an error to your caller: it moves to the dead-letter queue.

> **Note:** Planned: a schema registry that versions each event type and fails `mainsdk check` with `MAIN-EVENT-0004` on a breaking change under the same version. Event types are not versioned in code yet.

## Swapping drivers

Change one line. Your `events.publish()`, `events.on()`, `jobs.on()`, `jobs.run()` and `jobs.schedule()` calls stay the same.

```yaml title="main.yaml"
jobs:
  driver: db   # was local
```

The idempotency record is MainSDK's own, so a duplicate key stays a duplicate across the swap. What happens to jobs already queued on the old driver is not decided yet, so drain the queue before you swap.

## Testing

`createTestApp` runs the queue in memory on a fake clock. Step the clock in small pieces so the timer each retry arms fires in order.

```ts title="app/jobs/reports.test.ts"
import { describe, it } from "node:test";
import assert from "node:assert/strict";
import { createTestApp } from "@mainsdk/testing";

async function pass(app: ReturnType<typeof createTestApp>, ms: number) {
  for (let t = 0; t < ms; t += 500) await app.clock.tick(500);
}

describe("report export", () => {
  it("webhooks-jobs AC5 retries a job that always throws, then dead-letters it", async () => {
    const app = createTestApp();
    let tries = 0;
    app.jobs.on("reports.export", () => {
      tries++;
      throw new Error("always fails");
    });
    const { id } = await app.jobs.run("reports.export", { month: "2026-10" }, { retries: 3 });
    await pass(app, 30_000);
    assert.equal(tries, 4);
    const dead = await app.jobs.dead.list();
    assert.equal(dead.items[0]?.id, id);
    assert.equal(dead.items[0]?.error?.code, "MAIN-JOB-0002");
  });

  it("webhooks-jobs AC6 applies the side effect once when an attempt is retried", async () => {
    const app = createTestApp();
    let sent = 0;
    let attempts = 0;
    app.jobs.on("receipt.send", async () => {
      attempts++;
      await app.idempotency.once("email.send", { to: "a@pinecart.dev" }, () => void sent++);
      if (attempts === 1) throw new Error("crashed before the acknowledgement");
    });
    await app.jobs.run("receipt.send", { orderId: "o1" });
    await pass(app, 10_000);
    assert.equal(attempts, 2);
    assert.equal(sent, 1);
  });
});
```

`jobs.idle()` (on `app.jobs`) waits until nothing is running or due, which helps in a graceful shutdown.

## Security and privacy

- Jobs carry their producer's trace and tenant, so audit and logs follow the work into the background. A job runs inside its tenant.
- Delivery is at least once with idempotent effects; exactly-once delivery across providers is not promised.
- Job inputs are stored in the queue, which is your database with the `db` driver; keep personal data out of inputs where an ID will do.
- While the instance is under resource pressure, the worker stops claiming new attempts except for jobs named in `resources.critical`.

## FAQ

**Events or jobs?** Publish an event when something happened and any number of parts of your app may react. Run a job when you want one specific piece of work done, with retries.

**What if I don't pass an idempotency key?** One is made from the action and its inputs. Pass your own when the natural key is clearer.

**Do I need a separate worker process?** No. Every instance that registers handlers with `jobs.on()` works the queue. With the `db` driver they share it and a crashed instance's job is taken over after `lease`.

**Are Inngest, Vercel Queues, Trigger.dev and Amazon SQS supported?** Not yet. Inngest and Vercel Queues are planned; the spec leaves Trigger.dev and Amazon SQS to the driver kit, later. A Node worker adapter is planned too.

**What is not decided yet?** What happens to jobs already queued on the old driver when you swap.

## Next

- [Webhooks](https://mainsdk.dev/docs/interfaces/webhooks): hand slow webhook work to a job.
- [Durable workflows](https://mainsdk.dev/docs/interfaces/workflows): several steps, waits and approvals in one function.
- [Run background jobs](https://mainsdk.dev/docs/guides/run-background-jobs): the same calls as a walkthrough.
- [Logs, audit and telemetry](https://mainsdk.dev/docs/interfaces/log): follow a job through its trace.