Run work in the background

Queue a job that retries with backoff, applies its side effect once, lands in a dead-letter queue when it keeps failing, and runs on a schedule.

Preview

Status Status is where the work is: Planned (in a milestone, not started), Building (tickets in progress), Preview (usable, may change), then the stability it ships at.

Stability The promise an interface ships with.

Stable Follows semantic versioning; breaking changes need a new major, 12 months of overlap and a codemod.
Beta Usable in production; may change before it becomes stable.
Experimental May change or go away; needs an explicit opt-in in main.yaml.

Stability levels

View as Markdown Open in Claude Edit

Updated 2026-10-28

You will move a slow task out of a request into a background job, make its side effect safe to retry, deal with a job that keeps failing, and run one on a schedule. It takes about 20 minutes.

You need

A MainSDK app with main.yaml and Node 22 or later.
For production, a db section so the queue lives in your database. Development runs the queue in process.

1. Declare the queue

main.yaml

jobs:
  driver: db          # the app's own database, shared by every instance
  dead-letter: true   # keep jobs that fail their last attempt
idempotency: { ttl: 24h }

main.env.dev.yaml

jobs:
  driver: local       # in-process queue for development

mainsdk check --env prod fails with MAIN-CHECK-0026 if jobs resolves to local, so the in-process queue cannot reach production by accident.

2. Register a handler and queue a job

A handler is code you register by name with jobs.on(). Register it in a module your app imports at start, because only an instance that has registered a name runs that job. A handler that throws fails the attempt and the queue retries it.

app/jobs/reports.ts

import { idempotency, jobs } from "mainsdk";

type ExportInput = { userId: string; month: string; email: string };
declare function sendReport(input: ExportInput): Promise<void>;

jobs.on("reports.export", async (input: ExportInput, ctx) => {
  console.log(`export ${input.month}, attempt ${ctx.attempt} of ${ctx.maxAttempts}`);
  await idempotency.once("email.send", { to: input.email, month: input.month }, () =>
    sendReport(input),   // your code; the side effect that must not happen twice
  );
});

Queue it from a route, an action or a script. jobs.run() writes one record and returns at once.

app/api/reports/route.ts

import { auth, jobs } from "mainsdk";
import "@/jobs/reports";

export async function POST(req: Request) {
  const session = await auth.require(req);
  const { month, email } = (await req.json()) as { month: string; email: string };
  const { id, deduped } = await jobs.run(
    "reports.export",
    { userId: session.user, month, email },
    { idempotencyKey: `export:${session.user}:${month}`, retries: 3 },
  );
  return Response.json({ id, deduped }, { status: 202 });
}

You should see the handler log attempt 1 of 4 (retries: 3 means up to four attempts). Calling jobs.run() again with the same idempotencyKey returns the same id with deduped: true and does not queue a second job. Check a job with await jobs.get(id); its state is queued, running, done, dead or failed.

3. Make the side effect safe to retry

A worker can crash after it sends the email and before it tells the queue. The retry then reaches the same line again. idempotency.once(action, input, fn) runs fn once for that key and returns the first result on a repeat, so the report is sent once. Calls you make through MainSDK, such as email.send, already carry a key made from the job and the call; wrap your own side effects with idempotency.once().

To see it, make the first attempt throw after the side effect. The log shows The job reports.export failed attempt 1 of 4; it will run again, a second attempt 2 of 4, and the send happens once. Retries wait about 1s, 2s, 4s and so on (backoff: "exponential", capped at 5 minutes); pass backoff: "fixed" for a flat 1s.

4. Deal with a job that keeps failing

A job that fails its last attempt moves to the dead-letter queue with the code and message of its last failure, and logs MAIN-JOB-0002.

scripts/dead-letters.ts

import { jobs } from "mainsdk";

const { items } = await jobs.dead.list();                    // oldest first, current tenant
console.log(items.map((j) => [j.name, j.state, j.attempts, j.error?.code]));
await jobs.dead.retry(items[0].id);                          // back in the queue with a fresh set of attempts

You should see [ 'reports.broken', 'dead', 2, 'MAIN-JOB-0002' ] for a job that failed both attempts, and jobs.dead.retry returns the job with state: "queued". Fix the cause first; a retried job that fails again goes back to the dead-letter queue.

5. Run a job on a schedule

app/jobs/schedules.ts

import { jobs } from "mainsdk";

declare function buildNightlyReport(region: string): Promise<void>;

jobs.on("reports.nightly", async (input: { region: string }) => {
  await buildNightlyReport(input.region);   // your code
});
jobs.schedule("reports.nightly", "0 3 * * *", { region: "in" });   // 03:00 UTC every day

The schedule is a five-field cron expression in UTC. Every instance may call schedule; each firing uses one idempotency key, so the job is queued once however many instances run.

6. Test it

createTestApp gives you the queue on a fake clock. Step the clock in small pieces so the timer each retry arms fires in order.

app/jobs/reports.test.ts

import { describe, it } from "node:test";
import assert from "node:assert/strict";
import { createTestApp } from "@mainsdk/testing";

async function pass(app: ReturnType<typeof createTestApp>, ms: number) {
  for (let t = 0; t < ms; t += 500) await app.clock.tick(500);
}

describe("report export", () => {
  it("webhooks-jobs AC6 applies the side effect once when an attempt is retried", async () => {
    const app = createTestApp();
    let sent = 0;
    let attempts = 0;
    app.jobs.on("reports.export", async () => {
      attempts++;
      await app.idempotency.once("email.send", { to: "a@pinecart.dev" }, () => void sent++);
      if (attempts === 1) throw new Error("crashed before the acknowledgement");
    });
    await app.jobs.run("reports.export", { month: "2026-10" });
    await pass(app, 10_000);
    assert.equal(attempts, 2);
    assert.equal(sent, 1);
  });

  it("webhooks-jobs AC5 dead-letters a job that always fails", async () => {
    const app = createTestApp();
    app.jobs.on("reports.export", () => {
      throw new Error("always fails");
    });
    await app.jobs.run("reports.export", { month: "2026-10" }, { retries: 3 });
    await pass(app, 30_000);
    const dead = await app.jobs.dead.list();
    assert.equal(dead.items.length, 1);
    assert.equal(dead.items[0]?.attempts, 4);
  });
});

Verify

bash

npx mainsdk check --env prod
node --test app/jobs/reports.test.ts

A pass prints mainsdk check: passed (prod) and both tests pass. With jobs.driver: local in main.yaml, the prod check fails with MAIN-CHECK-0026.

If it fails

Code	Cause	Fix
`MAIN-CHECK-0026`	`jobs` resolves to the local queue in production	Set `jobs.driver: db` in `main.yaml` and keep `local` in `main.env.dev.yaml`
`MAIN-JOB-0001`	The job name is not lowercase letters, digits, dots, dashes and colons, `retries` is outside 0 to 50, or the input is not JSON or is over 256 KB	Fix the name or `retries`; store large data and pass its id
`MAIN-JOB-0003`	A job ran with no handler registered for its name	Call `jobs.on(name, handler)` in a module the app imports at start
`MAIN-JOB-0004`	The cron expression is not five valid fields	Write it like `0 3 * * *`
`MAIN-JOB-0005`	`jobs.dead.retry` got an id that is not a dead job in the current tenant	List with `jobs.dead.list()`
`MAIN-JOB-0006`	The job queue could not be reached (503, retryable)	Check the database the `db` driver uses, then retry

For agents

text

Goal: move one slow task into a background job that is safe to retry.
Touch: main.yaml (jobs, idempotency), main.env.dev.yaml (jobs.driver: local), one module that calls jobs.on(), the caller that calls jobs.run(), one test.
Do: pass an idempotencyKey built from the business key; wrap each side effect in idempotency.once() or a MainSDK call; keep job inputs to ids, not personal data.
Do not: set jobs.driver to local in main.yaml, build a retry loop or a queue by hand, or loosen a check.
Verify: npx mainsdk check --env prod, then node --test on the job's test file.
Stop and ask if: the work cannot be repeated safely, or the app has no database for the queue.

Events and background jobs: every option, error and the instance broadcast.
Build a durable workflow: when one job is not enough and the work has to wait.
Send and receive webhooks: hand slow webhook work to a job.

Was this page helpful?