---
title: "AI gateway"
description: "Use an AI gateway for MainSDK AI calls - one endpoint routing to many model providers, main.yaml, keys, testing and swapping."
section: "integrations"
canonical: "https://mainsdk.dev/docs/integrations/ai-gateway"
updated: "2026-10-28"
---

# AI gateway

> Use an AI gateway for MainSDK AI calls - one endpoint routing to many model providers, main.yaml, keys, testing and swapping.

Canonical: https://mainsdk.dev/docs/integrations/ai-gateway · Updated: 2026-10-28

An AI gateway is a service that sits between your app and many model providers and routes each call to the model you name. The `gateway` driver behind [`ai`](https://mainsdk.dev/docs/interfaces/ai) sends calls to a gateway instead of to one provider. Pick it when you want one key and one bill for models from several makers.

## Interfaces it serves

### AI calls (`ai`)

- Status: Building for 0.3 · ships as a beta API
- Driver: `gateway` (AI gateway). built; reached today through a route such as gateway/<model>, not yet as ai.driver
- Docs: https://mainsdk.dev/docs/interfaces/ai.md

```yaml
ai:
  driver: gateway
```

Swap to another provider: `local`, `anthropic`, `openai`, `bedrock`, `ollama`, `llama-cpp`, `openai-compatible`.

## Before you start

1. Create an account with the gateway you use and an API key for your app, one per environment.
2. Turn on the providers and models you want to reach through it.
3. Check the gateway's data terms, and those of each provider behind it, against `ai.data`.

## Configure

```yaml title="main.yaml"
ai:
  driver: gateway
  key: secret:AI_GATEWAY_API_KEY     # suggested name; [TBD] key field name for drivers that need one
  models: [anthropic/claude-*]       # allowlist, across every provider the gateway reaches
  data: { training: never, retention: zero }
```

Which gateways the driver supports, and how it is pointed at one, are not specified yet [TBD].

## Use it

```ts title="lib/summarise.ts"
import { ai } from "mainsdk";

export async function summarise(text: string, user: { id: string }) {
  return ai.generate({
    model: "anthropic/claude-sonnet",   // [TBD] model identifiers per driver
    prompt: `Summarise: ${text}`,
    feature: "summary",
    user,
  });
}
```

## Local development and testing

Use `ai.driver: local` in `main.env.dev.yaml` to build without a key. `createTestApp()` from `@mainsdk/testing` runs `ai` on its in-memory fake, and `chaos.on("ai", { outage: true })` simulates the gateway being down.

## What MainSDK adds on top

- The key never in your code, only a `secret:` reference.
- Providers that train on or keep prompts are refused (`MAIN-AI-0001` from `mainsdk check`).
- Personal data redacted before the call, so neither the gateway nor the provider sees it.
- Spend limits per user, per feature and in total, and every call audited without the prompt.

See [AI calls](https://mainsdk.dev/docs/interfaces/ai).

## Limits and notes

- The gateway driver comes with the `ai` spec (phase 1, milestone 0.3).
- A gateway adds a party that sees redacted prompts. Its data terms must meet `ai.data` as well as the provider's [TBD how that is declared].
- `ai.raw()` returns the gateway client; `mainsdk check` lists every use.

## Swap to or from it

```yaml title="main.yaml"
ai:
  driver: gateway                    # was bedrock
```

AI calls are stateless, so `mainsdk swap` tests the new driver and switches with no migration.

## FAQ

**Do I still need `ai.models` with a gateway?** Yes. The allowlist applies whichever driver routes the call.

**Do caps use the gateway's prices?** Caps are enforced in your app before the call; how the gateway driver estimates cost is [TBD].

## Next

- [AI calls](https://mainsdk.dev/docs/interfaces/ai)
- [Amazon Bedrock](https://mainsdk.dev/docs/integrations/amazon-bedrock), [Anthropic](https://mainsdk.dev/docs/integrations/anthropic) and [OpenAI](https://mainsdk.dev/docs/integrations/openai)