Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/calm-keys-listen.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@ai-sdk/gateway': patch
---

feat(provider/gateway): bind realtime client secrets to origins and trusted provider options
68 changes: 36 additions & 32 deletions content/providers/01-ai-sdk-providers/00-ai-gateway.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -251,58 +251,62 @@ const model = gateway.experimental_realtime('openai/gpt-realtime-2');

The Gateway normalizes realtime the same way it normalizes every other modality: your client speaks the [normalized AI SDK realtime protocol](/docs/ai-sdk-core/realtime) and the Gateway translates to and from the upstream provider server-side. This means the same client code works regardless of which provider backs the model.

Gateway realtime v0 is server-side only. Calling `gateway.experimental_realtime()` in a browser throws an error. `getToken()` returns the Gateway auth token resolved by the provider (`apiKey`, `AI_GATEWAY_API_KEY`, or Vercel OIDC token), so do not expose it to browser clients.
Realtime sessions run in the browser with a single-use, short-lived Gateway
client secret. Mint the secret on your server with a Gateway API key, then
return only the `vcst_` token to the browser. The long-lived Gateway credential
never leaves your server.

```ts
```ts filename='app/api/realtime/setup/route.ts'
import { gateway } from '@ai-sdk/gateway';

// Server-side only. Do not return this token to browser clients.
const token = await gateway.experimental_realtime.getToken({
model: 'openai/gpt-realtime-2',
});
export async function POST() {
const token = await gateway.experimental_realtime.getToken({
model: 'openai/gpt-realtime-2',
expiresAfterSeconds: 60,
allowedOrigins: ['https://app.example.com'],
});

return Response.json(token);
}
```

<Note>
Gateway realtime v0 does not mint a provider-style short-lived client secret.
Gateway-minted ephemeral secrets are planned as future work; browser support
should wait for that flow.
</Note>

<Note>
The Gateway WebSocket route transports auth via the versioned
`Sec-WebSocket-Protocol` subprotocol and the model id via the `?ai-model-id=`
query - the WebSocket transport of the `Authorization` and `ai-model-id`
headers the HTTP routes use. Subprotocol values must fit the WebSocket token
grammar, and the complete `Sec-WebSocket-Protocol` header should stay compact
(under an 8 KiB safe header budget).
</Note>
`allowedOrigins` binds redemption to the listed browser origins. Client secrets
are also bound to the exact model, expire after at most five minutes, and can be
redeemed only once. Authenticate and rate-limit the setup endpoint using your
application's normal controls.

### Provider Options

Gateway [provider options](#gateway-provider-options) — `tags`, `user`, `byok`, compliance flags, and so on — are set under `providerOptions.gateway` in the **session configuration**, mirroring how they ride the request body on the non-realtime routes. Include them in the session configuration you send to the Gateway, and the Gateway applies them server-side:
Set trusted attribution and quota options under `providerOptions.gateway` in
the session configuration passed to `getToken()`. The Gateway seals these
values into the client secret, so browser frames cannot override them:

```ts
import type { GatewayProviderOptions } from '@ai-sdk/gateway';

const gatewayOptions: GatewayProviderOptions = {
tags: ['cooking-coach', 'v2'],
user: 'user-123',
quotaEntityId: 'workspace-123',
};

const sessionConfig = {
instructions: 'You are a concise voice assistant.',
providerOptions: { gateway: gatewayOptions },
};
const token = await gateway.experimental_realtime.getToken({
model: 'openai/gpt-realtime-2',
sessionConfig: {
instructions: 'You are a concise voice assistant.',
providerOptions: { gateway: gatewayOptions },
},
});
```

The `GatewayProviderOptions` type is exported from `@ai-sdk/gateway` for stable client-facing fields. The type also accepts service-owned option keys so the Gateway service can add or change server-side options without requiring an SDK release for every schema change. Runtime validation is owned by the Gateway service.
The client-secret flow currently pins `quotaEntityId`, `tags`, and `user`.
Other routing options are not used for realtime model selection.

<Note>
Provider options are sent in the first `session.update` frame (after the
socket opens), so connect-time options such as `byok` and quota selection
require a Gateway that resolves them from that frame. Routing knobs like
`order` / `only` do not apply to realtime, where the Gateway selects the
WebSocket-capable provider.
<Note type="warning">
Request-scoped `byok` is not supported with browser client secrets. Do not put
provider credentials in client code or session configuration. Configure BYOK
credentials in AI Gateway team settings so the Gateway can select them
server-side without exposing one user's credentials to another.
</Note>

## Available Models
Expand Down
23 changes: 17 additions & 6 deletions packages/gateway/src/gateway-provider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,12 @@ import { GatewayVideoModel } from './gateway-video-model';
import { GatewayRerankingModel } from './gateway-reranking-model';
import { GatewaySpeechModel } from './gateway-speech-model';
import { GatewayTranscriptionModel } from './gateway-transcription-model';
import { GatewayRealtimeModel } from './gateway-realtime-model';
import {
GatewayRealtimeModel,
type GatewayRealtimeFactory,
type GatewayRealtimeFactoryGetTokenOptions,
type GatewayRealtimePinnedProviderOptions,
} from './gateway-realtime-model';
import type { GatewayEmbeddingModelId } from './gateway-embedding-model-settings';
import type { GatewayImageModelId } from './gateway-image-model-settings';
import type { GatewayRerankingModelId } from './gateway-reranking-model-settings';
Expand All @@ -55,8 +60,6 @@ import type {
SpeechModelV4,
TranscriptionModelV4,
Experimental_VideoModelV4,
Experimental_RealtimeFactoryV4 as RealtimeFactoryV4,
Experimental_RealtimeFactoryV4GetTokenOptions as RealtimeFactoryV4GetTokenOptions,
ProviderV4,
} from '@ai-sdk/provider';
import { VERSION } from './version';
Expand Down Expand Up @@ -171,7 +174,7 @@ export interface GatewayProvider extends ProviderV4 {
* Creates an experimental realtime model for bidirectional audio/text
* communication over WebSocket, normalized through the AI Gateway.
*/
experimental_realtime: RealtimeFactoryV4;
experimental_realtime: GatewayRealtimeFactory;

/**
* Gateway-specific tools executed server-side.
Expand Down Expand Up @@ -298,6 +301,8 @@ export function createGateway(
const mintRealtimeClientSecret = async (params: {
modelId: string;
expiresAfterSeconds?: number;
allowedOrigins?: string[];
gatewayOptions?: GatewayRealtimePinnedProviderOptions;
}): Promise<{ token: string; expiresAt?: number }> => {
assertGatewayRealtimeServerEnvironment();
const auth = await getRealtimeAuthToken();
Expand All @@ -312,6 +317,12 @@ export function createGateway(
...(params.expiresAfterSeconds != null && {
expiresIn: params.expiresAfterSeconds,
}),
...(params.allowedOrigins != null && {
allowedOrigins: params.allowedOrigins,
}),
...(params.gatewayOptions != null && {
providerOptions: { gateway: params.gatewayOptions },
}),
},
successfulResponseHandler: createJsonResponseHandler(
gatewayClientSecretResponseSchema,
Expand Down Expand Up @@ -536,7 +547,7 @@ export function createGateway(
provider.experimental_realtime = Object.assign(
(modelId: GatewayRealtimeModelId) => createRealtimeModel(modelId),
{
getToken: async (tokenOptions: RealtimeFactoryV4GetTokenOptions) => {
getToken: async (tokenOptions: GatewayRealtimeFactoryGetTokenOptions) => {
const { model: modelId, ...secretOptions } = tokenOptions;
const model = createRealtimeModel(modelId);
const secret = await model.doCreateClientSecret(secretOptions);
Expand All @@ -547,7 +558,7 @@ export function createGateway(
};
},
},
) as RealtimeFactoryV4;
) as GatewayRealtimeFactory;

provider.chat = provider.languageModel;
provider.embedding = provider.embeddingModel;
Expand Down
62 changes: 58 additions & 4 deletions packages/gateway/src/gateway-realtime-model.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ describe('GatewayRealtimeModel', () => {
);
});

it('forwards modelId + expiresAfterSeconds to the mint hook and surfaces expiresAt', async () => {
it('forwards client-secret bindings to the mint hook and surfaces expiresAt', async () => {
const createClientSecret = vi.fn(async () => ({
token: 'vcst_minted',
expiresAt: 1_700_000_060,
Expand All @@ -81,16 +81,53 @@ describe('GatewayRealtimeModel', () => {

const result = await model.doCreateClientSecret({
expiresAfterSeconds: 120,
allowedOrigins: ['https://app.example.com'],
sessionConfig: {
providerOptions: {
gateway: {
quotaEntityId: 'quota_1',
tags: ['realtime'],
user: 'user_1',
order: ['openai'],
},
},
},
});

expect(createClientSecret).toHaveBeenCalledWith({
modelId: 'openai/gpt-realtime-2',
expiresAfterSeconds: 120,
allowedOrigins: ['https://app.example.com'],
gatewayOptions: {
quotaEntityId: 'quota_1',
tags: ['realtime'],
user: 'user_1',
},
});
expect(result.token).toBe('vcst_minted');
expect(result.expiresAt).toBe(1_700_000_060);
});

it('rejects request-scoped BYOK credentials before minting', async () => {
const createClientSecret = vi.fn(async () => ({ token: 'unused' }));
const model = new GatewayRealtimeModel('openai/gpt-realtime-2', {
provider: 'gateway.realtime',
baseURL: 'https://ai-gateway.vercel.sh/v4/ai',
createClientSecret,
});

await expect(
model.doCreateClientSecret({
sessionConfig: {
providerOptions: {
gateway: { byok: { openai: [{ apiKey: 'sk-secret' }] } },
},
},
}),
).rejects.toThrow(/Request-scoped BYOK credentials cannot be used/);
expect(createClientSecret).not.toHaveBeenCalled();
});

it('omits expiresAt when the mint hook does not return one', async () => {
const result = await createTestModel().doCreateClientSecret();
expect('expiresAt' in result).toBe(false);
Expand Down Expand Up @@ -146,9 +183,8 @@ describe('GatewayRealtimeModel', () => {
});

it('passes session-update provider options through unchanged', () => {
// "Support all the things": gateway provider options (tags/user/byok/...)
// ride session.update exactly as they ride the request body on the
// non-realtime routes. The identity codec must not strip them.
// The identity codec stays transport-neutral. Client-secret sessions
// apply their trusted options from the minted token on the Gateway.
const event = {
type: 'session-update',
config: {
Expand Down Expand Up @@ -192,6 +228,16 @@ describe('gateway.experimental_realtime', () => {
const result = await mintGateway.experimental_realtime.getToken({
model: 'openai/gpt-realtime',
expiresAfterSeconds: 120,
allowedOrigins: ['https://app.example.com'],
sessionConfig: {
providerOptions: {
gateway: {
quotaEntityId: 'quota_1',
tags: ['voice'],
user: 'user_1',
},
},
},
});

// Minted token (not the raw key), and the mint hit the v1 route on the
Expand All @@ -210,6 +256,14 @@ describe('gateway.experimental_realtime', () => {
expect(JSON.parse(init.body as string)).toMatchObject({
model: 'openai/gpt-realtime',
expiresIn: 120,
allowedOrigins: ['https://app.example.com'],
providerOptions: {
gateway: {
quotaEntityId: 'quota_1',
tags: ['voice'],
user: 'user_1',
},
},
});
// Authenticated with the long-lived key.
const sentHeaders = new Headers(init.headers);
Expand Down
73 changes: 69 additions & 4 deletions packages/gateway/src/gateway-realtime-model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,39 @@ import type {
Experimental_RealtimeModelV4ClientEvent as RealtimeModelV4ClientEvent,
Experimental_RealtimeModelV4ClientSecretOptions as RealtimeModelV4ClientSecretOptions,
Experimental_RealtimeModelV4ClientSecretResult as RealtimeModelV4ClientSecretResult,
Experimental_RealtimeFactoryV4GetTokenResult as RealtimeFactoryV4GetTokenResult,
Experimental_RealtimeModelV4ServerEvent as RealtimeModelV4ServerEvent,
Experimental_RealtimeModelV4SessionConfig as RealtimeModelV4SessionConfig,
} from '@ai-sdk/provider';
import { getGatewayRealtimeProtocols } from './gateway-realtime-auth';
import type { GatewayRealtimeModelId } from './gateway-realtime-model-settings';
import type { GatewayProviderOptions } from './gateway-provider-options';

export type GatewayRealtimePinnedProviderOptions = Pick<
GatewayProviderOptions,
'quotaEntityId' | 'tags' | 'user'
>;

export type GatewayRealtimeClientSecretOptions =
RealtimeModelV4ClientSecretOptions & {
/**
* Browser origins allowed to redeem the client secret. When set, a
* WebSocket upgrade without a matching Origin header is rejected.
*/
allowedOrigins?: string[];
};

export type GatewayRealtimeFactoryGetTokenOptions = {
model: GatewayRealtimeModelId;
} & GatewayRealtimeClientSecretOptions;

export interface GatewayRealtimeFactory {
(modelId: GatewayRealtimeModelId): RealtimeModelV4;

getToken(
options: GatewayRealtimeFactoryGetTokenOptions,
): Promise<RealtimeFactoryV4GetTokenResult>;
}

export type GatewayRealtimeModelConfig = {
provider: string;
Expand All @@ -21,6 +50,8 @@ export type GatewayRealtimeModelConfig = {
createClientSecret: (params: {
modelId: string;
expiresAfterSeconds?: number;
allowedOrigins?: string[];
gatewayOptions?: GatewayRealtimePinnedProviderOptions;
}) => PromiseLike<{ token: string; expiresAt?: number }>;
};

Expand Down Expand Up @@ -52,18 +83,23 @@ export class GatewayRealtimeModel implements RealtimeModelV4 {
* credential. The customer's server calls this (via
* `gateway.experimental_realtime.getToken`) and hands the returned token to
* the browser, which connects with it through the `ai-gateway-auth.<token>`
* subprotocol. `expiresAfterSeconds` is forwarded to the mint endpoint;
* `sessionConfig` is intentionally unused here — it is applied later via the
* normalized `session-update` event.
* subprotocol. `expiresAfterSeconds`, `allowedOrigins`, and the safe Gateway
* options from `sessionConfig` are sealed into the token by the mint
* endpoint. Credentials are deliberately excluded from this flow.
*/
async doCreateClientSecret(
options?: RealtimeModelV4ClientSecretOptions,
options?: GatewayRealtimeClientSecretOptions,
): Promise<RealtimeModelV4ClientSecretResult> {
const gatewayOptions = getPinnedGatewayOptions(options?.sessionConfig);
const secret = await this.config.createClientSecret({
modelId: this.modelId,
...(options?.expiresAfterSeconds != null && {
expiresAfterSeconds: options.expiresAfterSeconds,
}),
...(options?.allowedOrigins != null && {
allowedOrigins: options.allowedOrigins,
}),
...(gatewayOptions != null && { gatewayOptions }),
});
return {
token: secret.token,
Expand Down Expand Up @@ -102,6 +138,35 @@ export class GatewayRealtimeModel implements RealtimeModelV4 {
}
}

function getPinnedGatewayOptions(
sessionConfig: RealtimeModelV4SessionConfig | undefined,
): GatewayRealtimePinnedProviderOptions | undefined {
const gatewayOptions = sessionConfig?.providerOptions?.gateway;
if (gatewayOptions == null) {
return undefined;
}
if (typeof gatewayOptions !== 'object' || Array.isArray(gatewayOptions)) {
throw new Error('providerOptions.gateway must be an object.');
}
const options = gatewayOptions as Record<string, unknown>;
if ('byok' in options && options.byok !== undefined) {
throw new Error(
'Request-scoped BYOK credentials cannot be used with Gateway realtime client secrets. Configure BYOK credentials in the AI Gateway instead of sending them through a browser session.',
);
}

const pinned: Record<string, unknown> = {};
for (const key of ['quotaEntityId', 'tags', 'user'] as const) {
if (key in options && options[key] !== undefined) {
pinned[key] = options[key];
}
}

return Object.keys(pinned).length > 0
? (pinned as GatewayRealtimePinnedProviderOptions)
: undefined;
}

/**
* Build the Gateway realtime WebSocket URL. The HTTP(S) base URL is upgraded to
* WS(S) and the model id rides the `?ai-model-id=` query — the WS transport of
Expand Down
Loading