Vercel AI SDK + AI Gateway Integration Guide

Technology: vercel-ai-sdk · Category: ai · Last reviewed: 2026-08-23

Source: https://tech-stack.codeamanilabs.org/guide/vercel-ai-sdk

Insight:

This is the unification layer: one generateText/streamText interface over every provider, with the AI Gateway adding a single key, automatic fallback chains, and cost/latency dashboards. Passing model as a string (provider/model) auto-routes through the Gateway — turning the hand-rolled routing in AI_WORKFLOWS.md into config instead of code.

██╗   ██╗███████╗██████╗  ██████╗███████╗██╗          █████╗ ██╗    ███████╗██████╗ ██╗  ██╗
██║   ██║██╔════╝██╔══██╗██╔════╝██╔════╝██║         ██╔══██╗██║    ██╔════╝██╔══██╗██║ ██╔╝
██║   ██║█████╗  ██████╔╝██║     █████╗  ██║         ███████║██║    ███████╗██║  ██║█████╔╝
╚██╗ ██╔╝██╔══╝  ██╔══██╗██║     ██╔══╝  ██║         ██╔══██║██║    ╚════██║██║  ██║██╔═██╗
 ╚████╔╝ ███████╗██║  ██║╚██████╗███████╗███████╗    ██║  ██║██║    ███████║██████╔╝██║  ██╗
  ╚═══╝  ╚══════╝╚═╝  ╚═╝ ╚═════╝╚══════╝╚══════╝    ╚═╝  ╚═╝╚═╝    ╚══════╝╚═════╝ ╚═╝  ╚═╝

Vercel AI SDK + AI Gateway Integration Guide

Focus: One TypeScript interface (generateText / streamText / generateObject) over every provider codeAmani uses — Anthropic, OpenAI, Gemini, DeepSeek, HuggingFace, ElevenLabs — with AI Gateway adding a single API key, automatic fallback chains, and built-in cost/latency observability. This replaces the hand-rolled routing in AI_WORKFLOWS.md.

Overview

Here is the big picture — one interface, many providers, all routed through a single Gateway:

flowchart LR
  A["Your app code"] --> B["generateText / streamText / generateObject"]
  B --> C["model as string<br/>provider/model"]
  C --> D["AI Gateway<br/>one AI_GATEWAY_API_KEY"]
  D --> E["Anthropic"]
  D --> F["OpenAI"]
  D --> G["Google · DeepSeek · others"]
  E --> H["Streamed tokens + usage"]
  F --> H
  G --> H
  H --> A

Two complementary pieces:

Net effect for codeAmani: the AI routing policy (Claude primary, OpenAI for structured output, HF/others as fallbacks) becomes config, not app code.

Official Documentation

Resource URL
AI SDK intro https://ai-sdk.dev/docs/introduction
Getting started (Node) https://ai-sdk.dev/docs/getting-started/nodejs
Providers list https://ai-sdk.dev/providers/ai-sdk-providers
Tool calling https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling
AI Gateway docs https://vercel.com/docs/ai-gateway
Gateway auth (AI_GATEWAY_API_KEY) https://vercel.com/docs/ai-gateway/authentication
Gateway provider options (order/only/sort) https://vercel.com/docs/ai-gateway/models-and-providers/provider-options
Gateway model fallbacks (gateway.models) https://vercel.com/docs/ai-gateway/models-and-providers/model-fallbacks

1. Install

npm install ai                       # core — v7 (ai@7.x)
# Optional explicit providers (only if NOT using the string/Gateway form):
npm install @ai-sdk/anthropic @ai-sdk/openai @ai-sdk/google @ai-sdk/deepseek
# React chat UI (separate package):
npm install @ai-sdk/react

Version drift: the core ai package is on v7 (7.x) but the provider and React packages track their own lower major lines — @ai-sdk/react, @ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google are all on 4.x, @ai-sdk/deepseek on 3.x. That mismatch is expected; always take the latest tag of each rather than trying to match version numbers to ai. The v4→v5→v7 shape changes are breaking — see the gotchas below and re-verify snippets against the docs before copying.

2. Credentials

# Gateway path (recommended) — one key for all providers:
AI_GATEWAY_API_KEY=...
# OR direct-provider path — one key each:
ANTHROPIC_API_KEY=...
OPENAI_API_KEY=...
GOOGLE_GENERATIVE_AI_API_KEY=...

On Vercel, the Gateway can also authenticate via the deployment's OIDC token (VERCEL_OIDC_TOKEN) with no key at all. Keep every key server-side.

3. Generate text (Gateway via model string)

import { generateText } from "ai";

const { text } = await generateText({
  model: "anthropic/claude-sonnet-5", // string -> routed through AI Gateway
  prompt: "Explain M-Pesa STK Push in one sentence.",
});

Switching providers is a one-line change — "openai/gpt-5", "google/gemini-2.5-flash", "deepseek/deepseek-chat". The surrounding code never changes.

v7 naming (breaking vs. v4/v5): the system prompt field is now instructions: (was system:), the per-response cap is maxOutputTokens: (was maxTokens:), tool schemas use inputSchema: (was parameters:), and multi-step tool loops use stopWhen: isStepCount(n) (import isStepCount from ai; was maxSteps: n). prompt and messages are unchanged.

4. Stream + structured output

import { streamText, generateObject } from "ai";
import { z } from "zod";

const result = streamText({ model: "google/gemini-2.5-flash", prompt });
for await (const chunk of result.textStream) process.stdout.write(chunk);

const { object } = await generateObject({
  model: "openai/gpt-5",
  schema: z.object({ amount: z.number(), phone: z.string() }),
  prompt: "Extract the payment amount and phone from: 'Send 500 to 0712345678'",
});

4a. useChat — streaming chat UI (AI SDK v7)

For a React chat UI, the useChat hook (from @ai-sdk/react) handles message state, streaming, and input wiring; the matching route handler runs streamText on the server and hands the typed stream back with createUIMessageStreamResponse({ stream: toUIMessageStream(...) }). This targets AI SDK v7 — the message shape is UIMessage (rendered via message.parts, not a flat content string), and the client posts through a DefaultChatTransport. Keep the API route server-side so your AI_GATEWAY_API_KEY never reaches the browser.

sequenceDiagram
  participant U as "Browser · useChat"
  participant R as "Route · api/chat"
  participant G as "AI Gateway"
  U->>R: "POST UIMessage list"
  R->>G: "streamText · convertToModelMessages"
  G-->>R: "typed part stream"
  R-->>U: "createUIMessageStreamResponse"

// app/(public)/chat/page.tsx
"use client";

import { useChat } from "@ai-sdk/react";
import { DefaultChatTransport } from "ai";
import { useState } from "react";

export default function Chat() {
  const { messages, sendMessage } = useChat({
    transport: new DefaultChatTransport({ api: "/api/chat" }),
  });
  const [input, setInput] = useState("");

  return (
    <div>
      {messages.map((m) => (
        <div key={m.id}>
          <strong>{m.role}: </strong>
          {m.parts.map((part, i) =>
            part.type === "text" ? <span key={i}>{part.text}</span> : null,
          )}
        </div>
      ))}

      <form
        onSubmit={(e) => {
          e.preventDefault();
          if (!input.trim()) return;
          sendMessage({ text: input });
          setInput("");
        }}
      >
        <input
          value={input}
          placeholder="Uliza chochote..."
          onChange={(e) => setInput(e.target.value)}
        />
      </form>
    </div>
  );
}

// app/api/chat/route.ts
import {
  streamText,
  convertToModelMessages,
  createUIMessageStreamResponse,
  toUIMessageStream,
  type UIMessage,
} from "ai";

export const maxDuration = 30; // allow streaming responses up to 30s

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: "anthropic/claude-sonnet-5", // string -> AI Gateway
    instructions: "You are a helpful assistant for Kenyan SMEs.",
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}

Gotcha: the v7 route handler wraps the result's typed part stream with toUIMessageStream({ stream: result.stream }) and returns it via createUIMessageStreamResponse(...). (The older result.toUIMessageStreamResponse() still works as a shorthand — Vercel's Gateway docs use it — but the two-call form above is the current canonical shape and is what the AI SDK docs show. Both replace the v4 toDataStreamResponse().) You must pass await convertToModelMessages(messages) to streamText — handing the raw UIMessage[] (with parts) straight to the model throws. The client reads message.parts, so there is no message.content string to render. Note the system prompt is instructions: in v7, not system:.

5. Fallback chains (the replacement for hand-rolled routing)

You can let the Gateway handle failover automatically — here is the order it walks:

flowchart TD
  A["Request"] --> B["Primary<br/>anthropic/claude-sonnet-5"]
  B --> Q1{"Primary OK?"}
  Q1 -->|"yes"| Z["Return text"]
  Q1 -->|"fails"| C["Fallback 1<br/>openai/gpt-5"]
  C --> Q2{"Fallback 1 OK?"}
  Q2 -->|"yes"| Z
  Q2 -->|"fails"| D["Fallback 2<br/>google/gemini-2.5-flash"]
  D --> Z

const { text } = await generateText({
  model: "anthropic/claude-sonnet-5", // primary
  prompt,
  providerOptions: {
    gateway: {
      models: ["openai/gpt-5", "google/gemini-2.5-flash"], // tried in order if primary fails
      // order: ["anthropic", "vertex"], // or pin provider routing order
    },
  },
});

The gateway.models fallback array now has its own docs page — Model Fallbacks (linked above). order / only / sort (provider routing) live on the Provider Options page. Both are configured under providerOptions.gateway.

Cost, latency, and tokens-per-model are visible in the AI Gateway dashboard — no custom metrics code needed.

codeAmani notes

Official docs: