Together AI Integration Guide

Technology: together-ai · Category: ai · Last reviewed: 2026-08-23

Source: https://tech-stack.codeamanilabs.org/guide/together-ai

Insight:

Together AI is the open-model marketplace tier — 200+ open-weight LLMs, vision, image (FLUX) and embedding models behind one OpenAI-compatible endpoint. Swap baseURL to https://api.together.ai/v1 and your existing OpenAI / AI-SDK code routes to Llama, Qwen, DeepSeek or gpt-oss with only a model-string change. Use it when you want open-weight flexibility, image generation, or fine-tuning without leaving the OpenAI call shape.

████████╗ ██████╗  ██████╗ ███████╗████████╗██╗  ██╗███████╗██████╗      █████╗ ██╗
╚══██╔══╝██╔═══██╗██╔════╝ ██╔════╝╚══██╔══╝██║  ██║██╔════╝██╔══██╗    ██╔══██╗██║
   ██║   ██║   ██║██║  ███╗█████╗     ██║   ███████║█████╗  ██████╔╝    ███████║██║
   ██║   ██║   ██║██║   ██║██╔══╝     ██║   ██╔══██║██╔══╝  ██╔══██╗    ██╔══██║██║
   ██║   ╚██████╔╝╚██████╔╝███████╗   ██║   ██║  ██║███████╗██║  ██║    ██║  ██║██║
   ╚═╝    ╚═════╝  ╚═════╝ ╚══════╝   ╚═╝   ╚═╝  ╚═╝╚══════╝╚═╝  ╚═╝    ╚═╝  ╚═╝╚═╝

Together AI Integration Guide

Focus: Serverless inference for 200+ open-weight models — chat, vision, embeddings, rerank, and FLUX image generation — through an OpenAI-compatible API plus a native SDK.

Overview

Together AI is an inference platform for open-source models. A single API key unlocks chat/completion models (Llama, Qwen, DeepSeek, gpt-oss, MiniMax), multimodal vision models, embedding + rerank models, audio (speech-to-text / TTS), and FLUX image + video generation — all billed per token / per image with no GPU management. The API is OpenAI-compatible, so existing OpenAI-SDK code works after changing the API key and base URL; for richer features (images, rerank, fine-tuning, dedicated endpoints) there's a first-party together-ai / together SDK.

For codeAmani, Together AI is the open-weight lever: it sits alongside Anthropic (frontier quality) and DeepSeek (budget reasoning) as the place to reach for open models, image generation, or a fine-tuned house model — without rewriting integration code.

The same OpenAI call shape simply points at Together and fans out to any open model:

flowchart LR
  A["Your app code"] --> B["OpenAI SDK<br/>baseURL · api.together.ai/v1"]
  B --> C{"Which model string?"}
  C -->|"chat"| D["Llama 3.3 · Qwen · DeepSeek<br/>gpt-oss · MiniMax"]
  C -->|"vision"| E["Qwen2.5-VL · Llama-Vision"]
  C -->|"embeddings"| F["multilingual-e5 · BGE"]
  C -->|"images"| G["FLUX.2 / FLUX.1"]

Official Documentation

Setup

Option A — OpenAI SDK (drop-in)

Together's API mirrors OpenAI's REST schema. Keep the OpenAI SDK; change the key and base URL.

npm install openai

import OpenAI from "openai";

const together = new OpenAI({
  apiKey: process.env.TOGETHER_API_KEY!,
  baseURL: "https://api.together.ai/v1",
});

const res = await together.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [{ role: "user", content: "Hello!" }],
});
console.log(res.choices[0].message.content);

Option B — Native Together SDK (full surface)

Use the first-party SDK for images, rerank, fine-tuning, batches, and dedicated endpoints. The client reads TOGETHER_API_KEY from the environment by default.

npm install together-ai     # Node / TypeScript
pip install together        # Python

import Together from "together-ai";

const client = new Together(); // picks up TOGETHER_API_KEY

const chat = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [{ role: "user", content: "Say this is a test!" }],
});
console.log(chat.choices);

from together import Together

client = Together()  # picks up TOGETHER_API_KEY

resp = client.chat.completions.create(
    model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
    messages=[{"role": "user", "content": "What is 2 + 2?"}],
)
print(resp.choices[0].message.content)

Core Patterns

Streaming

const stream = await client.chat.completions.create({
  model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  messages: [{ role: "user", content: prompt }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
// Cancel anytime with stream.controller.abort()

Vision (multimodal)

resp = client.chat.completions.create(
    model="Qwen/Qwen2.5-VL-72B-Instruct",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image."},
            {"type": "image_url", "image_url": {"url": "https://example.com/receipt.png"}},
        ],
    }],
)
print(resp.choices[0].message.content)

Embeddings

resp = client.embeddings.create(
    model="intfloat/multilingual-e5-large-instruct",
    input=["The cat sat on the mat", "A dog played in the park"],
)
for row in resp.data:
    print(len(row.embedding), "dimensions")

Image generation (FLUX)

const image = await client.images.generate({
  model: "black-forest-labs/FLUX.2-pro",
  prompt: "A vibrant Nairobi street market at golden hour, photorealistic",
  width: 1024,
  height: 768,
});
console.log(image.data[0].url); // hosted URL (or b64_json if requested)
// Budget / fastest: model "black-forest-labs/FLUX.1-schnell" with steps: 4

Next.js App Router streaming route

// app/api/ai/together/route.ts
import OpenAI from "openai";
import { NextRequest } from "next/server";

const together = new OpenAI({
  apiKey: process.env.TOGETHER_API_KEY!,
  baseURL: "https://api.together.ai/v1",
});

export async function POST(req: NextRequest) {
  const { messages } = await req.json();

  const stream = await together.chat.completions.create({
    model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
    stream: true,
    messages,
  });

  const encoder = new TextEncoder();
  return new Response(
    new ReadableStream({
      async start(controller) {
        for await (const chunk of stream) {
          const text = chunk.choices[0]?.delta?.content ?? "";
          if (text) controller.enqueue(encoder.encode(text));
        }
        controller.close();
      },
    }),
    { headers: { "Content-Type": "text/plain; charset=utf-8" } },
  );
}

Serverless vs. Dedicated Endpoints

Together runs models two ways. Pick by traffic shape:

flowchart TD
  A["Need to run an open model"] --> B{"Predictable<br/>high volume?"}
  B -->|"no — bursty / dev"| C["Serverless<br/>pay-per-token, shared pool"]
  B -->|"yes — steady QPS"| D["Dedicated endpoint<br/>reserved GPUs, flat hourly"]
  C --> E["Zero ops, instant, per-token billing"]
  D --> F["Stable latency, no rate-limit contention"]

• Serverless — pay per token/image on a shared pool. Default for development and bursty workloads. Just call the model ID.
• Dedicated endpoints — reserved GPU capacity billed hourly; predictable latency for steady production traffic. Provision via the dashboard or the client.endpoints API.

AI Routing: Where Together AI Fits

In codeAmani's routing strategy, Together AI is the open-model + image tier:

// lib/ai.ts — Together slots in as the open-model tier
function selectModel(task: "reason" | "openchat" | "vision" | "image") {
  switch (task) {
    case "reason":   return { provider: "anthropic", model: "claude-sonnet-4-6" };
    case "openchat": return { provider: "together", model: "meta-llama/Llama-3.3-70B-Instruct-Turbo" };
    case "vision":   return { provider: "together", model: "Qwen/Qwen2.5-VL-72B-Instruct" };
    case "image":    return { provider: "together", model: "black-forest-labs/FLUX.2-pro" };
  }
}

Because Together is OpenAI-shaped, the resilient-fallback pattern from the DeepSeek guide applies unchanged — retry transient 429/5xx with jittered backoff, then fall back to Anthropic Claude.

Environment Variables

# Required
TOGETHER_API_KEY=...

# Optional — the native SDK reads TOGETHER_BASE_URL (defaults to
# https://api.together.ai/v1). Point it at a self-hosted / proxied
# deployment without touching code — the open-weights on-ramp story.
TOGETHER_BASE_URL=https://api.together.ai/v1

# Base URL is set in code (OpenAI SDK): https://api.together.ai/v1
# The native SDK reads TOGETHER_API_KEY automatically.

codeAmani Notes

• Secrets server-side only. TOGETHER_API_KEY lives in .env.local / Vercel env vars and is used only from API routes, server actions, or edge functions — never shipped to the browser. Proxy all calls through app/api/**.
• Open weights = data-sovereignty option. For KDPA-sensitive or government workloads, the same open models (Llama, Qwen, DeepSeek) Together hosts can later be self-hosted with no code change beyond the base URL — Together is the low-friction on-ramp before committing to GPUs.
• Images for African-market UI. FLUX on Together is a cheap way to generate localized marketing/illustration assets (Nairobi scenes, Swahili signage prompts) without a separate image provider.
• Multilingual embeddings for Swahili RAG. intfloat/multilingual-e5-large-instruct embeds Swahili and English in one model — a better fit for code-switched East African text than English-only embeddings. Store vectors in Supabase pgvector or Pinecone.
• Cost discipline. Open models are far cheaper than frontier APIs for high-volume SME tasks (classification, summarization, RAG synthesis) — route the bulk there and reserve Claude for quality-critical calls.
• Mobile-first latency. For 2G/3G users, prefer smaller models (Meta-Llama-3.1-8B-Instruct-Turbo, 8B-class) and FLUX.1-schnell (4-step) to keep response and image-generation times low; stream chat responses so the UI paints early.

Troubleshooting

Official docs:

• https://docs.together.ai/docs/quickstart
• https://docs.together.ai/docs/inference/openai-compatibility
• https://docs.together.ai/reference
• https://docs.together.ai/docs/serverless-models
• https://docs.together.ai/docs/dedicated-endpoints