Server-side AI with Claude

The ai feature gives every scaffolded app a chat surface backed by Claude. The interesting part is not the chat UI — it is where the API key lives, and where it never goes. The browser sends a message to your backend; the backend talks to Anthropic; the key exists only as a deployment environment variable. Nothing that ships to a client ever contains it.

The shape: a Node action behind an internal boundary

On Convex, the flow is two files. convex/ai.ts holds the public surface: queries for the message list and a mutation that stores the user's message, inserts an empty assistant message, and schedules the generation. convex/aiNode.ts opens with "use node" and holds the actual Claude call as an internalAction — the Anthropic SDK needs Node APIs, and internal means no client can invoke it directly. The action reads the conversation history through an internal query, calls Claude, and patches the assistant message as text arrives. On the Supabase variant the same job is done by an ai-chat Edge Function with the key in function secrets — different runtime, same boundary.

Streaming without a streaming transport

The UX choice worth stealing: the action uses the SDK's streaming API, but the client never holds a stream. Instead, the action accumulates text and patches the assistant message row with a status of streaming, throttled to at most about four database writes per second. Convex's reactive queries do the rest — the message list updates live in every open tab, for free. When the stream finishes, one final patch flips the status to done; errors flip it to error with the message in the content. The client is just rendering a table.

The model default is claude-opus-4-8 with adaptive thinking and a 16,000-token budget — a deliberate quality-first default for an MVP assistant, and a one-line change if you want something faster or cheaper.

Setting the key

npx convex env set ANTHROPIC_API_KEY sk-ant-…          # dev deployment
npx convex env set ANTHROPIC_API_KEY sk-ant-… --prod   # production

If the key is missing, the assistant does not crash or hang — the action writes an error message into the chat that contains the exact command above. It is the doctor-check discipline applied inside the app: the failure names its fix, so an agent (or you) reads the chat bubble and knows precisely what to run.

Customizing the system prompt

The system prompt lives inline in the action — one string that starts as a concise in-app assistant for your app's title, answering in the user's language. Editing it is the highest-leverage customization in the feature: give it your domain vocabulary, your tone, your refusal rules. Because the prompt is server-side, users cannot see or override it, and changing it does not touch the client at all.

Why not just call the API from the browser?

Because every alternative is worse. A key in client code is public the moment you deploy — extracted from the bundle in minutes and billed to you indefinitely. A key behind a thin proxy without auth is the same thing with extra steps. The server-side action gives you the full set: the key stays secret, every request is tied to a signed-in user, the conversation history is assembled server-side so a client cannot forge it, and you get one place to add rate limits, logging, or tool use later. The pattern costs two files in the scaffold and closes the whole category of leaked-key incidents.

The browser talks to your backend; only your backend talks to Anthropic.

Enable it with --features ai, set one environment variable, and edit one string. That is the entire integration surface.