DEV Community

Cuelark
Cuelark

Posted on

Gemini 4 Neon in a React chat: going past 64K output without breaking your token meter

By the Cuelark Team. Every model detail below comes from Google's launch post and Gemini API docs, and the SDK details come from the Vercel AI SDK's release notes and published package, all checked on 21 November 2026. The code ran on our machine against a local mock of the documented streamGenerateContent stream. We had no Gemini API key for this test, so read it as a wiring test, not a quality review.

Google released Gemini 4 Neon (gemini-4-neon) on November 18. It sits between Hydrogen and Carbon in the Gemini 4 family and is aimed at complex coding, knowledge work and science. The launch post leads with benchmark rankings. We didn't check those, so we'll skip them.

The part that matters for a chat UI is quieter. Neon can produce up to 1M tokens in a single answer, but changing the model ID alone doesn't enable that. Any one request still stops at 65,536 tokens of thinking plus output. Here's how the longer answers work with the AI SDK, and two things we only noticed once we ran it.

What the docs say

From What's new in Gemini 4 Neon and the release notes:

  • Price: $3.00 per 1M input tokens and $15.00 per 1M output tokens, thinking included, flat across the window. Cached input is $0.15 per 1M (a 95% discount from implicit caching).
  • Paid tier only. Unlike Hydrogen, there's no free tier.
  • Thinking levels: low, medium, high (the default) and xhigh. minimal is rejected.
  • One shared window: input, thinking and output share 1,048,576 tokens.

The long decode guide explains the rest. Generation runs in hops of up to 64K tokens. If a hop hits that cap and budget is left, the response ends with finishReason: CONTINUATION and an encrypted continuationToken. You send the same request again with that token and don't add the partial output to contents, because the token carries the state. When the whole max_output_tokens budget is used, the response ends with MAX_TOKENS instead.

Google is also clear about when this is worth it. Chat, Q&A and summaries finish well under 32K tokens and don't need it. It's for hard reasoning and "write the whole thing" jobs like a full migration spec. So we made it a per-message toggle, not the default.

SDK support

@ai-sdk/google 4.0.123 (November 20) added the gemini-4-neon ID with xhigh. Long decode support itself landed earlier, in 4.0.95: the provider continues CONTINUATION hops for you, sums usage, and exposes initialPromptTokenCount. We used ai@7.0.179 and @ai-sdk/google@4.0.124.

The route

// app/api/chat/route.ts
import { streamText, convertToModelMessages, type UIMessage } from 'ai';
import { google } from '@ai-sdk/google';

export const maxDuration = 900; // hops can take minutes; check your host's limit

export type ChatMeta = {
  finishReason?: string;
  contextTokens?: number;     // what the conversation actually occupies
  billedInputTokens?: number; // summed over every hop, for cost only
  continuationToken?: string;
};

export async function POST(req: Request) {
  const { messages, long = false, continuationToken }: {
    messages: UIMessage[]; long?: boolean; continuationToken?: string;
  } = await req.json();

  const result = streamText({
    model: google('gemini-4-neon'),
    messages: await convertToModelMessages(messages),
    // Above 65,536 the provider runs the 64K hops for you
    maxOutputTokens: long ? 262_144 : undefined,
    providerOptions: {
      google: {
        thinkingConfig: { thinkingLevel: long ? 'high' : 'medium', includeThoughts: true },
        ...(continuationToken && { continuationToken }),
      },
    },
  });

  return result.toUIMessageStreamResponse<UIMessage<ChatMeta>>({
    sendReasoning: true,
    messageMetadata: ({ part }) => {
      if (part.type !== 'finish-step') return;
      const g = part.providerMetadata?.google as
        { initialPromptTokenCount?: number; continuationToken?: string } | undefined;
      const input = part.usage.inputTokens ?? 0;
      const output = part.usage.outputTokens ?? 0;
      return {
        finishReason: part.finishReason,
        billedInputTokens: input,
        contextTokens: (g?.initialPromptTokenCount ?? input) + output,
        continuationToken: g?.continuationToken,
      };
    },
  });
}
Enter fullscreen mode Exit fullscreen mode

What the test showed

Our mock replaces fetch and streams SSE chunks shaped like the docs' examples. It isn't a captured Gemini response. We drove the route the way useChat does.

  1. Without maxOutputTokens, the answer just stops. One request, finishReason: 'length', half a spec. The UI shows it as finished unless you check.
  2. With 262,144, the hops are invisible. Three requests went out. The second and third carried the previous continuationToken, and contents was identical each time. The first hop was all thinking with empty text, which the docs call normal. The client got one reasoning block and one text block, as if it were a single response.
  3. usage.inputTokens is not your context size. It came back as 199,608, the sum of each hop's input (1,000 + 66,536 + 132,072), because every hop re-reads everything generated so far. The real context was 162,072. The docs warn about this. If your app trims or summarises history when a token count gets close to the limit, feed it initialPromptTokenCount plus output, not inputTokens, or it'll compact far too early.

For the cut-off case we added a Continue button. It sends the original prompt plus the token and a bigger budget. regenerate already drops the partial answer from the request, which is what Google asks for:

{status === 'ready' && meta?.finishReason === 'length' && meta.continuationToken && (
  <button onClick={() => {
    setCarry(carry + textOf(last!)); // the new message holds only the rest
    regenerate({ body: { long: true, continuationToken: meta.continuationToken } });
  }}>
    Hit the token limit. Continue
  </button>
)}
Enter fullscreen mode Exit fullscreen mode

The rest of the component is a normal useChat list with a details block for reasoning. We type-checked it and ran this flow through the AI SDK Chat class: one user message went out, the token reached Gemini, and the displayed text joined up.

Honest limits

  • We didn't call the real API. Before you ship, run one real long request and compare the hop count and usage with what you log.
  • Dropped streams don't resume from checkpoints. After the first 64K on Neon, the stream sends a checkpoint token about every 5,000 tokens, and Google's REST loop resumes from the last one. In our test, a stream cut after a checkpoint ended as finishReason: 'interrupted' with no token in the metadata, and the text after the checkpoint stayed in the message. With this SDK version you resend the request instead.
  • The resumed message is only the tail. Our carry fixes the display, but the stored history holds just the second part. Merge them before the next turn.
  • Time limits. Google suggests HTTP timeouts of at least 15 minutes, and the server closes a stream after 20. Check your host's function time limit before relying on maxDuration. For truly long jobs, Google recommends background execution on the Interactions API, which this route doesn't use.
  • Cost. Every hop is billed, including thinking-only hops. Repeated input in each hop gets the 95% cache discount, but if the model uses the whole 262,144-token budget, that's about $3.90 of output for one answer.

Sources

If you'd rather not hand-build the reasoning block and message list, Cuelark UI has shadcn-style components for AI chat that work with this setup.

Top comments (0)