Open-weights models: A case (and first step) for Gemma over Gemini for budget & privacy

TL;DR:

Gemini is the premier frontier model (family) from Google DeepMind. In the current (2026) era of cutting generative AI (genAI) costs and maintaining data sovereignty, using open-weights models is a viable alternative. If you're a technical professional wanting to learn more about and why you should consider them, you're in the right place. As a Gemini user, you'll be happy to know Google DeepMind also makes an open-weights model (family): Gemma. Finally, if you have code already using Gemini via its API (whether from the Google AI or Vertex AI^ platforms), learn how to immediately switch to Gemma to kickstart open-weights your journey. In a future post, we'll get Gemma running locally on your laptop.

Introduction

Welcome to another installment on the blog covering Google APIs for Python (and sometimes Node.js) developers. I love covering content you won't find in Google's documentation, all while sharing best practices when using Google developer technologies. These are the primary categories I've covered so far:

• Google Cloud/GCP (AI/ML, serverless)
• Google Workspace/GWS (Drive, Docs, Sheets, Gmail, etc.)
• YouTube
• Maps
• Generative AI with Gemini... and now Gemma (this series)
• Credentials (API keys, OAuth client IDs, service accounts)

We stay on genAI again in this post, targeting those catching up to the steep growth in usage of "open-weights" models, why you should consider them, and moving from Gemini to its open-weights cousin, Gemma.

Open-weights surpassed frontier model usage in 2026

You are late to the game, but "better late than sorry." The date was April 20, 2026. According to OpenRouter, that was the last week when closed-weights models had more usage. Since then, the majority has been with open-weights models. More specifically, OpenRouter's chart (by week) below measures the "share of tokens processed by open-weight vs. closed-weight models" where stealth models are excluded.

SOURCE: OpenRouter AI USAGE: This image was composited using Nano Banana 2 (Gemini 3.1 Flash Image) and final output verified by the author.

Background

If you haven't been living under a rock, you would know that roughly 80% of all enterprises burned through most or all of their artificial intelligence (AI) budgets early in 2026. It was easy when employees were strongly encouraged to use AI, resulting in a phenomenon known as "tokenmaxxing" a workplace practice where companies incentivize employees to maximize AI token usage, treating that volume as proof of productivity.

It's no wonder such budgets were exceeded so early on during the year, resulting in the jump at commandeering the term, "tokenomics", originally created for the crypto industry, a portmanteau blending of "token" and "economics," to refer to getting control and better management of token consumption, meaning cost.

Cost savings with open source

Reducing AI usage costs deserves its own series of posts as it's a massive topic with a variety of techniques resulting in some sort of cost reduction, but, we're here to explore one technique, one analogous to the cost savings transitioning away from proprietary software to "open source." Open-source software is publicly-available source code anyone can inspect, modify, improve, or alter to do something different, and distribute.

As a member of Red Hat, one of the original open source companies, I'm even more convinced in the open source business model and its impact on innovation. Nothing comes for free, so while open source companies producing software typically make the base project code available for free, they'll generate revenue by producing enhanced software, e.g., Red Hat Enterprise Linux, Red Hat AI product suite, etc., and/or have a professional services business that helps build and implement customizations for customers.

The real questions for us today are: Can a similar model (pun intended) apply to today's world of genAI and LLMs (large language models)? Do companies have a viable open source alternative to closed-source, proprietary LLMs? The answer is, "Yes, but."

Can models be open source for models?

The "gotcha" is it's challenging to label LLMs as "open source" because models themselves are not just code. An AI model has innate behavior resulting from training over a large data set, meaning it includes the data as well as numerical parameters rather than hardcoded rules represented by pure code.

In order for an AI model to be fully open source, all the code used to train the model must be open source along with either the training data (or its equivalent, e.g., data description & sources, as some may be proprietary), and the complete set of the model's weights and biases, the core numerical parameters inside the model's neural network that store its learned knowledge, enabling it to make decisions that it does. Learn more about weights and biases in IBM's "What are model parameters?" post.

The "code" also includes any data processing tools or training workflow scripts used to prepare the training data and build the model. An appropriately-skilled team should be able to study & audit all of this "source" and rebuild a substantially-equivalent model.

It's likely that all of this combined represents proprietary intellectual property, so most models won't be open source by definition. While not common, there are some models like the OLMo model family by the Allen Institute for AI that does provide the full training data set along with everything else, serving as one of the clearest examples of a truly open-source model.

Open weights: closest thing to open source for models

While most companies won't release fully open-sourced models outside their walled gardens, the next best thing is being able to download a model and run it yourself, on your own hardware, in your own cost center. You can download models from a repository like HuggingFace or through the UI (user interface) of an inference tool you're using to run models.

So while not open source, open-weights models are as close as you can get from this perspective, and it's also why they're loosely (though erroneously) referred to as "open-source models."

What do you get when you "download a model?"

When you download a model, you'll get at least its weights, architecture, tokenizer, and possibly additional metadata. What you actually get depends on what you intend to do with an open-weights model. If you're going to serve a model for many users on a capable machine, you'll get all of those files (weights, architecture, tokenizer, extra metadata) in a folder.

However, if you're going to run the model on your local computer using llama.cpp or any of the UIs or apps running on top of llama.cpp, like Ollama, LM Studio, Jan, Unsloth Desktop, etc., the weights, architecture, tokenizer and metadata will be combined into a single GGUF (.gguf) file.

What can you do with open-weights models?

By downloading an open-weights model, having the weights available gives you the ability to do a variety of things with the model:

None of these are possible for you to do with frontier or closed-weights models. Model vendors can provide finetuned versions, distilled models, or various quantized versions, but these processes are under their control, not yours. The most obvious thing you can do is to run a model locally, on your own infrastructure, and perform inference (model queries) on it. Let's elaborate a bit more on local inference while the others are outside the scope of this post.

What are other benefits of open-weights models?

When running the model locally, you'll either have a chat service connected to it or have it available via API endpoints to serve it to your employees or customers. By using models you run yourself, you've freed yourself from several disadvantageous attributes from using frontier models:

Open-weights model families and licenses

Now that you're excited about open-weights models, what are some of the more well-known ones? If you've heard of DeepSeek, Gemma (Google), GLM (Z.ai), Kimi (Moonshot.ai), Llama (Meta), Mistral, Nemotron (NVIDIA), or Qwen (Alibaba), they are all examples open-weights model families. Besides their features and effectiveness, one key differentiator between them is the license they each use.

As a user of open-weights models, you must investigate whether their licenses are permissive enough for your use case. Most open-weights models will have an Open Source Initiative (OSI) approved license like the Apache or MIT licenses. However, it could also be more restrictive, Meta's Llama Community Use licenses; see the main Llama page (scroll down to "Llama license & use policy").

Are open-weights models "as good as" frontier models?

There's no post out there that will be accurate across time and the numerous models available out there, and similar to the consensus that proprietary software "is better than" open-source software, there's a clear distinction between features people are used to from their subscriptions vs. what's available from open source. Similarly frontier models still hold the edge in advanced, state-of-the-art features over what you get with open-weights models.

As we are most of the way through 2026 at this snapshot in time, the general consensus is that frontier models hold an advantage over open-weights models, but the latter catches up in well under a year, perhaps only half a year:

1. The UK-based AI Security Institute estimates that "[open-weights] models compare to frontier models related 4-5 months prior on [a] narrow [set of] cyber tasks."
2. LessWrong is "an online forum and community dedicated to improving human reasoning and decision-making," where AI is a central topic. In one of their posts, the author estimates that, "on private benchmarks, where the data is not publicly accessible, open models are roughly 8-10 months behind the closed frontier, while for public benchmarks the gap is roughly 4-6 months."
3. In this last example, DeepSeek estimates frontier models are ahead by 3 to 6 months.

That being said, time goes by fast when you're having fun, so if you're open to exploring the use of open-weights models, read on.

Gemma code samples

Now that you're caught up with LLMs and dived into open-weights vs. frontier models, no doubt you're here because you're curious about Gemma as an alternative to Gemini. Below are some code samples calling Gemma via the Gemini API:

Python on Google AI (GAI)

from google import genai
from settings import API_KEY

PROMPT = 'Describe a cat in a few sentences'
MODEL = 'gemma-4-26b-a4b-it'
print('** GenAI text: %r model & prompt %r\n' % (MODEL, PROMPT))

GENAI = genai.Client(api_key=API_KEY)
response = GENAI.models.generate_content(model=MODEL, contents=PROMPT)
print(response.text)

Python on Vertex AI (GCP)

from google import genai
from settings import GCP_METADATA

PROMPT = 'Describe a cat in a few sentences'
MODEL = 'gemma-4-26b-a4b-it-maas'
print('** GenAI text: %r model & prompt %r\n' % (MODEL, PROMPT))

GENAI = genai.Client(vertexai=True, **GCP_METADATA)
response = GENAI.models.generate_content(model=MODEL, contents=PROMPT)
print(response.text)

Node.js on GAI

import 'dotenv/config';
import { GoogleGenAI } from "@google/genai";

const PROMPT = 'Describe a cat in a few sentences';
const MODEL = 'gemma-4-26b-a4b-it';
console.log(`** GenAI text: '${MODEL}' model & prompt '${PROMPT}'\n`);

const GENAI = new GoogleGenAI({ apiKey: process.env.API_KEY });

async function main() {
  const response = await GENAI.models.generateContent({
    model: MODEL,
    contents: PROMPT
  });
  console.log(response.text);
}

main();

Node.js on GCP

import 'dotenv/config';
import { GoogleGenAI } from '@google/genai';

const PROMPT = 'Describe a cat in a few sentences';
const MODEL = 'gemma-4-26b-a4b-it-maas';
console.log(`** GenAI text: '${MODEL}' model & prompt '${PROMPT}'\n`);

const CONFIG = JSON.parse(process.env.GCP_METADATA);
const GENAI = new GoogleGenAI({ vertexai: true, ...CONFIG });

async function main() {
  const response = await GENAI.models.generateContent({
    model: MODEL,
    contents: PROMPT
  });
  console.log(response.text);
}

main();

Above are the ECMAscript modules. All config files and source code can be found in the repo (links below), including the Node.js CommonJS scripts if you prefer those.

Config files

The config files include the current client library packages and apply to both platforms. Be sure to copy the settings file for your platform without the _TMPL and update the values as appropriate for your API key or GCP metadata.

Script files

To keep this brief and not fully repeat content, see the previous post covering the unified client library Gemini 2.5 post for full details including:

1. Installing API client libraries
2. Set up auth credentials
3. Cost considerations
4. Configuration files

Updating model usage in samples

If you intend to update earlier samples from using Gemini 2.5 to 3.8, the updates are as simple as these in Python and Node.js:

< MODEL = 'gemini-2.5-flash'
---
> MODEL = 'gemini-3.8-flash'

< const MODEL = 'gemini-2.5-flash';
---
> const MODEL = 'gemini-3.8-flash';

It's a similar change when moving to Gemma. We're taking only small steps, and while you can download Gemma and run it locally, that will be the subject of the next post. Today's goal is to move you to Gemma quickly and seamlessly by having you stay on the Gemini API and only doing a quick model change:

< MODEL = 'gemini-3.8-flash'
---
> MODEL = 'gemma-4-26b-a4b-it'

< const MODEL = 'gemini-3.8-flash';
---
> const MODEL = 'gemma-4-26b-a4b-it';

Open-weights model versions & naming

How did we choose the model? And what kind of name is gemma-4-26b-a4b-it (or gemma-4-26b-a4b-it-mass). Gemma is one of many open-weights models, and we chose it because it's also created by Google DeepMind, and Gemma is the closest open-weights model in similarity to Gemini. There are many other popular open-weights model families, and you may be familiar with these:

• DeepSeek
• Gemma (Google)
• GLM (Z.ai)
• Granite (IBM)
• Kimi (Moonshot.ai)
• Llama (Meta)
• Mistral
• Nemotron (NVIDIA)
• Qwen (Alibaba)

Model variants based on sizes

With frontier models, you just pick one and a version, e.g., Claude Opus 5.5, Fable 5.1, etc. You get what the vendor gives you. There are plenty more options in the open-weights world. For example, even with Gemma 4, you have a choice between 5 different parameter sizes (and many quantization choices for each), from 2 to 31 billion: E2B, E4B, 12B, and 31B as the regular dense models, and a fifth, 26B (A4B), that's an MoE (mixture-of-experts) model.

If you're curious about each one, take a look at the models overview as well as the benchmarks pages.

Model variants have quantized versions

Furthermore, the Gemma 4 models default to a standard precision of 16-bit, however to a variety of hardware, they're available with different lower-precision quantization levels. As briefly introduced above, that is the process of reducing a model's memory footprint so it can run on less capable, lower cost devices. Reducing the size lowers the precision, so it's up to your use case. Google's documentation explicitly states:

"The different sizes and precisions represent a set of trade-offs for your AI application. Models with higher parameters and bit counts (higher precision) are generally more capable, but are more expensive to run in terms of processing cycles, memory cost and power consumption. Models with lower parameters and bit counts (lower precision) have less capabilities, but may be sufficient for your AI task." (SOURCE: Gemma page on parameter sizes and quantization)

How to understand model IDs

When you see the model ID, gemma-4-26b-a4b-it, each delimited value has a different meaning. It is a Gemma 4 model, specifically the one with 26 billion parameters, it's a MoE model with 4 billion active parameters, and is further fine-tuned specifically to be able to follow your instructions... it means instruction-tuned.

Choosing what derivative of an open-weights model is out-of-scope for this post, but it's a good option for an average engineering laptop like my Apple Macbook Pro (M3 Pro MBP), giving me a model that's slightly smaller than the largest 31 billion parameter model but as a 26-billion parameter MoE model with 4 million active parameters, moves almost as fast as the actual 4 billion parameter model.

Not convinced about Gemma?

Because of Gemma's relationship with Gemini, some commonly-occurring skeptical comments about it have cropped up, like these:

1. "Gemma is just a smaller Gemini"
2. "Gemma is less capable, so it's worse"
3. "If I use Gemma, Google can still see my data"
4. "Gemini is always faster"

Check out this post where another blogger "mythbusts" those misconceptions.

There is one caveat to #3 above though: if you use the Gemini API on the free tier to access Gemma, Google does see your data, and you consent to it possibly being used to finetune future versions. This is itemized in the Google AI pricing page as: "Content used to improve our products." (Note that for the paid tier, this changes to: "Content not used to improve our products."

Another misconception is that Gemma is a direct distillation or compressed clone of Gemini. According to DeepMind, "Gemma is a family of lightweight, state-of-the-art open models built from the same research and technology used to create the Gemini models." (source) To be fully transparent, knowledge distillation does play a significant role in how parts of the Gemma family are trained. The why and how it was introduced is found in the Gemma 2 research paper.

Wrap-up

So while starting with Gemma above is fairly quick -- we only needed to change the model ID -- we're not done yet. Yes, you're (still) using the Gemini API, so yes, you're still sending data to Google and possibly incurring token costs. However it's the smallest, easiest step to take, and you got started with an open-weights model. In an upcoming post, we'll run Gemma 4 locally.

If you found an error in this post, a bug in the code, or have a topic you want me to cover in the future, drop a note in the comments below or file an issue at the repo. Also check out other posts in the Gemini API series as well as the GCP serverless series. Thanks for reading, and I hope to meet you at an upcoming event soon... see the travel calendar at the bottom of my consulting site.

PREV POST: Gemini 2.5 API missing manual

References

Below are various links relevant to this post as well as Gemini, Gemma, DeepMind, Gemini API, and even pricing.

Code samples

• Sample in this post (Python & Node.js)
• Code samples for Gemini posts (except this one)
• Code samples for all posts

Gemini API (Google AI) general

• General GenAI docs
• Gemini API docs
• API reference
• API SDKs/supported languages
• API quickstart
• Jupyter Notebook QuickStart
• Gemini API pricing (free tier available)
• Google AI migration guide
• GCP Vertex AI for Google AI users

Gemini API client libraries

• OLD GAI client library
• NEW GAI GenAI client library
• OLD GCP Vertex AI/AI Platform client library
• NEW GCP GenAI client library

Other Gemini API content by the author

• Gemini 2.5 API missing manual post
• Other posts in Gemini API series

Google DeepMind models

• Gemini home page
• Gemini Flash models
• Gemma home page
• All DeepMind models

Gemini models

• Developer overview
• Cookbook repo

Gemma

• Gemma launch) post
• Using Gemma from Vertex
• Gemma docs
• Gemma 4 launch post
• Gemma 4 (latest) model card

Pricing

• Google AI (GAI) Gemini pricing
• Google AI (GAI) Gemma pricing
• Google Cloud (GCP) Gemini pricing
• Google Cloud (GCP) Gemma pricing
• All other (GCP) Gemini Enterprise Agent Platform pricing

Non-Google AI links

• Open Source AI Definition
• Open Weights vs Open Source: The Real Difference
• IBM "What are model parameters?" post

^ — Originally launched in 2016 as Google Cloud Machine Learning, quietly renamed to "AI Platform" that summer, unified in 2021 with the "building block" APIs backed by pre-trained predictive models and renamed as "Vertex AI". In Spring 2026, it received its 4th name, Gemini Enterprise Agent Platform, and possibly its 5th as simply "Gemini platform" for short a mere half-year later when Gemini 4 Argon became available on GCP. Google isn't the best at naming, but it sure does a lot of renaming... same story with Google Workspace (2020), formerly G Suite (2016), formerly Google Apps (2007). Hey, I can't help it if promotions are based on launches!

DISCLAIMER: I was a member of the GCP serverless product team 2009-2013 and 2020-2023. While product information is as accurate as I can find or recall, the opinions are my own.

WESLEY CHUN, MSCS, is a Google Developer Expert (GDE) in Google Cloud (GCP) & Google Workspace (GWS), author of Prentice Hall's bestselling "Core Python" series, co-author of "Python Web Development with Django", and has written for Linux Journal & CNET. He's currently an AI Technical Program Manager at Red Hat focused on upstream open source projects that make their way into Red Hat AI products. In his spare time, Wesley helps clients with Google integrations, App Engine migrations, and Python training & engineering. He was one of the original Yahoo!Mail engineers and spent 13+ years on various Google product teams, speaking on behalf of their APIs, producing sample apps, codelabs, and videos for serverless migration and GWS developers Wesley holds degrees in Computer Science, Mathematics, and Music from the University of California, is a Fellow of the Python Software Foundation, and loves to travel to meet developers worldwide. Follow he/him @wescpy on Tw/X, BS, and his technical blog. Find this content useful? Contact CyberWeb for professional services or buy him a coffee (or tea)!