Gemma 2
Google's open Gemma 2 family. Strong chat for the size; 2B is competitive with much larger models on simple tasks.
What Gemma 2 is for
Gemma 2 is a general-purpose open-weights model from Google, released in 2024-07 with 3 sizes (2B, 9B, 27B) and 8K of context, licensed Gemma License. The catalogue records its strength as Google-quality chat at small sizes.
Sizes and memory
The 2B needs about 2 GB of weights at Q4_K_M and the 27B about 17 GB, a spread of 15 GB across the family. The catalogue's default pick is 9B. Recommended VRAM adds roughly half again on top of the weights for the KV cache and the runtime.
| Size | Weights at Q4_K_M | Recommended VRAM | Smallest card in our guide that holds it |
|---|---|---|---|
| 2B | 2 GB | 3 GB | NVIDIA Jetson Orin Nano 8GB |
| 9B | 6 GB | 9 GB | NVIDIA Jetson Orin Nano 8GB |
| 27B | 17 GB | 26 GB | NVIDIA RTX 3090 |
Hardware that runs Gemma 2
Every one of the 19 cards with dedicated memory in our hardware guide holds at least one size of Gemma 2 at Q4_K_M. The cheapest that does is the NVIDIA Jetson Orin Nano 8GB at about $199, running the 9B.
The largest size any of them holds is the 27B at 17 GB, on the NVIDIA RTX 3090 (24 GB at 936 GB/s). Bandwidth, not capacity, sets the speed once a size fits, so a card further down the list with faster memory will generate more tokens per second on the same weights than a larger, slower one.
What people use it for
The catalogue lists 3 recorded uses for Gemma 2: chat, edge, low-resource. It is tagged chat, instruction, lightweight.
Where Gemma 2 sits in the Gemma family
Our catalogue holds 3 Gemma entries. Ordered by release date, Gemma 2 (2024-07) is the 1st of them, and Gemma 4 (2026-05) is newer.
It ships under the Gemma License, which is neither a standard open-source licence nor a closed one. Read the terms before shipping anything commercial on top of it.
Context window: 8K
8K tokens comes to roughly 6,000 words at the usual three-quarters-of-a-word-per-token ratio: a long chat, a single source file, one chapter. It is the band where a retrieval step stops being optional, because the interesting documents do not fit. The recommended VRAM figure above assumes a working context rather than the maximum, and filling this one will push past it.
How to run Gemma 2 locally
Our catalogue does not record registry tags, so look the current tag up in the Ollama library or on Hugging Face before pasting these. The commands below are the shape of the workflow, not a copy-and-paste recipe.
Option 1: Ollama (simplest)
# find the tag for Gemma 2 at ollama.com/library
ollama run <tag>Option 2: Mullama (production)
mullama pull <tag>
mullama run <tag>Option 3: llama.cpp (CLI)
# download a GGUF from Hugging Face, searching for "Gemma 2 GGUF"
./llama-cli -m model.Q4_K_M.gguf -p "Hello, AI!"Option 4: Python with Mullama or llama-cpp-python
from mullama import Model, Context
model = Model.load("model.Q4_K_M.gguf", n_gpu_layers=99)
ctx = Context(model, n_ctx=4096)
print(ctx.generate("Hello, AI!", 256))Sources
Model record from src/data/models.json (2026-06-29); card capacities from src/data/gpus.json (2026-06-29). Upstream: Curated from ollama.com/library + community benchmarks + paperswithcode + huggingface.