OLMo 2
Allen AI's fully open OLMo 2. Training data, code, and weights all released. Best for research reproducibility.
What OLMo 2 is for
OLMo 2 is a general-purpose open-weights model from Allen AI, released in 2024-11 with 2 sizes (7B, 13B) and 4K of context, licensed Apache 2.0. The catalogue records its strength as fully open training data + weights.
Sizes and memory
The 7B needs about 5 GB of weights at Q4_K_M and the 13B about 9 GB, a spread of 4 GB across the family. The catalogue's default pick is 7B. Recommended VRAM adds roughly half again on top of the weights for the KV cache and the runtime.
| Size | Weights at Q4_K_M | Recommended VRAM | Smallest card in our guide that holds it |
|---|---|---|---|
| 7B | 5 GB | 8 GB | NVIDIA Jetson Orin Nano 8GB |
| 13B | 9 GB | 14 GB | NVIDIA RTX 3060 12GB |
Hardware that runs OLMo 2
Every one of the 19 cards with dedicated memory in our hardware guide holds at least one size of OLMo 2 at Q4_K_M. The cheapest that does is the NVIDIA Jetson Orin Nano 8GB at about $199, running the 7B.
The largest size any of them holds is the 13B at 9 GB, on the NVIDIA RTX 3060 12GB (12 GB at 360 GB/s). Bandwidth, not capacity, sets the speed once a size fits, so a card further down the list with faster memory will generate more tokens per second on the same weights than a larger, slower one.
What people use it for
The catalogue lists 3 recorded uses for OLMo 2: research, reproducibility, fine-tuning. It is tagged open-source, chat, research.
Where OLMo 2 sits in the OLMo family
OLMo 2 is the only OLMo entry in our catalogue, so there is no in-family alternative to weigh it against; the comparison to make is against the models listed at the foot of this page.
The Apache 2.0 licence puts no usage ceiling on it, which matters if the thing you are building has users. That is the practical difference between this and a vendor community licence.
Context window: 4K
4K tokens is a short window by 2026 standards — on the order of 3,000 words at the usual three-quarters-of-a-word-per-token ratio. That is one conversation turn with a document, not a session with a codebase. Anything longer has to be retrieved and injected rather than held.
How to run OLMo 2 locally
Our catalogue does not record registry tags, so look the current tag up in the Ollama library or on Hugging Face before pasting these. The commands below are the shape of the workflow, not a copy-and-paste recipe.
Option 1: Ollama (simplest)
# find the tag for OLMo 2 at ollama.com/library
ollama run <tag>Option 2: Mullama (production)
mullama pull <tag>
mullama run <tag>Option 3: llama.cpp (CLI)
# download a GGUF from Hugging Face, searching for "OLMo 2 GGUF"
./llama-cli -m model.Q4_K_M.gguf -p "Hello, AI!"Option 4: Python with Mullama or llama-cpp-python
from mullama import Model, Context
model = Model.load("model.Q4_K_M.gguf", n_gpu_layers=99)
ctx = Context(model, n_ctx=4096)
print(ctx.generate("Hello, AI!", 256))Sources
Model record from src/data/models.json (2026-06-29); card capacities from src/data/gpus.json (2026-06-29). Upstream: Curated from ollama.com/library + community benchmarks + paperswithcode + huggingface.