Mistral Small

What Mistral Small is for

Mistral Small is a general-purpose open-weights model from Mistral AI, released in 2025-03 with 2 sizes (22B, 24B) and 32K of context, licensed MRL License (open-weights). The catalogue records its strength as sub-30B quality on consumer hardware.

Sizes and memory

The 24B needs about 15 GB of weights at Q4_K_M. That is the whole range; there is no larger or smaller variant in this record. Recommended VRAM adds roughly half again on top of the weights for the KV cache and the runtime.

Size	Weights at Q4_K_M	Recommended VRAM	Smallest card in our guide that holds it
24B	15 GB	23 GB	NVIDIA RTX 3090

Hardware that runs Mistral Small

11 of the 19 cards in our hardware guide hold at least one size of Mistral Small at Q4_K_M. The cheapest that does is the NVIDIA RTX 3090 at about $700, running the 24B.

The largest size any of them holds is the 24B at 15 GB, on the NVIDIA RTX 3090 (24 GB at 936 GB/s). Bandwidth, not capacity, sets the speed once a size fits, so a card further down the list with faster memory will generate more tokens per second on the same weights than a larger, slower one.

What people use it for

The catalogue lists 4 recorded uses for Mistral Small: chat, RAG, tools, agentic. It is tagged chat, tools, instruction.

Where Mistral Small sits in the Mistral family

Our catalogue holds 4 Mistral entries. Ordered by release date, Mistral Small (2025-03) is the 4th of them, and the most recent.

It ships under the MRL License (open-weights), which is neither a standard open-source licence nor a closed one. Read the terms before shipping anything commercial on top of it.

Context window: 32K

32K tokens is about 24,000 words at the usual ratio — enough for a handful of files or a long report held whole. It is the point at which the KV cache starts to be a serious share of memory: doubling the context you actually use costs roughly double the cache, on top of the 15 GB of weights.

How to run Mistral Small locally

Our catalogue does not record registry tags, so look the current tag up in the Ollama library or on Hugging Face before pasting these. The commands below are the shape of the workflow, not a copy-and-paste recipe.

Option 1: Ollama (simplest)

# find the tag for Mistral Small at ollama.com/library
ollama run <tag>

Option 2: Mullama (production)

mullama pull <tag>
mullama run <tag>

Option 3: llama.cpp (CLI)

# download a GGUF from Hugging Face, searching for "Mistral Small GGUF"
./llama-cli -m model.Q4_K_M.gguf -p "Hello, AI!"

Option 4: Python with Mullama or llama-cpp-python

from mullama import Model, Context
model = Model.load("model.Q4_K_M.gguf", n_gpu_layers=99)
ctx = Context(model, n_ctx=4096)
print(ctx.generate("Hello, AI!", 256))

Sources

Model record from src/data/models.json (2026-06-29); card capacities from src/data/gpus.json (2026-06-29). Upstream: Curated from ollama.com/library + community benchmarks + paperswithcode + huggingface.