Mistral Large

What Mistral Large is for

Mistral Large is a large mixture-of-experts model: only a fraction of its parameters activate per token, but all of them have to be resident, so total size decides whether you can run it at all. Mistral AI published it in 2024-11 under MRL License with a single 123B size and 128K of context. The catalogue records its strength as frontier-tier chat on a single 96GB GPU.

Sizes and memory

The 123B needs about 70 GB of weights at Q4_K_M. That is the whole range; there is no larger or smaller variant in this record. Recommended VRAM adds roughly half again on top of the weights for the KV cache and the runtime.

Size	Weights at Q4_K_M	Recommended VRAM	Smallest card in our guide that holds it
123B	70 GB	105 GB	Apple M3 Ultra

Hardware that runs Mistral Large

5 of the 19 cards in our hardware guide hold at least one size of Mistral Large at Q4_K_M. The cheapest that does is the Apple M3 Ultra at about $4000, running the 123B.

The largest size any of them holds is the 123B at 70 GB, on the Apple M3 Ultra (192 GB at 800 GB/s). Bandwidth, not capacity, sets the speed once a size fits, so a card further down the list with faster memory will generate more tokens per second on the same weights than a larger, slower one.

What people use it for

The catalogue lists 4 recorded uses for Mistral Large: frontier chat, tools, agentic, multilingual. It is tagged chat, tools, multilingual, frontier.

Where Mistral Large sits in the Mistral family

Our catalogue holds 4 Mistral entries. Ordered by release date, Mistral Large (2024-11) is the 3rd of them, and Mistral Small (2025-03) is newer.

It ships under the MRL License, which is neither a standard open-source licence nor a closed one. Read the terms before shipping anything commercial on top of it.

Context window: 128K

128K tokens is roughly 96,000 words at the usual ratio, which is a small repository or a whole book. Two cautions come with a window this size: the KV cache at full length can rival the weights for memory, and models rarely use the far end of their advertised context as well as they use the near end. Treat it as headroom, not as a promise.

How to run Mistral Large locally

Our catalogue does not record registry tags, so look the current tag up in the Ollama library or on Hugging Face before pasting these. The commands below are the shape of the workflow, not a copy-and-paste recipe.

Option 1: Ollama (simplest)

# find the tag for Mistral Large at ollama.com/library
ollama run <tag>

Option 2: Mullama (production)

mullama pull <tag>
mullama run <tag>

Option 3: llama.cpp (CLI)

# download a GGUF from Hugging Face, searching for "Mistral Large GGUF"
./llama-cli -m model.Q4_K_M.gguf -p "Hello, AI!"

Option 4: Python with Mullama or llama-cpp-python

from mullama import Model, Context
model = Model.load("model.Q4_K_M.gguf", n_gpu_layers=99)
ctx = Context(model, n_ctx=4096)
print(ctx.generate("Hello, AI!", 256))

Sources

Model record from src/data/models.json (2026-06-29); card capacities from src/data/gpus.json (2026-06-29). Upstream: Curated from ollama.com/library + community benchmarks + paperswithcode + huggingface.