BGE-M3

What BGE-M3 is for

BGE-M3 is an embedding model, not a chat model: it turns text into vectors so a retrieval system can find the passages worth showing a language model. BAAI released it in 2024-01 under MIT, with a single 567M size and 8K of context. It is the piece you run alongside a generator in a local RAG stack, and the catalogue records its strength as multilingual embeddings (100+ languages).

Sizes and memory

Our catalogue does not carry a per-size memory breakdown for BGE-M3. What it records instead is a floor of 1 GB and a recommended 2 GB for the default configuration. Until a per-size figure exists, treat those two numbers as the whole guidance rather than reading a size table that would have to be invented.

Hardware that runs BGE-M3

Without a per-size memory figure there is no honest way to say which card holds BGE-M3. Its recorded floor of 1 GB rules out every consumer card in our hardware guide, which tops out well below that.

What people use it for

The catalogue lists 3 recorded uses for BGE-M3: multilingual RAG, hybrid retrieval, long-document. It is tagged embedding, multilingual, RAG, retrieval.

Where BGE-M3 sits in the BGE family

BGE-M3 is the only BGE entry in our catalogue, so there is no in-family alternative to weigh it against; the comparison to make is against the models listed at the foot of this page.

The MIT licence puts no usage ceiling on it, which matters if the thing you are building has users. That is the practical difference between this and a vendor community licence.

Context window: 8K

8K tokens comes to roughly 6,000 words at the usual three-quarters-of-a-word-per-token ratio: a long chat, a single source file, one chapter. It is the band where a retrieval step stops being optional, because the interesting documents do not fit. The recommended VRAM figure above assumes a working context rather than the maximum, and filling this one will push past it.

How to run BGE-M3 locally

Our catalogue does not record registry tags, so look the current tag up in the Ollama library or on Hugging Face before pasting these. The commands below are the shape of the workflow, not a copy-and-paste recipe.

Option 1: Ollama (simplest)

# find the tag for BGE-M3 at ollama.com/library
ollama run <tag>

Option 2: Mullama (production)

mullama pull <tag>
mullama run <tag>

Option 3: llama.cpp (CLI)

# download a GGUF from Hugging Face, searching for "BGE-M3 GGUF"
./llama-cli -m model.Q4_K_M.gguf -p "Hello, AI!"

Option 4: Python with Mullama or llama-cpp-python

from mullama import Model, Context
model = Model.load("model.Q4_K_M.gguf", n_gpu_layers=99)
ctx = Context(model, n_ctx=4096)
print(ctx.generate("Hello, AI!", 256))

Sources

Model record from src/data/models.json (2026-06-29); card capacities from src/data/gpus.json (2026-06-29). Upstream: Curated from ollama.com/library + community benchmarks + paperswithcode + huggingface.