SmolLM2

What SmolLM2 is for

SmolLM2 is a general-purpose open-weights model from HuggingFace, released in 2024-11 with 3 sizes (135M, 360M, 1.7B) and 8K of context, licensed Apache 2.0. The catalogue records its strength as small chat and tool use on any hardware.

Sizes and memory

The 135M needs about 0 GB of weights at Q4_K_M and the 1.7B about 1 GB, a spread of 1 GB across the family. The catalogue's default pick is 1.7B. Recommended VRAM adds roughly half again on top of the weights for the KV cache and the runtime.

Size	Weights at Q4_K_M	Recommended VRAM	Smallest card in our guide that holds it
135M	0 GB	0 GB	NVIDIA Jetson Orin Nano 8GB
360M	0 GB	0 GB	NVIDIA Jetson Orin Nano 8GB
1.7B	1 GB	2 GB	NVIDIA Jetson Orin Nano 8GB

Hardware that runs SmolLM2

Every one of the 19 cards with dedicated memory in our hardware guide holds at least one size of SmolLM2 at Q4_K_M. The cheapest that does is the NVIDIA Jetson Orin Nano 8GB at about $199, running the 1.7B.

The largest size any of them holds is the 1.7B at 1 GB, on the NVIDIA Jetson Orin Nano 8GB (8 GB at 68 GB/s). Bandwidth, not capacity, sets the speed once a size fits, so a card further down the list with faster memory will generate more tokens per second on the same weights than a larger, slower one.

What people use it for

The catalogue lists 4 recorded uses for SmolLM2: chat, edge, IoT, Raspberry Pi. It is tagged chat, small, edge, tools.

Where SmolLM2 sits in the SmolLM family

SmolLM2 is the only SmolLM entry in our catalogue, so there is no in-family alternative to weigh it against; the comparison to make is against the models listed at the foot of this page.

The Apache 2.0 licence puts no usage ceiling on it, which matters if the thing you are building has users. That is the practical difference between this and a vendor community licence.

Context window: 8K

8K tokens comes to roughly 6,000 words at the usual three-quarters-of-a-word-per-token ratio: a long chat, a single source file, one chapter. It is the band where a retrieval step stops being optional, because the interesting documents do not fit. The recommended VRAM figure above assumes a working context rather than the maximum, and filling this one will push past it.

How to run SmolLM2 locally

Our catalogue does not record registry tags, so look the current tag up in the Ollama library or on Hugging Face before pasting these. The commands below are the shape of the workflow, not a copy-and-paste recipe.

Option 1: Ollama (simplest)

# find the tag for SmolLM2 at ollama.com/library
ollama run <tag>

Option 2: Mullama (production)

mullama pull <tag>
mullama run <tag>

Option 3: llama.cpp (CLI)

# download a GGUF from Hugging Face, searching for "SmolLM2 GGUF"
./llama-cli -m model.Q4_K_M.gguf -p "Hello, AI!"

Option 4: Python with Mullama or llama-cpp-python

from mullama import Model, Context
model = Model.load("model.Q4_K_M.gguf", n_gpu_layers=99)
ctx = Context(model, n_ctx=4096)
print(ctx.generate("Hello, AI!", 256))

Sources

Model record from src/data/models.json (2026-06-29); card capacities from src/data/gpus.json (2026-06-29). Upstream: Curated from ollama.com/library + community benchmarks + paperswithcode + huggingface.