Llama TinyLlama team Released 2024-01 chatsmalledge

TinyLlama

1.1B Llama trained on 3T tokens. The smallest capable chat model. Runs on phones and Raspberry Pi.

Best for tiny chat on any hardware
Sizes 1.1B
Context 2K
License Apache 2.0
Min VRAM (default size, Q4) 1 GB
Rec VRAM 2 GB

What is TinyLlama?

TinyLlama is TinyLlama team's open-weights language model. Released in 2024-01, it ships in 1 size (1.1B) and is licensed Apache 2.0. The most popular use case is tiny chat on any hardware.

VRAM and hardware

The 1.1B size fits in 1 GB of VRAM.

Size Min VRAM (Q4_K_M) Recommended VRAM Tokens/sec on 3090
1.1B 1 GB 2 GB ~34 tok/s

How to run TinyLlama locally

Option 1: Ollama (simplest)

ollama pull tinyllama
ollama run tinyllama

Option 2: Mullama (production)

mullama pull tinyllama
mullama run tinyllama

Option 3: llama.cpp (CLI)

# Download a GGUF from Hugging Face (search "tinyllama gguf")
./llama-cli -m tinyllama.Q4_K_M.gguf -p "Hello, AI!"

Option 4: Python with Mullama or llama-cpp-python

from mullama import Model, Context
model = Model.load("tinyllama.Q4_K_M.gguf", n_gpu_layers=99)
ctx = Context(model, n_ctx=4096)
print(ctx.generate("Hello, AI!", 256))

What you can build with TinyLlama

  • chat
  • edge
  • IoT
  • Raspberry Pi

Hardware it fits on

  • Apple Silicon (8GB): 1.1B in MLX
  • Apple Silicon (16GB): 1.1B in MLX
  • Apple Silicon (32GB): 1.1B in MLX or GGUF
  • Apple Silicon (64GB+): 1.1B in MLX
  • RTX 3090 (24GB): 1.1B at Q4_K_M
  • RTX 4090 (24GB): same as 3090, ~30% faster
  • RTX 5090 (32GB): 1.1B at Q8_0
  • 2× RTX 6000 Ada (96GB total): 1.1B at Q4_K_M

Related models in the Llama family

  • Llama 3.1 — 8B, 70B, 405B · general-purpose chat, agentic workflows, code
  • Llama 3.2 — 1B, 3B · low-end hardware, mobile, edge deployment
  • Llama 3.2 Vision — 11B, 90B · vision + chat on desktop GPUs
  • Llama 3.3 — 70B · best-in-class 70B for desktops

Other models you might consider

  • Qwen 2.5 (Alibaba) — 0.5B, strongest 7B-32B on consumer hardware
  • Qwen 2.5-Coder (Alibaba) — 0.5B, best open-weight code model in 2024-2025
  • Qwen 3 (Alibaba) — 0.6B, best all-round local LLM in 2025-2026
  • Qwen 3-Coder (Alibaba) — 30B, long-context code agent workflows
  • DeepSeek R1 (DeepSeek) — 1.5B, reasoning, math, step-by-step problem solving
  • DeepSeek V3 (DeepSeek) — 671B (37B active MoE), frontier-class open-weights LLM (needs data-center GPUs)