Phi Microsoft Released 2024-04 chatlightweightsmall

Phi-3

Microsoft's small Phi-3 family. Mini (3.8B) is the best-in-class small chat; Medium (14B) competes with much larger models.

Best for small chat, edge deployment
Sizes Mini 3.8B · Medium 14B
Context 4K (Mini), 128K (Medium)
License MIT
Min VRAM (default size, Q4) 3 GB
Rec VRAM 6 GB

What is Phi-3?

Phi-3 is Microsoft's open-weights language model. Released in 2024-04, it ships in 2 sizes (Mini 3.8B, Medium 14B) and is licensed MIT. The most popular use case is small chat, edge deployment.

VRAM and hardware

The smallest Mini 3.8B size needs at least 3 GB VRAM; the largest Medium 14B needs around 3 GB.

Size Min VRAM (Q4_K_M) Recommended VRAM Tokens/sec on 3090
Mini 3.8B 3 GB 5 GB ~28 tok/s
Medium 14B 3 GB 5 GB ~28 tok/s

How to run Phi-3 locally

Option 1: Ollama (simplest)

ollama pull phi-3
ollama run phi-3

Option 2: Mullama (production)

mullama pull phi-3
mullama run phi-3

Option 3: llama.cpp (CLI)

# Download a GGUF from Hugging Face (search "phi-3 gguf")
./llama-cli -m phi-3.Q4_K_M.gguf -p "Hello, AI!"

Option 4: Python with Mullama or llama-cpp-python

from mullama import Model, Context
model = Model.load("phi-3.Q4_K_M.gguf", n_gpu_layers=99)
ctx = Context(model, n_ctx=4096)
print(ctx.generate("Hello, AI!", 256))

What you can build with Phi-3

  • chat
  • edge
  • mobile
  • low-resource

Hardware it fits on

  • Apple Silicon (8GB): Mini 3.8B in MLX
  • Apple Silicon (16GB): Medium 14B in MLX
  • Apple Silicon (32GB): Medium 14B in MLX or GGUF
  • Apple Silicon (64GB+): Medium 14B in MLX
  • RTX 3090 (24GB): Medium 14B at Q4_K_M
  • RTX 4090 (24GB): same as 3090, ~30% faster
  • RTX 5090 (32GB): Medium 14B at Q8_0
  • 2× RTX 6000 Ada (96GB total): Medium 14B at Q4_K_M

Related models in the Phi family

  • Phi-4 — 14B · strong 14B on reasoning benchmarks
  • Phi-4 Mini — 3.8B · small chat with function calling and tool use
  • Phi-4 Reasoning — 14B · strong reasoning in a 14B model

Other models you might consider

  • Llama 3.1 (Meta) — 8B, general-purpose chat, agentic workflows, code
  • Llama 3.2 (Meta) — 1B, low-end hardware, mobile, edge deployment
  • Llama 3.2 Vision (Meta) — 11B, vision + chat on desktop GPUs
  • Llama 3.3 (Meta) — 70B, best-in-class 70B for desktops
  • Llama 4 (Meta) — Scout 17B (109B MoE), vision + long context (10M tokens)
  • Qwen 2.5 (Alibaba) — 0.5B, strongest 7B-32B on consumer hardware