Which is better for on-device LLM on mobile in 2026?

For Flutter apps, Llamafu. For native iOS apps, MLC LLM has a slight edge on Apple Silicon. For native Android apps, both work but Llamafu is easier to integrate. For web apps, MLC LLM's WebGPU is the only choice.

What models can I run on a phone?

Both support any GGUF model (Llamafu) or any MLC-compiled model (MLC LLM). For phones, stick to 1B-3B models at Q4_K_M. Phi-3 mini, Llama 3.2 1B/3B, Qwen 2.5 0.5B/1.5B/3B are all good choices.

Does inference happen entirely on-device?

Yes. Both run the model on the device's CPU/GPU/NPU. No data leaves the phone. No API calls. Works offline.

On iPhone 15 Pro: 15-25 tok/s for Llama 3.2 1B Q4. On Pixel 8 Pro: 10-18 tok/s for the same model. Enough for chat UI but not for real-time voice.

MLC LLM vs Llamafu: on-device LLM on mobile in 2026

If you want to ship an LLM inside a mobile app in 2026, these are the two main options. MLC LLM is the academic / research-backed option with broad platform support. Llamafu is the Flutter / FFI option focused on the cognisoc family.

Overview

	MLC LLM	Llamafu
Owner	CMU + community	cognisoc (Srushta Media)
First release	2023	2025
Primary binding	Swift (iOS), Java/Kotlin (Android), JavaScript (Web)	Dart (Flutter), Android, iOS
Model format	MLC (own, compiled)	GGUF (universal)
Backend	Metal, OpenCL, Vulkan, WebGPU	llama.cpp (Metal, OpenCL, Vulkan)
Model coverage	curated set, needs compilation	any GGUF model
Apple Neural Engine	partial (experimental)	partial
Vision / multimodal	✓ (mlc-vlm)	✓ (LLaVA, Qwen2-VL)
Function calling	✓	✓
Structured JSON	✓	✓ (GBNF)
LoRA hot-swap	✗	✓
Flutter / Dart	via FFI	first-class
Web	✓ (WebGPU)	✗
License	Apache 2.0	MIT

When to use each

Choose MLC LLM when…

You are building a native iOS app in Swift, or a native Android app in Kotlin / Java.
You need WebGPU support (for web-based apps that run LLMs in the browser).
You want the broadest platform coverage including less-common targets (Raspberry Pi, smart TVs, embedded).
You are okay with the model compilation step (each model needs to be pre-compiled to MLC format).

Choose Llamafu when…

You are building a Flutter app (Llamafu is a first-class Flutter FFI plugin).
You want to use any GGUF model without a pre-compilation step.
You need LoRA hot-swap (load a new personality on the fly).
You want the simpler integration (single dependency, no model compilation).
You are already in the cognisoc ecosystem (mullama, llamafu, etc.).

Performance

Tokens per second on flagship phones, Llama 3.2 1B Q4:

Device	MLC LLM	Llamafu
iPhone 15 Pro	22	18
iPhone 14	16	14
Pixel 8 Pro	14	12
Galaxy S24 Ultra	12	11
OnePlus 12	11	10

On the same hardware, the two are within ~15% of each other. MLC has a slight lead on Apple Silicon because of its Metal-specific optimizations. Llamafu is more consistent across Android devices because llama.cpp has more hardware coverage.

Setup

MLC LLM (iOS, Swift)

// Package.swift
dependencies: [
    .package(url: "https://github.com/mlc-ai/mlc-swift", branch: "main"),
]

// Use
let engine = MLCEngine()
try await engine.reload(modelPath: "Llama-3.2-1B-q4f16_1-MLC")
let response = try await engine.chat(
    messages: [ChatMessage(role: .user, content: "Hello")]
)

MLC LLM (Android, Kotlin)

// build.gradle
implementation("ai.mlc:mlc-chat-android:0.2.0")

// Use
val engine = MLCEngine()
engine.reload(modelPath = "Llama-3.2-1B-q4f16_1-MLC")
val response = engine.chat("Hello")

Llamafu (Flutter)

import 'package:llamafu/llamafu.dart';

final llamafu = await Llamafu.init(
  modelPath: '/path/to/llama-3.2-1b-q4_k_m.gguf',
  threads: 4,
  contextSize: 2048,
  nGpuLayers: 99,  // offload all
);

final result = await llamafu.complete(
  prompt: 'Hello, AI!',
  maxTokens: 256,
);

Llamafu (Android, Java/Kotlin)

implementation("com.cognisoc:llamafu-android:0.1.0")

Llamafu (iOS, Swift)

.package(url: "https://github.com/cognisoc/llamafu", branch: "main")

Model preparation

	MLC LLM	Llamafu
Pre-compiled models	mlc-ai/models on HF	none (any GGUF)
Compile from HF	`mlc_llm.convert` (CLI)	none
Compile time	5-30 min per model	0
Custom fine-tunes	needs recompile	drop in GGUF

If you are using a fine-tuned model or a new release the same day it ships, Llamafu is the clear winner — no compilation step.

Decision matrix

Your situation	Use
Flutter app	Llamafu
Native Swift iOS app	MLC LLM (slight perf edge)
Native Kotlin Android app	either, Llamafu is simpler
Web app (WebGPU)	MLC LLM
Need LoRA hot-swap	Llamafu
Using a custom fine-tuned model	Llamafu (no compile)
Using an off-the-shelf model	MLC LLM (slight perf edge)
Need the broadest model coverage	Llamafu (any GGUF)
Embedded / smart TV / Pi	MLC LLM
Apple Neural Engine	MLC LLM (experimental)
Want production-grade stability	Llamafu (uses llama.cpp, battle-tested)

MLC LLM vs Llamafu: On-Device LLM on Mobile in 2026