Fine-Tuning Apache-2.0

Unsloth

2-5x faster fine-tuning with 40% less VRAM. Single-GPU QLoRA champion. The fastest way to fine-tune a 7B-13B model on a 24GB GPU.

Website
Platforms: windowsmacoslinux

Unsloth is a fine-tuning library that makes QLoRA and LoRA training dramatically faster and lighter on a single GPU. The headline numbers are real: on Llama-3-8B QLoRA, Unsloth typically delivers 2-5x more steps/second than a vanilla HF trl run and uses 30-40% less peak VRAM, which is the difference between a 7B model fitting on a 16GB card or not. It achieves this by hand-writing the forward and backward kernels for the supported model families in Triton and CUDA, fusing operations, and replacing the HF trainer’s generic autograd path with a specialized one. The API stays Hugging Face-compatible — you still use SFTTrainer, you still produce standard LoRA adapters — so dropping Unsloth into an existing script is a few import changes. For solo practitioners and small labs training on one consumer GPU, it is the default in 2026.

Key Features

  • Hand-tuned kernels for popular families. Llama, Mistral, Qwen, Gemma, Phi, and DeepSeek have custom forward/backward implementations. Unsupported models fall back to standard HF transformers, so you never get a hard failure — just less speedup.
  • 2-5x throughput, ~40% less VRAM. The exact multiple depends on the model and sequence length, but QLoRA on a 7-13B model on a single 24GB RTX card is Unsloth’s sweet spot and where the speedup is largest.
  • Drop-in HF compatibility. FastLanguageModel.from_pretrained and SFTTrainer mirror the HF API. Adapters merge and export to GGUF in one call via save_pretrained_gguf, which is the single most useful feature for people targeting llama.cpp / Ollama deployment.
  • Free and Pro tiers. The open-source core covers the common cases. Unsloth Pro adds multi-GPU and longer-context optimizations for paid users, but the free library is genuinely complete for single-GPU work.

When to Use Unsloth

Use Unsloth when you are fine-tuning on a single GPU, your model is in the supported family list, and your method is LoRA or QLoRA. That covers the overwhelming majority of hobbyist and small-team runs. Switch to Axolotl when you need DPO/RLHF/full FT, multi-GPU via DeepSpeed, or a reproducible YAML config to publish alongside a model. Switch to LLaMA-Factory when you want a GUI. Unsloth is the speed champion within its lane, not the broadest tool.

Setup

# CUDA 12.1+, Python 3.10+ recommended
pip install "unsloth[cu121-torch230] @ git+https://github.com/unslothai/unsloth.git"

# Or the standard wheel set:
pip install unsloth
pip install --no-deps trl peft accelerate bitsandbytes

# Minimal QLoRA on Llama-3-8B:
python - <<'PY'
from unsloth import FastLanguageModel
from trl import SFTTrainer, SFTConfig

model, tok = FastLanguageModel.from_pretrained("unsloth/llama-3-8b-bnb-4bit",
                                               max_seq_length=8192,
                                               load_in_4bit=True)
model = FastLanguageModel.get_peft_model(model, r=16, target_modules=["q_proj","k_proj","v_proj","o_proj"])
trainer = SFTTrainer(model=model, tokenizer=tok, train_dataset=ds,
                     args=SFTConfig(output_dir="out", per_device_train_batch_size=2,
                                    max_seq_length=8192, num_train_epochs=3))
trainer.train()
model.save_pretrained_gguf("out-gguf", quantization_method="q4_k_m")  # -> Ollama-ready
PY

Output .gguf files can be dropped straight into an Ollama or KoboldCpp models directory.

See also