LLaMA-Factory

LLaMA-Factory is a fine-tuning framework with a built-in web UI for configuring and launching training runs. Where Axolotl asks you to hand-write YAML and Unsloth is a code library, LLaMA-Factory puts a Gradio dashboard in front of the whole process: you pick a base model from a dropdown, choose a training method (SFT, LoRA, QLoRA, full FT, DPO, KTO, ORPO, PPO, reward modeling, or continued pre-training), upload or select a dataset, set hyperparameters via sliders and fields, and click “Start”. Under the hood it is Hugging Face transformers / trl / peft / deepspeed, so the runs are equivalent to what you would write by hand — the UI just removes the boilerplate and the YAML errors. Its other distinguishing trait is breadth: it supports more model families and more training algorithms than any other single framework in 2026, and it also ships an evaluation tab (MT-Bench, AlpacaEval, CMMLU) so you can score a model without leaving the tool.

Key Features

• Full web-UI training workflow. The llamafactory-cli webui command opens a dashboard covering dataset selection, model config, training args, and launch. You can still drive it from YAML or CLI (llamafactory-cli train config.yml), but the UI is the selling point for non-researchers.
• Broadest method coverage. SFT, LoRA, QLoRA, full-parameter, DPO, KTO, ORPO, IPO, SimPO, PPO RLHF, reward modeling, and continued pre-training are all selectable from the same UI. No second framework needed when you move from SFT to preference tuning.
• Built-in evaluation. Run benchmark evaluations (MMLU, CMMLU, MT-Bench, AlpacaEval, BLEU/ROUGE) from the Evaluate tab. This closes the loop — train, merge, score, decide — inside one tool.
• Wide model and dataset support. Llama, Qwen, Mistral, Gemma, Yi, DeepSeek, ChatGLM, and dozens more are pre-registered, as are ShareGPT, Alpaca, ChatML, and the major open datasets. Adding a custom dataset is a one-row edit to data/dataset_info.json.

When to Use LLaMA-Factory

Use LLaMA-Factory when you want fine-tuning without writing training scripts or YAML — it is the best on-ramp for users coming from a UI background, for teams sharing a training server, and for anyone who wants to try several methods (SFT then DPO then KTO) without learning three libraries. Choose Axolotl when you need reproducible, version-controlled YAML configs for publishing runs. Choose Unsloth when you are on a single GPU and need maximum speed on a supported model. LLaMA-Factory’s breadth and UI come at a small performance cost relative to Unsloth, and its configs are less portable than Axolotl’s.

Setup

# Recommended: dedicated env, CUDA 12.1+, Python 3.10-3.11
pip install "llamafactory[torch,metrics]"

# Or from source for the latest:
git clone https://github.com/hiyouga/LLaMA-Factory
cd LLaMA-Factory
pip install -e ".[torch,metrics]"

# Launch the web UI:
llamafactory-cli webui        # opens http://localhost:7860

# Or run from a YAML config:
llamafactory-cli train examples/train_qlora/llama3_lora_sft.yaml

# Export / merge adapters to a checkpoint:
llamafactory-cli export examples/merge_lora/llama3_merge_lora.yaml

Datasets are registered in data/dataset_info.json; drop a JSONL in data/ and add a one-line entry to make it selectable in the UI. Trained adapters can be merged and converted to GGUF with llama.cpp for Ollama/KoboldCpp deployment.

See also

• Axolotl — YAML-first, reproducible-config alternative
• Unsloth — faster single-GPU QLoRA
• Fine-tuning a local model guide — choosing a framework