Aulafy
ESEN
Courses
Courses/Fine-tuning and post-training for LLMs/Axolotl for reproducible training
Lesson

Axolotl for reproducible training

When training stops being a one-off experiment, you need versionable configuration. Axolotl lets you describe the dataset, model, LoRA, and training in clear files.

  • Create a readable YAML configuration for fine-tuning.
  • Version datasets, hyperparameters, and checkpoints.
  • Separate quick experiments from reproducible pipelines.

Minimal reference YAML

TerminalCode
base_model: Qwen/Qwen3-4B-Instruct
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer

datasets:
  - path: data/train.jsonl
    type: alpaca

sequence_len: 2048
adapter: lora
lora_r: 16
lora_alpha: 32
lora_dropout: 0.05

learning_rate: 0.0002
num_epochs: 1
micro_batch_size: 1
gradient_accumulation_steps: 8
output_dir: outputs/soporte-qwen-lora

What to version

  • YAML config.
  • Dataset hash or version.
  • Exact base model.
  • Library versions.
  • Eval results.
  • Decision to publish or discard.
Fast SFT with UnslothEvals, Overfitting, and Regressions