Aulafy
ESEN
Courses
Courses/Fine-tuning and post-training for LLMs/Map: SFT, LoRA, QLoRA, and DPO
Lesson

Map: SFT, LoRA, QLoRA, and DPO

Fine-tuning is not the hammer for every nail. Before training, decide whether the problem can be solved with a better prompt, RAG, tools, clean data, or actual model adaptation.

  • Distinguish prompt engineering, RAG, SFT, LoRA, QLoRA, and DPO.
  • Choose the right technique based on cost, data, and goal.
  • Avoid training a model when you only need to retrieve information.

Quick decision

  • Prompt: you want to change format, tone, or simple instructions.
  • RAG: you want it to use updatable or private documents.
  • SFT: you want it to learn question-answer patterns from your domain.
  • LoRA: you want to train few parameters and save a lightweight adapter.
  • QLoRA: you want LoRA using quantization to reduce memory.
  • DPO: you want to align preferences by comparing good and bad responses.

Decision card

TerminalCode
objetivo: "responder emails de soporte con tono de marca"
datos_disponibles:
  ejemplos_buenos: 800
  ejemplos_malos: 120
  documentos_actualizables: true
mejor_opcion:
  - RAG para políticas que cambian
  - SFT/LoRA para tono y formato
no_entrenar_para:
  - memorizar precios
  - guardar contratos privados
  - sustituir permisos
High-quality instruction datasets