Aulafy
Courses/Fine-tuning and post-training for LLMs/High-quality instruction datasets

High-quality instruction datasets

Fine-tuning quality depends more on the dataset than on the training command. A small, clean, and representative dataset beats thousands of noisy examples.

  • Create useful instruction examples for SFT.
  • Split train, validation, and test without contaminating results.
  • Remove sensitive data, duplicates, and mediocre responses.

Simple format

TerminalCode
{"instruction":"Clasifica el email","input":"Hola, quiero cambiar mi factura...","output":"categoria: facturacion\nprioridad: media\naccion: pedir numero de factura"}
{"instruction":"Redacta respuesta breve","input":"Cliente pide plazo de entrega","output":"Hola, gracias por escribir. El plazo estimado es..."}
{"instruction":"Extrae campos","input":"Presupuesto para 3 licencias anuales","output":"{"producto":"licencia anual","cantidad":3}"}

Cleaning checklist

  • No emails, phone numbers, national IDs, keys, or real names unless you have a legal basis.
  • No duplicate examples between train and test.
  • No contradictory responses for the same instruction.
  • Include real-world errors from the domain, not only perfect cases.
  • Include examples where the model should reject or ask for clarification.
TerminalCode
dataset/
  train.jsonl
  validation.jsonl
  test.jsonl
  README.md
  data_card.md
  redactions.log