Ollama, vLLM, SGLang, and MLX: What to Use

• Choose a runtime based on use case: testing, RAG, agents, or production.
• Avoid premature migrations that add complexity without delivering value.
• Measure latency, memory, and errors before deciding.

Quick decision

• Ollama: get started fast, demos, desktop, courses, and prototypes.
• llama.cpp: fine-grained control, GGUF, simple CPU/GPU, and small deployments.
• vLLM: high request volume, compatible API, batching, and NVIDIA GPU.
• SGLang: advanced serving, structured flows, and complex workloads.
• MLX: Apple Silicon, efficient local experimentation on Mac.

Mide antes de opinar:

caso: "chat_rag_pyme"
modelo: "qwen-coder/local"
hardware: "Mac M4 24GB"
runtime: "Ollama"
metricas:
  first_token_ms: 850
  tokens_s: 28
  ram_gb: 11
  errores_20_preguntas: 2
decision: "suficiente para prototipo; no migrar todavía"

Official sources

• Ollama GitHub
• vLLM documentation
• SGLang documentation
• MLX documentation