LessonThe RTX 3090 remains a popular choice for its 24 GB of VRAM and used-market price, but a homelab isn't just about buying GPUs: you need a power supply, case, heat management, noise control, drivers, electrical safety, and realistic expectations.
- Calculate whether a used RTX 3090 makes sense for your use case.
- Understand single-GPU and multi-GPU limits without NVLink.
- Design a homelab you can maintain without the headache.
When it makes sense
- You want to learn local serving, llama.cpp, Docker GPU, and observability.
- You'll run medium-sized models frequently.
- You need privacy or predictable costs.
- The maintenance is worth it compared to paying for APIs or subscriptions.
When it doesn't
- You want zero maintenance.
- You can't tolerate noise, heat, or power consumption.
- You only need AI for a few hours a month.
- You don't want to debug drivers, power supplies, or containers.
TerminalCode
# llama.cpp multi-GPU: ejemplo conceptual
./llama-server \
-m ./models/modelo.gguf \
--n-gpu-layers 99 \
--split-mode layer \
--tensor-split 1,1 \
--ctx-size 8192 \
--port 8080
Buying checklist
- Real photos, mining history if available, and return policy.
- Power supply with sufficient headroom and proper connectors.
- Case with enough physical space and airflow.
- Wall outlet power consumption measurement.
- Backup plan and secure remote access.
Official sources
- llama.cpp: Using multiple GPUs
- llama.cpp: quantize README