AI Homelab with Used RTX 3090s

• Calculate whether a used RTX 3090 makes sense for your use case.
• Understand single-GPU and multi-GPU limits without NVLink.
• Design a homelab you can maintain without the headache.

When it makes sense

• You want to learn local serving, llama.cpp, Docker GPU, and observability.
• You'll run medium-sized models frequently.
• You need privacy or predictable costs.
• The maintenance is worth it compared to paying for APIs or subscriptions.

When it doesn't

• You want zero maintenance.
• You can't tolerate noise, heat, or power consumption.
• You only need AI for a few hours a month.
• You don't want to debug drivers, power supplies, or containers.

# llama.cpp multi-GPU: conceptual example
./llama-server \
  -m ./models/modelo.gguf \
  --n-gpu-layers 99 \
  --split-mode layer \
  --tensor-split 1,1 \
  --ctx-size 8192 \
  --port 8080

Buying checklist

• Real photos, mining history if available, and return policy.
• Power supply with sufficient headroom and proper connectors.
• Case with enough physical space and airflow.
• Wall outlet power consumption measurement.
• Backup plan and secure remote access.

Official sources

• llama.cpp: Using multiple GPUs
• llama.cpp: quantize README