- Start a vLLM server for local or lab testing.
- Connect an app using the OpenAI client pointed at your endpoint.
- Measure latency, tokens per second, and basic errors.
Minimal server
python -m venv .venv source .venv/bin/activate pip install -U vllm vllm serve Qwen/Qwen3-8B \ --host 127.0.0.1 \ --port 8000
OpenAI client with vLLM
import OpenAI from "openai";
const client = new OpenAI({
apiKey: "local",
baseURL: "http://127.0.0.1:8000/v1",
});
const response = await client.chat.completions.create({
model: "Qwen/Qwen3-8B",
messages: [{ role: "user", content: "Dame una checklist de despliegue LLM." }],
});
console.log(response.choices[0].message.content);