Migrate a cloud agent to local

• Inventory cloud dependencies before migrating.
• Test compatibility of prompts, tools, and context.
• Decide what stays local and what keeps an external fallback.

migration_matrix:
  tasks:
    classify_email:
      target: "local"
      model: "ollama/qwen2.5:7b"
      risk: "low"
    draft_legal_reply:
      target: "hybrid"
      local_first: true
      cloud_requires_approval: true
    edit_codebase:
      target: "local_with_review"
      requires_tests: true

Tests before changing production

• Same input against cloud and local.
• Compare quality, latency, and format.
• Test minimal tool calling.
• Measure real and truncated context.
• Record failures that require fallback.

Official sources

• LiteLLM routing and load balancing
• LiteLLM reliability and fallbacks
• Ollama API
• vLLM OpenAI-compatible server