Name: AI for small businesses and freelancers
Availability: InStock

Design model groups by task, cost, and sensitivity.
Use fallbacks without hiding important errors.
Log when a task moved from local to cloud.

TerminalCode

model_list:
  - model_name: local-fast
    litellm_params:
      model: ollama/qwen2.5:7b
      api_base: http://localhost:11434

  - model_name: local-gpu
    litellm_params:
      model: openai/Qwen/Qwen2.5-14B-Instruct
      api_base: http://localhost:8000/v1
      api_key: none

  - model_name: frontier-review
    litellm_params:
      model: openai/gpt-4.1-mini
      api_key: os.environ/OPENAI_API_KEY

Practical policy

Local-fast: drafts, classification, summaries of non-sensitive documents.
Local-gpu: long tasks, internal RAG, and agents with more context.
Frontier-review: final review, difficult reasoning, or repeated local failure.

TerminalCode

routing_rules:
  sensitive_data: local_only
  max_local_retries: 2
  allow_cloud_fallback:
    - public_code_review
    - generic_planning
    - final_quality_check
  require_human_approval:
    - customer_data
    - legal_docs
    - financial_docs

Official sources

LiteLLM Docs
LiteLLM routing and load balancing
LiteLLM fallbacks
Ollama API