Tool calling with local models

• Distinguish reasoning failure, parser failure, and permissions failure.
• Test tools with a minimal case before using real repos.
• Choose when to use Ollama, Open WebUI, Hermes, MCP, or vLLM.

Layered diagnosis

• Model: does it know how to follow JSON/tool format?
• Runtime: do Ollama, Hermes, Open WebUI, or vLLM expect the same format?
• Parser: does it extract arguments or stick with free text?
• Permissions: does the tool exist and can it run?

Minimal test:
Available tool: get_time({ "timezone": "Europe/Madrid" })

Prompt:
"Use the get_time tool to tell me the time in Madrid.
Don't make up the time. If you can't call the tool, say so."

Expected result:
- structured call to get_time
- valid arguments
- response based on actual output

Common failures

• The model describes the tool in text but doesn't invoke it.
• It generates invalid JSON or with made-up fields.
• The runtime hides the error and it seems like "nothing happens".
• A quantization or conversion has degraded tool behavior.
• The tool executes arbitrary code without human review.

Checklist:
- Full prompt log
- Tool schema log
- Model output log before parser
- Parsed arguments log
- Actual execution log
- Human approval rule for writes

Official sources

• Ollama API
• Open WebUI: Tools & Functions
• MCP specification: tools
• Hermes Agent documentation