OpenClaw and Ollama: run local models

OpenClaw can run entirely on a local model: install Ollama, pull a model that supports tool calling, and point OpenClaw at the native Ollama endpoint on port 11434 with the provider setting api: "ollama". The rule that trips most people up is in the official docs: do not use the OpenAI-compatible /v1 URL for Ollama, because it breaks tool calling and the model starts printing raw tool-call JSON as text. A 9B-class model such as qwen3.5:9b or gemma4 fits a 12-16 GB GPU and gives you a private assistant with zero per-message cost; on CPU-only hardware it works, but you will wait.

Why run OpenClaw on a local model

OpenClaw is a personal AI agent gateway: it sits between a messaging app (Telegram in our setup) and an LLM, and runs tools such as shell commands, a browser or MCP servers on the model's behalf. Point it at Ollama and every prompt, file and tool result stays on hardware you control, with zero per-token cost and no provider rate limits. The price is capability: a model that fits on one GPU is weakest at exactly what makes an agent an agent, choosing a tool and producing a well-formed call.

Step 1: install Ollama and pull a tool-capable model

On Linux, the official installer sets Ollama up as a systemd service on 127.0.0.1:11434:

curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen3.5:9b
curl http://localhost:11434/api/tags

The last command lists the models on disk; if it answers, the daemon is up. Ollama also ships as a Docker image: docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama, adding --gpus=all for NVIDIA cards, or for AMD the ollama/ollama:rocm tag together with --device /dev/kfd --device /dev/dri.

Model choice matters most. OpenClaw's setup page states the minimum: tool support and at least 16K tokens of context. Ollama's own OpenClaw integration page recommends at least 64K for local models. Each Ollama library page shows capability badges; look for "tools" (ollama.com/search can filter by it). As of September 2026, qwen3.5 (vision, tools, thinking) and gemma4 (vision, tools, thinking, audio) both qualify; gemma4 is the default the OpenClaw setup page suggests. A model without the tools badge chats fine and silently fails the moment OpenClaw asks it to run something.

Step 2: point OpenClaw at Ollama (native API, not /v1)

When Ollama and the gateway share a machine, the fastest path is discovery:

export OLLAMA_API_KEY="ollama-local"
openclaw models list --provider ollama
openclaw models set ollama/qwen3.5:9b
openclaw models status

Per the model-discovery docs, when OLLAMA_API_KEY is set and no explicit provider block exists, OpenClaw queries http://127.0.0.1:11434, reads the catalogue from /api/tags and asks /api/show for each model's context window and its vision, tools and thinking flags. The value ollama-local is a placeholder for local or LAN hosts; the docs say any value works there.

For anything beyond the default (another host, pinned context, timeouts), declare the provider explicitly in ~/.openclaw/openclaw.json. The docs read that file as JSON5, so plain JSON like the block below is always accepted. This block follows the official recipe:

{
  "models": {
    "providers": {
      "ollama": {
        "baseUrl": "http://127.0.0.1:11434",
        "apiKey": "ollama-local",
        "api": "ollama",
        "timeoutSeconds": 300,
        "maxTokens": 8192,
        "models": [
          {
            "id": "qwen3.5:9b",
            "name": "qwen3.5:9b",
            "reasoning": true,
            "input": ["text"],
            "contextTokens": 32768,
            "params": { "num_ctx": 32768, "thinking": false, "keep_alive": "15m" }
          }
        ]
      }
    }
  },
  "agents": {
    "defaults": {
      "model": { "primary": "ollama/qwen3.5:9b" }
    }
  }
}

Three details matter. First, baseUrl has no /v1 suffix. Ollama does expose an OpenAI-compatible endpoint and most third-party tutorials use it, but the OpenClaw docs are explicit: /v1 selects OpenAI-compatible mode, where tool calling is unreliable and models can emit raw tool-call JSON as text. Native mode talks to Ollama's /api/chat, which handles streaming and tool calling together. Second, a non-empty models list disables discovery, so list every model you want. Third, model references are provider/model: ollama/qwen3.5:9b.

The gateway hot-applies many config changes, but the docs say some need a restart, so restart it after a provider change (in our deployments it runs as the ghcr.io/openclaw/openclaw container). Then verify end to end with the probe from the local-models docs:

openclaw infer model run --gateway --model ollama/qwen3.5:9b --prompt "Reply with exactly: pong" --json

Which models fit which RAM and VRAM

Download sizes and the quoted VRAM figures come from the Ollama library and Ollama's OpenClaw integration page, September 2026; the "realistic home" column is our estimate from those numbers. Download size is the quantised weight file; you need that much memory plus room for context, and a 32K-64K context on a 9B model adds several gigabytes.

Two checks. ollama ps shows a PROCESSOR column: "100% GPU" means the whole model sits in VRAM, a mixed figure means it is split with system RAM and speed falls off a cliff. And the OpenClaw local-models docs warn that a model which answers a short prompt may still fail an agent turn; test with a real task.

Where Ollama runs: same VM or a separate GPU box

Same VM, CPU only. The cheapest setup; a 4B-9B model with a small context works, but expect replies in tens of seconds and tool-heavy turns in minutes, depending on the CPU. Fine for learning the stack before paying for GPU time. One trap when the gateway runs in Docker, as ours does: inside the container, 127.0.0.1 is the container, not the VM, so http://127.0.0.1:11434 gets "connection refused"; the Ollama troubleshooting page in the OpenClaw docs lists exactly this case, a baseUrl pointing at localhost while the gateway runs in Docker or on another host. Either run Ollama as a second container on the same Docker network and address it by name, or bind Ollama to a non-loopback address, as Ollama's FAQ documents for systemd:

sudo systemctl edit ollama.service
# add under [Service]:
Environment="OLLAMA_HOST=0.0.0.0:11434"
sudo systemctl daemon-reload && sudo systemctl restart ollama

A local Ollama daemon has no built-in authentication (that is why OpenClaw's placeholder key is never checked), so if you bind it to all interfaces, firewall port 11434 to the gateway host only. Never expose it, or the OpenClaw gateway port, to the internet: the gateway already had a one-click remote-code-execution CVE (CVE-2026-25253).

Separate GPU box on the LAN. This is the shape the OpenClaw recipes show: the gateway on a small always-on VM, inference on a machine with a proper GPU. Set baseUrl to that host, raise timeoutSeconds so a cold load does not time out, and pin keep_alive so the model stays resident (Ollama unloads after 5 minutes by default). The docs even show two Ollama providers at once, ollama-fast on a small machine and ollama-large on a GPU box as a fallback. If you do not own a GPU, a rented one with 16-24 GB of VRAM covers the 9B-27B range; see our AI servers and what server to run a local AI model.

Context window: the setting everyone misses

Ollama's default context length is 4096 tokens. That is enough to chat and not enough for an agent: OpenClaw's system prompt, tool definitions and history alone can exceed it, and then the model loses the start of the turn and tool use degrades. OpenClaw's floor is 16K; Ollama's OpenClaw page recommends 64K. Raise it globally with OLLAMA_CONTEXT_LENGTH=32768 in the Ollama service environment, or per model in openclaw.json as above, where contextTokens caps what OpenClaw sends and params.num_ctx tells Ollama how much to allocate (if num_ctx is absent, OpenClaw derives it from contextTokens, per the advanced page). Context costs memory, so this is the knob you trade against model size on a fixed GPU.

Latency and quality compared with hosted APIs

Hosted models win on raw capability and time to first token. A 9B local model wins on privacy, cost at volume and predictability: no rate limits, no deprecations, no bill for a runaway loop. Where it loses is the agent workload itself; the OpenClaw local-models page warns of malformed tool calls and oversized prompts on full agent turns. Two documented mitigations: enable Tool Search (tools.toolSearch: { "mode": "tools" }) so the model sees tool schemas on demand, and as a last resort set compat.supportsTools: false on the provider to make that model a plain chat bot.

Also easy to overlook: local models lack the hosted providers' safety filters. The docs say to keep tool permissions and prompt-injection defences appropriate for the model: allowFrom restricted to the owner, tools scoped per sender, and a sandbox for anything that executes.

Mixing a local model with a hosted one

OpenClaw supports per-agent providers, so "cheap local model for chat, hosted model for hard tasks" is achievable as a manual split, not an automatic difficulty router. Three documented ways:

Fallback chain. agents.defaults.model.fallbacks is tried in order when the primary fails. Local primary, hosted fallback: the hosted key is spent only when the local box is down. Resilience, not routing.

Per-agent models. Under agents.entries each agent carries its own model, and bindings map channels, accounts or specific peers to agents. A minimal split (model ids as shown in the OpenClaw docs; use whatever your hosted account offers):

{
  "agents": {
    "entries": {
      "chat": { "default": true, "model": "ollama/qwen3.5:9b" },
      "work": { "model": "anthropic/claude-sonnet-4-6",
                "tools": { "allow": ["exec", "read", "write"] } }
    }
  },
  "bindings": [
    { "agentId": "work", "match": { "channel": "telegram", "accountId": "work-bot" } }
  ]
}

The multi-agent page documents the match fields and their precedence (exact peer first, down through account and channel to the default agent) and shows this very pattern for Telegram: one bot per agent, matched on channel: "telegram" plus the accountId defined under channels.telegram.accounts. So: one bot on the local agent for everyday questions, one on the hosted agent with exec enabled.

Switch in the conversation. The /model chat command changes the model on the fly: /model ollama/qwen3.5:9b -s pins it for the current session, -a updates the agent default and -g the global default (both owner-only). Aliases under agents.defaults.models allow /model local or /model big.

Troubleshooting checklist

• Connection refused: ollama serve, then curl http://localhost:11434/api/tags; from a remote gateway, openclaw gateway status --deep. Check firewalls and the Docker loopback trap.
• Tool calls appear as plain text: you are on /v1. Remove the suffix and set "api": "ollama".
• No models available: pull the model, or list it under models.providers.ollama.models.
• Slow first reply: timeoutSeconds 300 and keep_alive 15m.
• Odd answers on long chats: maxTokens 8192, contextTokens and num_ctx 32768, and confirm with ollama ps that the model is fully on GPU.
• General health: openclaw doctor --fix.

If you would rather skip the gateway installation and start at step 1 of this article, our ready OpenClaw server (from 9.35 EUR per month) ships with the gateway preinstalled in Docker, locked to your Telegram account, tools in a sandbox; adding Ollama is then the config block above. For the wider cost picture, see self-hosted AI cost vs API.

Questions

Can I use Ollama's OpenAI-compatible /v1 endpoint with OpenClaw?

The OpenClaw docs say not to. The /v1 path puts the provider into OpenAI-compatible mode, where tool calling is unreliable and the model may print tool-call JSON as text. Use baseUrl without /v1 and "api": "ollama".

Which local model should I start with?

For a 12-16 GB GPU, qwen3.5:9b or gemma4; both carry the tools badge and both are named in the OpenClaw and Ollama docs. On CPU-only hardware start with qwen3.5:4b and a 16K context to see whether the latency is acceptable before buying GPU time.

Does OpenClaw route easy questions to the local model and hard ones to a hosted API automatically?

No. The documented mechanisms are a fallback chain (hosted model used only when the local one fails), per-agent models with channel or peer bindings, and the /model command to switch within a session. Smarter routing you build on top, for example a second bot bound to the hosted agent.

Why does a model that chats fine break when OpenClaw uses tools?

Usually one of three things: the model has no tools capability, the context is at Ollama's 4096 default so the tool schemas push the conversation out of memory, or the provider is on the /v1 path. Fix those in order; if a small model still produces malformed calls, enable Tool Search or set compat.supportsTools: false.