Ollama can allocate a context window based on server defaults, app settings, or request/model configuration. This guide checks the window that is actually loaded (CONTEXT) and whether the model stayed on the GPU (PROCESSOR), then changes only the setting that the running server actually honors.
If the input exceeds the allocated context window, Ollama or the client may truncate, compact, or discard earlier context depending on the runtime and client path. Verify the actual allocated context and inspect the logs rather than assuming the model-card maximum is active.
If `ollama ps` shows `100% CPU` even with a small context window, stop here and use the dedicated GPU troubleshooting lesson. This article does not diagnose basic GPU detection.
Ollama Not Using GPU on Windows
Connect Claude Code to Your Local AI
Quick diagnosis
Use the model the same way you normally do: CLI, API, or coding agent. Then inspect the loaded runner.
Command
ollama ps
Expected result
A row for the loaded model. On current Ollama versions, look especially for CONTEXT and PROCESSOR.
NAME ID SIZE PROCESSOR CONTEXT UNTIL model:latest ... ... 100% GPU 32768 ...
Do not depend on exact SIZE values for the diagnosis.
Decision
- Empty output: the model is not currently loaded. Send one short request, then run `ollama ps` again.
- No CONTEXT column: check `ollama --version` and make sure you are inspecting the same Ollama server that handles the request.
- CONTEXT is 4096 or 32768 and the workload needs 64K or more: go to Cause 1.
- CONTEXT remains at the server default after changing a client-side setting: go to Cause 2.
- CONTEXT is large and PROCESSOR becomes 100% CPU or a CPU/GPU split: go to Cause 3.
- CONTEXT matches the intended value and PROCESSOR remains 100% GPU, but early instructions or files disappear on large prompts: go to Cause 4.
- PROCESSOR is already 100% CPU with a small 4K/8K context: this is probably a GPU detection/offload problem, not a context problem.
The central diagnostic loop is:
ollama ps
↓
CONTEXT
What window is actually allocated?
↓
PROCESSOR
Did the model remain on GPU?Why Ollama's own documentation appears to disagree
Two official Ollama documentation pages currently describe the unconfigured default differently. The context-length page documents VRAM-based defaults: less than 24 GiB uses 4K, 24–48 GiB uses 32K, and 48 GiB or more uses 256K. It recommends at least 64000 tokens for web search, agents, and coding tools. The FAQ also states a 4096-token default.
Because behavior can vary by Ollama version, detected VRAM, app settings, and how the server was started, do not infer the active context size from documentation alone. The operational source of truth is the CONTEXT value shown by the runner serving the request.
ollama ps
Collect evidence before changing anything
Collect three facts: the advertised model context, the allocated CONTEXT, and the current PROCESSOR split.
Advertised model context
ollama show MODEL
Replace MODEL with the tag you actually use. Some models expose context information in their metadata. Treat this as the model's advertised or trained capability, not proof that the runtime allocated that window.
Allocated runtime window
ollama ps
- CONTEXT: the context allocated to the running model.
- PROCESSOR: whether execution is on GPU, CPU, or split.
Logs
Use logs only as supporting evidence. Ollama's output and warning wording can vary between releases.
Windows: open the current server log under:
%LOCALAPPDATA%\Ollama
macOS:
tail -n 200 ~/.ollama/logs/server.log
Linux with systemd:
journalctl -u ollama --no-pager -n 200
Use logs to identify which server started, configuration applied at startup, GPU offload behavior, and prompt-limit warnings when present. The absence of a particular warning is not proof that the advertised model context is active.
Cause 1: Ollama applied a smaller default context
What this means
The maximum context advertised by a model is not necessarily the context allocated by the Ollama server. A model capable of 128K or 256K may be running with CONTEXT 4096 or CONTEXT 32768, depending on the version, configuration, and available resources.
Check
ollama ps
Expected result
CONTEXT is smaller than the context required by the workload. Current Ollama guidance recommends at least 64000 tokens for agent and coding workloads when resources allow.
Fix: Ollama app
Use the context-length slider in Ollama settings. Restart the app after changing a server-level setting when required by the installed version.
Fix: foreground server on macOS or Linux
OLLAMA_CONTEXT_LENGTH=64000 ollama serve
This is macOS/Linux-style shell syntax. It only affects requests if this process is the Ollama server actually listening on port 11434. Do not start a second server and assume clients are using it.
Fix: Linux systemd
sudo systemctl edit ollama.service
Add:
[Service] Environment="OLLAMA_CONTEXT_LENGTH=64000"
Then reload the service configuration and restart the actual server:
sudo systemctl daemon-reload sudo systemctl restart ollama
Fix: macOS application environment
launchctl setenv OLLAMA_CONTEXT_LENGTH 64000
Quit and reopen the Ollama app. Do not assume this survives a reboot without a separately configured persistent launch environment.
Fix: Windows
- Quit Ollama from the taskbar or system tray.
- Open Edit environment variables for your account.
- Create or edit OLLAMA_CONTEXT_LENGTH with the value 64000.
- Start Ollama again from the Start menu.
Interactive Ollama session
/set parameter num_ctx 64000
This changes the interactive model/session behavior; it does not change every client or daemon default.
Native Ollama API
curl http://localhost:11434/api/generate -d '{
"model": "MODEL",
"prompt": "Reply with ok.",
"options": {
"num_ctx": 64000
}
}'Verify
ollama ps
Confirm that CONTEXT equals the intended value, then inspect PROCESSOR. If CONTEXT did not change, verify which process owns port 11434.
Linux:
ss -ltnp | grep 11434
Windows:
netstat -ano | findstr 11434
Why you cannot set Ollama context size through the OpenAI-compatible API
Ollama exposes several configuration paths, but they are not interchangeable. The native Ollama API accepts options.num_ctx; the server can use OLLAMA_CONTEXT_LENGTH; the app exposes a context setting; and ollama run supports /set parameter num_ctx.
The OpenAI API does not define a model context-size parameter equivalent to Ollama's num_ctx. A client using /v1/chat/completions therefore cannot assume that a client-side context-size field changed Ollama's allocation.
For a stable per-model setting behind the OpenAI-compatible endpoint, create a Modelfile:
FROM MODEL PARAMETER num_ctx 64000
ollama create model-64k -f Modelfile
Point the client at model-64k, run a request, and verify again with `ollama ps`.
Official Ollama OpenAI compatibility documentation
Cause 2: The client never changed num_ctx
What this means
The server is still using its default. This can happen when the client uses an OpenAI-compatible endpoint, exposes a context UI that does not control Ollama's allocation, talks to another instance, or the expected server was not restarted.
Check
ollama ps
Expected result
CONTEXT remains at the server default rather than the value expected from the client configuration.
Fix
Prefer OLLAMA_CONTEXT_LENGTH or the Ollama app setting. For a stable per-model configuration, use a Modelfile with PARAMETER num_ctx 64000.
Verify
Start the actual client or agent, run a request, then inspect NAME, CONTEXT, and PROCESSOR with `ollama ps`. Do not trust only the client's settings screen.
Cause 3: Increasing context caused CPU/GPU offload
What this means
A larger context needs more memory, particularly for the KV cache. Increasing context can turn PROCESSOR from 100% GPU into a CPU/GPU split. The model may still run, but performance can fall dramatically.
Check
Run `ollama ps` before and after increasing context. Record both CONTEXT and PROCESSOR.
Expected result
If the larger context no longer fits efficiently in VRAM, PROCESSOR may cease to show 100% GPU.
Fix
Do not immediately reduce an agent or coding workload to 32K without considering alternatives. If 64K causes CPU offload, reduce unnecessary parallelism, consider a smaller model or weight quantization, and consider KV-cache quantization where supported.
For a single workload, reducing parallelism may help. Adapt this service setting to the actual way Ollama is launched:
[Service] Environment="OLLAMA_NUM_PARALLEL=1"
KV-cache quantization
- f16: highest KV-cache memory requirement and default precision.
- q8_0: about half the f16 KV-cache memory in current Ollama documentation, with a small precision trade-off.
- q4_0: about one quarter of f16 KV-cache memory, with a greater possible quality trade-off.
KV-cache quantization requires Flash Attention and is currently server-wide. Confirm support for the active backend and model before enabling it.
OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0
Set these in the environment format appropriate to the server, then restart that server. Do not set 256K merely because the model advertises 256K if it causes significant CPU offload.
What if 64K still does not fit?
32K is possible, but coding agents and tool-heavy workloads may lose useful working context below Ollama's recommended 64K. Do not treat 32K as equivalent to 64K.
Verify
ollama ps
The goal is sufficient CONTEXT for the workload while keeping PROCESSOR at 100% GPU. If the system cannot satisfy both, make the trade-off explicit. If it remains 100% CPU after returning to a small context, use the GPU troubleshooting lesson.
Cause 4: The working set exceeds the allocated context
What this means
The effective working set can include system instructions, conversation history, user prompts, files, repository context, tool schemas, MCP definitions, and tool results. If these exceed CONTEXT, not all information can remain available. Exact truncation or compaction depends on the client and API path.
Check
Compare the CONTEXT shown by `ollama ps` with the material the client is attempting to keep active. Inspect server and client logs when available.
Expected result
The model may still return a fluent answer while failing to preserve earlier instructions or files because the allocated context is smaller than the working set.
Fix
- Increase context using Cause 1 when VRAM allows.
- Use Cause 3 if the increase causes CPU offload.
- Start a fresh session and remove unused tools or MCP servers.
- Shorten oversized instruction files and avoid repeatedly injecting irrelevant repository data.
Verify
Repeat a task that depends on an instruction placed early in the context, then inspect `ollama ps`. If it behaves correctly with a larger verified CONTEXT, the earlier allocation was insufficient for that working set. This does not prove one universal truncation mechanism.
Safety
- Restarting Ollama unloads models from memory. It does not delete the model.
- Do not set OLLAMA_HOST=0.0.0.0:11434 to solve a context problem. It exposes the service to the network and is unrelated to num_ctx.
- Debug logging can significantly increase log volume. Disable it after diagnosis.
- OLLAMA_KV_CACHE_TYPE and Flash Attention configuration can affect all models served by that Ollama process.
- Do not modify GPU drivers, firewall settings, or CUDA_VISIBLE_DEVICES as part of this context troubleshooting flow.
If it still doesn't work
- CONTEXT does not change after setting OLLAMA_CONTEXT_LENGTH: confirm that you restarted the process that actually owns port 11434. It may be the desktop app, foreground server, systemd, Docker, WSL, or native Windows service.
- CONTEXT increases but PROCESSOR leaves 100% GPU: the increase probably exceeded the efficient GPU memory budget. Use Cause 3.
- CONTEXT is correct and fully on GPU but a coding harness assumes another size: inspect the client's own compaction and context assumptions.
- Windows-native and WSL Ollama are both installed: make sure `ollama ps` reaches the same environment the client calls.
Do not reinstall Ollama merely to change a context configuration.
Primary sources
Ollama — Claude Code integration
Last reviewed: 30 August 2026.