Claude Code with Ollama: Run Local Models on Your Machine

Claude Code is Anthropic's terminal-based coding agent, and it was built for Anthropic's own Claude models. So the first honest thing to say is this: pairing it with a local model is an unofficial setup, not a supported feature. It works because Claude Code lets you override the server it talks to. Point that server at Ollama — the popular tool for running open models like Llama, Qwen, and Gemma locally — and you get a fully local coding agent. If you are new to the tool itself, start with our primer on what Claude Code is and come back.

This guide walks through how the connection works, the exact setup, the honest trade-offs, and when running the two together is actually a good idea.

What "Claude Code with Ollama" actually means

Under the hood, Claude Code sends requests to an endpoint that speaks Anthropic's Messages API. Normally that endpoint is Anthropic's cloud. But Claude Code reads an environment variable, ANTHROPIC_BASE_URL, that tells it where to send those requests. Change that URL and you can redirect the whole agent somewhere else.

For a long time the catch was format. Claude Code speaks the Anthropic Messages API, while most local runtimes speak OpenAI's chat-completions format — so you needed a translation layer in between. That changed when Ollama shipped a native Anthropic-compatible endpoint. Now Ollama can accept Anthropic-style requests directly, and using Claude Code with Ollama no longer requires a separate proxy for the basic case.

So there are two ways to wire this up: point Claude Code straight at Ollama's Anthropic endpoint, or run a small proxy or router in the middle for more control. We'll cover both.

Why run Claude Code with Ollama?

Three reasons make people try it:

• Privacy. Your code and prompts never leave your machine. For regulated work, proprietary codebases, or air-gapped environments, that alone is the whole argument.
• Cost. After the hardware, local inference is free. No per-token API bills, no usage caps, and no meter running while you experiment.
• Offline. On a plane, in a secure facility, or with flaky internet, a local model keeps working when the cloud is unreachable.

None of those benefits come without trade-offs, which we get to below. But if any of them is a hard requirement, using Claude Code with Ollama is worth the setup.

How to set up Claude Code with Ollama

Here's the direct path, assuming you already have Claude Code and Ollama installed.

1. Pull a capable model. Choose a coding-tuned model that supports tool calling. Check Ollama's library for the exact tag; a recent Qwen coding model is a reasonable starting point:

ollama pull qwen3-coder

2. Set the environment variables. These three tell Claude Code to talk to Ollama instead of Anthropic:

export ANTHROPIC_BASE_URL=http://localhost:11434
export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=""

The base URL points at Ollama's local server (port 11434). The auth token is a placeholder Ollama expects, and the empty API key stops Claude Code from trying to use a real Anthropic key. Anthropic documents these override variables in Claude Code's setup docs.

3. Launch Claude Code with your model:

claude --model qwen3-coder

4. Confirm it works. Ask the agent something trivial — "list the files in this folder" — and watch it call the file-reading tool. If it reads files and answers from the local model, you're connected. If you hit an auth error, double-check that ANTHROPIC_API_KEY is empty and that Ollama is actually running with ollama serve.

Give the model a generous context length (64k tokens or more if your hardware allows) so it can hold enough of your repository to behave like a real agent. On Windows, set the variables with setx or in your shell profile rather than export.

What hardware you'll need

Local models are memory-hungry, and the model has to share your GPU's VRAM with a large context window. As a rough guide:

More VRAM means you can load a stronger model and give it the 30k–50k tokens of context that serious agentic sessions need.

The proxy and router approach

The direct method is simplest, but a middle layer buys you flexibility. This is where a dedicated router earns its place. A claude-code-router setup can send different tasks to different models — a small local model for quick edits, a bigger cloud model for hard reasoning — all from one Claude Code session. LiteLLM plays a similar role and adds team features like load balancing, rate limiting, and audit logs.

You'd choose a proxy or router when you want to mix providers, fall back to the cloud automatically, or bridge older local runtimes that don't yet speak the Anthropic format. If you're mainly after cheaper cloud models rather than fully local ones, our guide to Claude Code with OpenRouter covers that route, and you can always change the model in Claude Code per session.

Here's how the options compare:

The real trade-offs of using Claude Code with Ollama

This is where honesty matters. Claude Code was tuned around the behavior of frontier Claude models, and swapping in a smaller local model changes the experience in ways that are easy to underestimate.

• Reasoning quality drops on hard tasks. Open models have closed much of the gap on average coding benchmarks — some now land within single-digit percentages of the best cloud models. But averages hide the pain. Tangled refactors, subtle concurrency bugs, and long multi-file changes are exactly where smaller models still stumble.
• Tool-use reliability slips. Claude Code is agentic: it reads files, runs commands, and edits code by calling tools in a loop. Smaller models call those tools less reliably, sometimes malforming arguments or giving up mid-task. A single dropped tool call can derail a whole session.
• Context and speed. Serious agentic sessions need tens of thousands of tokens of context. On a typical consumer GPU, a local model that holds that much context runs slower and can stall on large codebases.

A quick example: ask a small local model to rename a function used across twenty files and it may edit five, miss the rest, and report success anyway. A frontier model is far more likely to trace every reference and verify the result. In short, running Claude Code with Ollama trades peak capability for privacy, cost, and control. On routine edits it can feel great; on the gnarly 20% of work, you'll feel the difference.

When it makes sense — and the hybrid option

Reach for Claude Code with Ollama when privacy, offline capability, or zero marginal cost are genuine requirements, and when your work skews toward routine edits, boilerplate, and small, well-scoped changes. It's also a great way to learn how agentic coding works without watching a bill climb.

Reach for cloud Claude when you're doing hard architectural work, debugging something subtle, or running long autonomous sessions where every tool call has to land.

The most productive answer is usually hybrid: a local model for the everyday 80%, and a frontier Claude model for the 20% that earns its cost. Anthropic's lineup currently spans models like Claude Opus 4.8 and Claude Sonnet 4.6, and the latest of them still hold a clear edge on that hard 20% — which is exactly why keeping a cloud option open pays off. That's why comparing agents helps — seeing how other tools juggle local and cloud models, as in Claude vs Cline or OpenCode vs Claude Code, can clarify which one fits which job. You don't have to pick a single setup forever; you can switch per task.

Prefer LM Studio to Ollama? Claude Code with LM Studio covers the native Anthropic-compatible endpoint, so no translation proxy is needed.