Claude Code on Ollama: Local Models, Same CLI, Real Limits

We already have an Ollama plus Open WebUI stack. That is a chat window. This is the other itch: keep the Claude Code CLI, kill the Anthropic invoice.

HackerNoon published the $0 recipe on 16 September. The technical fact is older. Ollama’s 16 January blog says v0.14.0 and later speak the Anthropic Messages API, which is what Claude Code already talks. You set two environment variables, pull a local coder, and the same claude binary starts writing files on your disk. No API key that bills. No subscription that bills. Tokens still cost electricity and patience.

Goose will swap models too. This tutorial stays inside the CLI you already muscle-memorized.

What you are actually wiring

Claude Code is a client. It thinks the other end is Anthropic. Ollama’s compatibility page is the adapter: /v1/messages, streaming, system prompts, multi-turn, vision, tools, tool results, extended thinking. If the local model cannot do tools, you do not have an agent. You have a chatbot with a misleading binary name. Marktechpost’s 18 September harness roundup quotes Goose’s own docs on that: no tool calling, chat completion only.

The env vars from the official page:

export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_BASE_URL=http://localhost:11434

The token is required and ignored. Do not put a real Anthropic key in the same shell. If both ANTHROPIC_API_KEY and the fake token exist, you will spend a morning debugging the wrong host.

HackerNoon also maps the three default Claude tiers so the CLI stops asking for Sonnet:

export ANTHROPIC_DEFAULT_HAIKU_MODEL=glm-4.7-flash
export ANTHROPIC_DEFAULT_SONNET_MODEL=glm-4.7-flash
export ANTHROPIC_DEFAULT_OPUS_MODEL=glm-4.7-flash

Only do that after you have pulled that model. If you are on 8GB, those three lines will just fail in a more confusing way.

Official shortcut, if your Ollama is new enough:

ollama launch claude

That prompts for a model, writes the config, and starts the CLI. --config writes without launching. I still want you to know the env vars. The day launch changes, the env vars are the contract.

Install the two binaries, then pick a model that fits RAM

HackerNoon’s install block is the usual three paths: curl the Ollama script on Linux, brew install ollama on macOS, npm install -g @anthropic-ai/claude-code for the CLI. The Ollama blog repeats the Claude Code installer: curl -fsSL into bash on macOS/Linux/WSL, irm into iex on PowerShell. I am not pasting a truncated curl. Use the current page. Then:

ollama serve

Leave it running. Pull one model, not five.

HackerNoon’s hardware table is the part most “run it local” posts skip.

Hardware	Model	Download	Note
8GB	qwen2.5-coder:7b	5GB	HumanEval around 76% in that writeup
16GB	glm-4.7-flash	13GB	30B MoE, 3B active, 198K context, 79.5% tool-calling in their agent numbers, ~12 tok/s on M3 Pro, ~15 on RTX 3080
16GB	qwen3-coder-next	12GB	70.6% SWE-bench, 80B/3B MoE
24GB+ VRAM	qwen3-coder:30b	19GB	Their “best local coder”
32GB RAM	devstral:24b	14GB	46.8% SWE-bench Verified
32GB+ Mac	qwen3.6:27b	17GB	77.2% SWE-bench

Ollama’s own coding recs are glm-4.7, minimax-m2.1, and qwen3-coder. They warn that Qwen 3 coder at 30B wants at least 24GB of VRAM to run smoothly, and more if you stretch context. gpt-oss:20b is the other local they list. Cloud tags like glm-4.7:cloud work with Claude Code too. Those are not $0. They are Ollama’s cloud. If the point of this afternoon is to stop sending code to someone else’s GPU, do not pull :cloud.

ollama pull glm-4.7-flash
# or
ollama pull qwen3-coder

If a tool hardcodes claude-3-5-sonnet, the docs give you a rename:

ollama cp qwen3-coder claude-3-5-sonnet

That is a local alias. It is not Sonnet. Do not file a bug when it writes a worse patch.

Context length is the silent failure

A coding agent that sees 4k tokens will loop, rewrite the same file, and look “dumb.” It is starving.

Marktechpost cites Ollama’s context-length defaults: 4k under 24 GiB of VRAM, 32k from 24 to 48, 256k at 48 GiB and up. The same page says agents and coding tools should get at least 64,000 tokens:

OLLAMA_CONTEXT_LENGTH=64000 ollama serve

The January blog says at least 32K for Claude Code, and that Ollama cloud models always run at full context. Locally you pay in VRAM. If 64k does not fit, drop the model size before you drop below 32k. A 7B at 64k beats a 30B that evicts the system prompt.

HackerNoon’s extra knobs, if the process is paging:

export OLLAMA_FLASH_ATTENTION=1
export OLLAMA_KV_CACHE_TYPE=q8_0
export OLLAMA_NUM_PARALLEL=1

They claim q8_0 saves about 15% VRAM. NUM_PARALLEL=1 cuts fragmentation. Flash attention needs a backend that actually implements it. If ollama serve errors, unset and move on.

OpenCode’s local note in that Marktechpost piece is worth stealing even if you never install OpenCode: if tool calls fail, raise num_ctx into the 16k–32k range. Same failure mode, different harness.

Run it on a repo you can throw away

cd /path/to/throwaway-project
claude --model glm-4.7-flash

Or qwen2.5-coder:7b on the 8GB machine, or qwen3-coder:30b if you actually have the 24GB card. Watch git diff. The CLI can still edit files. Local does not mean sandboxed. The hooks tutorial is how you constrain the Anthropic-hosted version. The same discipline applies here: do not point this at a dirty tree with secrets in .env.

What I would try first, in order:

1. Ask it to explain one function. If it cannot stay on that file, the model or the context window is wrong. Stop.
2. Ask for a failing test, then the fix. Tool calling has to work or you are pasting diffs by hand.
3. Only then let it touch two files.

HackerNoon’s SWE-bench numbers are advertising. 70.6% on qwen3-coder-next is not 70.6% on your tangled Django app. glm-4.7-flash’s 79.5% tool-calling score is why they call it the 16GB default. If tools no-op, you picked a chat model.

OpenHands, in the same Marktechpost list, tells people to start with Qwen3.6-35B-A3B as of 21 May 2026, with 24GB VRAM or an Apple Silicon Mac with 64GB unified memory. That is a different harness. Steal the hardware honesty, not the binary.

Switching back, and what $0 does not buy

To talk to real Claude again:

unset ANTHROPIC_BASE_URL
unset ANTHROPIC_AUTH_TOKEN
export ANTHROPIC_API_KEY="your-real-key"
claude --model claude-sonnet-4-5

Use whatever model string your current CLI accepts. The point is the unset. Leave BASE_URL pointing at 11434 and you will keep talking to Qwen while you think you opened your wallet.

$0 is the inference line. You still bought the GPU. You still wait on 12 tokens a second. You still review every edit, because a 30B MoE with 3B active is not the model Anthropic serves. Privacy is the actual win: the repo does not leave the machine. Speed and quality are the trade.

Do not run this on production credentials to “see if local is good enough.” Run it on a clone. If the clone’s tests go green and the diff is small, you learned something. If the agent spins for twenty minutes and rewrites your linter config, you also learned something, and it cost you power instead of a four-figure API bill.

What breaks, in the order to check

The CLI starts and immediately asks you to log in. ANTHROPIC_BASE_URL is not set in this terminal. Desktop apps, tmux panes, and VS Code’s integrated terminal do not inherit a .zshrc you edited after they launched. Print the vars. If BASE_URL is empty, you are about to pay Anthropic or fail auth.

The CLI starts, then hangs on the first tool. Context is 4k. Raise OLLAMA_CONTEXT_LENGTH and restart ollama serve. Restart, not SIGHUP-and-hope. Confirm with a small prompt that the model still loads. If it unloads, the 64k window does not fit. Drop to 32k or drop the model.

The CLI writes a patch that does not compile, then “fixes” it by deleting tests. That is a weak coder plus too much permission. Local does not add a human-in-the-loop unless you add one. Cline’s local guide, in the Marktechpost list, maps 16–32GB RAM to small quantized models and wants approvals on every edit. You can get the same habit by reading git diff before you type yes. If you have been auto-accepting on the paid CLI, do not import that vice to a 7B.

Ollama is listening on 11434 and something else on the machine is also a “Claude-compatible” proxy. One leftover BASE_URL from a previous experiment will send traffic into a docker container you forgot. curl -s http://localhost:11434/api/tags should list the model you pulled. If it lists nothing, you pulled on a different Ollama (Linux brew vs distro package vs Docker).

ollama launch claude configured a cloud tag because you clicked the shiny one. Check the model string in the CLI banner. :cloud in the name means the repo is leaving the house. The January blog is explicit that cloud models work with Claude Code. That is a feature. It is also how people accidentally undo the privacy reason they came here.

npm global Claude Code is a different version than the one Anthropic’s curl script installed last month. If --model is rejected, you are on an old CLI that does not pass the model through, or a new CLI that renamed flags. Read claude --help. Do not copy a 2025 gist.

What this is not

This is not a replacement for hooks, skills, and MCP on the paid CLI. MCP servers still need a model that can call tools. A 7B that cannot emit a valid tool JSON will sit there. Paid Claude plus MCP is a different machine. Local Claude Code is “the same keyboard shortcuts, a smaller brain, a closed network.”

It is also not the Open WebUI stack. WebUI is for asking questions and pasting answers. Claude Code is for a process that is allowed to run git and pytest. If you only needed a private chat, stay on WebUI and save the VRAM you would have spent on a 64k window.

Marktechpost’s other harnesses exist because people got tired of pretending. OpenCode can ollama launch opencode and wants 64k too. OpenHands publishes VRAM numbers instead of vibes. If Claude Code’s compatibility layer fights you for an afternoon, those are the exits. The Ollama server stays. Only the client changes.

The HackerNoon headline is true in the narrow sense. The CLI will start. The invoice will be empty. The model in the loop is whoever you pulled, and Ollama’s compatibility layer will not make it Sonnet.