TL;DR: Three Weeks With Local LLMs

Cloud AI pricing changes pushed me to test local LLMs against GPT-5 and Claude for real engineering work. Three weeks in, here's the short version.

The catch

• Context across multiple projects is still weaker than proprietary models.
• Package and API knowledge lags behind slightly, costing occasional debugging time.
• Context compaction is the real limiter: usable context lands closer to 132K than the advertised 250K, giving roughly six to nine solid prompts before things slow down.

The upside

• Everything stays in house, which matters a lot for sensitive code and infrastructure.
• Working locally keeps you closer to the architecture instead of rubber-stamping generated code.
• The gap to frontier models has shrunk to roughly a year, and it's still closing.

The models

• Qwen 3.6: barely different from 3.5, sometimes worse for coding.
• Qwen 3.5: strong reasoning, but tool calling needed babysitting.
• Gemma 4: the standout. Fewer hallucinations, cleaner scaffolding, my daily driver.
• GPT-OSS: solid nine months ago, outmatched now.

Where it shines

Backend and mobile work with real structure: Ruby, Python, TypeScript, Go, Flutter, Swift, Kotlin, and especially Rust, where the compiler's strict feedback made the model genuinely reliable. C and C++ work fine but need closer review given the stakes of a subtle bug.

Where it doesn't

Anything HTML and CSS heavy. Repeated classes, broken layouts, no consistent pattern. 6/10 at best, and not something to hand off without supervision.

Bottom line

Not a full replacement for GPT-5 or Claude yet, but a legitimate complement: private, cost-effective, and closing the gap faster than expected.