Living Without API Keys in 2026

If you're reading this article, chances are you've been looking for an alternative to the API-key shackles we've grown accustomed to as developers.

I don't blame the big AI companies for this. Fair pricing and occasional price increases are expected. In fact, most of these LLM services are heavily subsidized to lower the barrier to entry. But June 1st was a dark day for me. The changes to Copilot weren't sudden, but they were very noticeable. I knew it was coming, I just didn't realize how much it would affect my workflow.

With that in mind, I spent the last three weeks seriously giving local LLMs a chance. And guess what? They aren't very good... But not nearly as much as I thought they would.

TL;DR;

Local models still can't maintain context across multiple projects the way proprietary models can. Coming from GPT-5 and Claude, that was a feature I relied on heavily. I could scaffold a mostly working integration in just two prompts.

They also don't always have the most current knowledge of every package or framework. This isn't a dealbreaker, just an occasional time cost: you'll debug code that looks correct, only to find the model referenced an API that's since changed or been deprecated.

The bigger challenge is context compaction. Even with a capable GPU, models advertised at 250K context behave closer to 132K in practice, with compaction kicking in around 100K. That gives me roughly six to nine prompts before performance starts to degrade, and tasks that take one prompt on a frontier model can take five locally, with compaction adding overhead each time.

But there's a real upside, too.

Running models locally keeps sensitive code and infrastructure details in house. It's worth pausing on how much of a company's codebase, sometimes including critical infrastructure logic, now routes through a handful of external providers, often across borders.

Working locally also keeps me more engaged with the architecture. Instead of accepting a thousand generated lines on faith, I stay closer to the design decisions being made, which is easy to lose when a frontier model prints a large, confident block of code in one pass.

And perhaps most notably, local models have closed a lot of ground. They're arguably about a year behind the frontier now, which is a much smaller gap than it used to be. I'd argue the race isn't primarily about scaling raw intelligence anymore. It's about delivering that intelligence efficiently, which is exactly the problem local LLMs have been chipping away at from day one.

The setup

Before getting into models and results, here's the hardware I used. This isn't a recommended minimum spec, it's simply what I already had on hand. In my experience, local inference is overwhelmingly GPU-bound, so the rest of the system matters less than you'd think.

• GPU: NVIDIA RTX 5090 (32 GB)
• Memory: 128 GB RAM
• CPU: AMD Ryzen 7 9700X (8 cores / 16 threads)
• OS: Ubuntu 24.04

The question I wanted to answer was simple: can a single consumer PC realistically replace GPT-5 and Claude for day-to-day software engineering?

The Models

Over three weeks I tested Qwen 3.5, Qwen 3.6, Gemma 4, and GPT-OSS, and eventually settled on Gemma 4 as my primary driver. Here's how each one stacked up.

Qwen 3.6: The version number alone raised my expectations, especially after hearing good things about Qwen 3.5. In practice, it felt very similar to its predecessor, and in some cases the coding-tuned variants of 3.5 held up better for my use case.

Qwen 3.5: Genuinely strong, and one of the better models I tested overall. My main friction was with tool calling, which failed often enough that I had to rework my system prompts. It would occasionally lose track of the correct calling format in longer sessions, and I ran into some failed replacements after extended use. Compaction helped, but the deeper issue was that multi-step reasoning wasn't consistent enough over long tasks for me to fully rely on it. I saw a milder version of the same behavior in Qwen 3.6.

Gemma 4: This was the model that made me take the idea of a local replacement seriously. It hallucinated less, its mistakes were more manageable, and I ran into far fewer of the "stuck" states where a model just stops producing anything useful. Scaffolding was strong across most of my projects, though it's worth noting these were already modular codebases designed with LLM collaboration in mind. Even accounting for that, Gemma 4 was the first model that felt like a legitimate alternative to a frontier model for daily work.

GPT-OSS: This one underperformed relative to my expectations, largely because the GPT name set a certain bar. To be fair, this model launched roughly nine months before I wrote this piece, so it's not a fair fight against the newest releases I tested it alongside. I'd expect a more recent OpenAI open release to be considerably more competitive.

The Tool: OpenCode

To actually evaluate these models in a realistic setting, I used OpenCode as my interface rather than a plain chat window. It let each model inspect the codebase, edit files, run commands, use tools, and work through multi-step tasks, much closer to how I'd actually use an assistant day to day. I wasn't trying to find out which model wrote the nicest isolated answer; I wanted to see how well I could work with each one over an extended period.

To keep comparisons fair, every model ran with the same context and output limits:

json

{
  "name": "Gemma 4 31B",
  "options": {
    "num_ctx": 131072
  },
  "limit": {
    "context": 131072,
    "output": 32768
  }
}

The model name changed between tests, but the surrounding configuration stayed constant, so the differences I observed came primarily from the models themselves rather than mismatched context or output limits.

Once every model had access to the same tools and environment, the differences between them became much clearer: tool calling reliability, memory of prior decisions, the quality of edits, recovery from mistakes, and consistency over longer tasks were all things I could observe directly instead of inferring from benchmarks. One thing this experiment made clear is that the model is only part of the equation. The interface, tooling, prompt design, context management, and how the project itself is structured all shape the final experience. A model that looks strong in isolation can be frustrating once it has real control over a codebase, and a weaker-looking model can perform well when the surrounding scaffolding supports it.

The Tasks: From my actual day to day

Infrastructure

I'll admit upfront that I leaned on this setup more than expected to help manage our clusters. I have a computer science background with a focus on AI, and while I've touched infrastructure work before, I've never been a systems administrator by trade; production-grade horizontal scaling isn't something I'd normally expect to take on solo. But in a small team, everyone ends up wearing a few extra hats, and this setup made that far more manageable. Once a solid base was in place, deploying new services became close to effortless.

A good example was our PenPot deployment, an open-source alternative to Figma that's become genuinely useful for our team's design collaboration. We still use Figma, but no longer exclusively, and I generally prefer open-source options where they're practical.

The deployment itself went smoothly, and the model's own recommendations led to a genuinely efficient storage setup with real high-availability characteristics. We have several nodes, and one common failure mode in "deploy and forget" setups is that HA exists on paper but not in practice: the service ends up pinned to a single node because of where the data lives. Our usual approach is a Synology iSCSI data class for anything latency-sensitive (caches, SQL databases, and similar workloads), paired with a slower, more shareable storage class for larger, latency-tolerant use cases like archives.

Instead of defaulting to that pattern, the model noticed we already had a MinIO instance running against our slower Synology backend (non-iSCSI) and suggested reusing it as the S3 target, a better fit than standing up something new. It also picked up on an existing LDAP setup on the same cluster and, with a couple of follow-up commands, tied everything together into a clean, complete deployment.

From my perspective, this was close to ideal:

• The service was deployed successfully
• Storage was fast, efficient, and genuinely highly available
• Every employee had role-appropriate access from the moment the service went live
• Manual intervention was limited to verification and secret provisioning, which I deliberately keep away from any LLM, local or otherwise, given the network access involved

Coding & Bug Fixes

Design

This is the area with the clearest gap, and it's worth being direct about it: none of the models I tested handled design-related work well, particularly anything involving HTML and CSS. Regardless of prompt clarity or available context, they struggled to work reliably in this space. Recurring issues included:

• Recreating classes that already existed
• Losing track of elements already defined
• Not settling on a consistent design pattern
• Producing markup or styles that broke rendering or unrelated JS/CSS

My takeaway is that HTML, CSS, and to some extent JS are verbose and loosely structured enough that a model this size struggles to get clear, validating feedback on whether something actually works. Languages and frameworks with stronger structure and a clearer sense of object-oriented composition (React, Flutter, Ruby) fared noticeably better than working directly against raw HTML. To be clear, there are well-established abstraction layers for this kind of work; I'm simply noting that direct HTML/CSS/JS work is still a real, common need in production systems, and it's where these models are currently weakest.

There is a silver lining: anything with more structure and stricter syntax checking performed better. It's still not perfect, since design work carries inherent ambiguity, but TypeScript in particular was the strongest case I saw in this category.

Overall, I'd put this use case at 6/10: usable, but it still requires real hands-on judgment. This isn't something you can fully hand off yet.

Backend & Web

This is where local models performed best overall. Structure is exactly what these systems are strongest at, and with solid guardrails in place, they build on it reliably.

Ruby: Solid, though not exceptional. Ruby leans monolithic, which occasionally made design-adjacent decisions harder than they needed to be. That said, backend work was a clear strength. Ruby's TDD-friendly culture gave the model a natural head start toward correct patterns, from seeds and schemas to a clean separation between dev, staging, and production. It was the best environment I had for seeing what's possible once real structure is baked into the codebase itself.

Python: Probably the single best-represented language in any of these models' training data, and it showed. Standard library usage, common web frameworks like FastAPI and Django, and typical data pipeline code were all handled comfortably, especially once type hints were in play. The main weakness was async code: models would occasionally mix synchronous and asynchronous patterns in ways that ran but weren't correct, which took closer review to catch than a straightforward syntax error would have.

TypeScript / Node.js: Backend TypeScript was a strong performer, for the same reasons it did well in frontend contexts: a real type system gives the model fast, concrete feedback when something's wrong. Where it slipped was dependency knowledge; less common npm packages, or packages that had recently changed their API surface, occasionally produced code that looked plausible but didn't actually compile.

Go: Go's small, deliberately simple language spec seemed to play to these models' strengths, and most straightforward service code came out clean on the first pass. Concurrency was the exception. Goroutine and channel-based code sometimes compiled and ran fine but hid subtle race conditions that only showed up under load, so this is an area where I'd want a careful human pass before shipping.

Mobile

Flutter: A strong fit, for reasons similar to Go and TypeScript. Flutter's widget-based structure and Dart's static typing gave the model clear guardrails to work within, and it handled state management, navigation, and component composition more reliably than it did anything HTML-based. Where it occasionally struggled was with less common packages or plugin-specific APIs, which sometimes required a manual nudge in the right direction.

Swift (iOS): Decent, but less consistent than Flutter. SwiftUI has moved quickly over the past few years, and I occasionally got code that mixed older UIKit patterns with newer SwiftUI idioms in the same file. It was usually a quick fix once flagged, but it meant I couldn't hand off iOS work with quite the same confidence as Flutter or backend work.

Kotlin (Android): One of the more pleasant surprises. Kotlin's similarity to Java, combined with strong static typing, meant the model leaned on patterns it clearly had a lot of exposure to. Jetpack Compose code was generally solid, with the occasional gap around very recent APIs, consistent with the broader "knowledge isn't always current" issue mentioned earlier.

Embedded & Systems

C: Usable for straightforward, well-scoped tasks, but this is the language where I trusted the model least by default. Memory management mistakes (off-by-one errors, missed bounds checks) were rare but did happen, and the consequences of a subtle bug here are higher than in a managed language. I treated anything C-related as a first draft that needed a careful manual review, not a finished artifact.

C++: A similar story to C, with the added complexity of templates and RAII. Modern C++ (17/20-style code) was handled reasonably well, but older-style codebases with heavy template metaprogramming caused more confusion and occasionally produced code that compiled only after several rounds of fixes.

Rust: One of the more interesting cases in this whole experiment. Rust's compiler is strict and gives extremely specific error messages, and the model was genuinely good at iterating against that feedback: borrow checker complaints in particular were resolved correctly more often than not. Once a project compiled, it tended to actually work, which made Rust one of the more trustworthy languages here despite its reputation for being hard to write.

Final Thoughts

After three weeks, my honest answer is: not yet, but closer than I expected. A single consumer PC running Gemma 4 can realistically cover a meaningful share of day-to-day engineering work across a surprisingly wide range of languages: backend services in Ruby, Python, TypeScript, and Go, mobile work in Flutter, Swift, and Kotlin, and even solid Rust with the right amount of review. It's noticeably weaker on front-end and design-heavy work, more cautious territory in C and C++, and it still can't match a frontier model's ability to hold context across a sprawling, multi-project workflow.

For teams weighing privacy, cost, and control against raw convenience, local models are a genuinely viable option today, not a full replacement for GPT-5 or Claude, but a serious complement to them. I expect that gap to keep narrowing, and local models will stay part of my toolkit going forward rather than being treated as a stopgap.