Local LLMs: When Self-Hosting Actually Beats the Cloud

Every few months someone tells me they've moved their team off ChatGPT because they bought a Mac Studio and now run everything locally. They sound proud. They also tend to be running a 7B model for a workload that needs a frontier model, and they're paying their own electricity bill for the privilege.

Self-hosting LLMs is a real option in 2026. It is also wildly oversold. The math works in a narrow set of cases. Most people who try it aren't in those cases.

The cloud is winning on price for most people

A Claude Pro subscription is $20 a month. A serious local rig — a Mac Studio with 192GB of unified memory, or a workstation with two RTX 5090s — runs $5,000 to $9,000 before electricity. To break even at $20/month you need 250 to 450 months. That's more than twenty years. Hardware doesn't last twenty years, and the model you can run on today's hardware will be embarrassing in eighteen months.

When people quote me a "local is cheaper" calculation, they leave out three things: their time maintaining it, the gap between what their hardware can run and what they'd actually want to use, and the inference cost they'd be paying if they billed their own hours.

If your workload fits inside a $20/month subscription, just pay the $20.

Where local actually pays off

There are real cases where self-hosting wins. The honest list is shorter than the hype suggests.

Data you legally cannot send to a third party. Healthcare records under HIPAA. Attorney work product. Defense contractor material. Anything covered by a strict data-residency clause your customer made you sign. If your compliance team would lose their mind seeing the content pasted into a cloud product, local isn't a preference, it's a requirement.

High-volume API workloads where you've actually done the math. If you're sending millions of tokens a day to a cloud provider for batch work that a 70B model handles well, a pair of RTX 5090s or a single H100 can beat the API bill. The keyword is batch. The moment latency matters, the math gets worse.

Experimentation that benefits from no rate limits. Fine-tuning, large eval runs, agent loops that hammer a model thousands of times. Running those locally means you stop watching a meter.

Privacy preferences you'll actually act on. If you will not paste your journal, your kids' schoolwork, or your therapy notes into a cloud product, a local 13B or 30B model is a reasonable answer. Not a great answer, but a reasonable one.

That's the list. "Cost" by itself is rarely on it.

The model gap is the real problem

The honest version of this post is that the open models are good and the frontier closed models are better. Llama 3.3 70B, Qwen 2.5 72B, and DeepSeek's open releases are genuinely useful on hardware you can buy. They handle summarization, structured extraction, code completion, and most mid-difficulty reasoning. They do not handle long-context, multi-step agentic work the way GPT-5 or Claude 4.5 will. For anything where you'd reach for "the smart model," you're going back to the cloud.

This isn't permanent. The open models are closing the gap. But planning your stack on "the gap will close" is a bet on the future, not a decision you can make today.

A sane way to start

If after all that you still want to try local, start cheap.

Pick a Mac you already own with at least 32GB of unified memory, install Ollama, and pull Llama 3.3 70B (quantized) or Qwen 2.5 32B. Run it as your default for one specific workflow: code completion in your editor, summarizing PDFs, or generating commit messages. Use it for two weeks. Pay attention to every time you reach for ChatGPT or Claude instead of the local model. That ratio tells you whether local is actually replacing cloud work or just sitting on your machine.

If the local model is handling 80% of what you tested, you've found a fit. Now you can think about better hardware. If it's handling 30%, you have an expensive paperweight in the making, and the $20/month was the smarter call.

What I actually run

I run a hybrid setup. Local Llama 3.3 70B for first-pass drafting, log analysis, and anything I don't want hitting an API. Cloud Claude for work that actually needs the smart model, and for anything agentic. The split is roughly 60/40 in favor of local by volume, but the cloud work is where most of the value lives.

The point isn't "local good, cloud bad." It's that you can use both, and most people would be better off doing exactly that.

If you do go local, the next question is which hardware. That's a different post.