Local AI is already good enough for most use cases, and the gap with frontier models will close within 18 months

Greg Isenberg puts the current utility threshold plainly: a model running on a gaming GPU or a decent Mac is good enough for about 80% of what most people use cloud services like ChatGPT or Claude for. That figure is doing real work in the market. Isenberg runs a local agent every 20 minutes, scanning for business opportunities at zero marginal cost, using what he calls the “Quen 37” model locally. The inference bill for that continuous process is nothing. For the solo builder or the cost-conscious team, that changes the calculation on what to build and how to run it.

The economics driving this shift are not subtle. Marc Andreessen estimates that distilling a model costs roughly 2% of the original pre-training cost. Dylan Patel notes that DeepSeek achieved GPT-4-level performance at 1/600th the cost of GPT-4. Harry Stebbings calculates that raw chip performance improves roughly 3x every 18 months, and that additional optimizations like quantization deliver another 3x on top, producing roughly a 10x improvement in tokens per unit of money every couple of years. Nikesh Arora predicts that long-term token pricing will fall to one-tenth of current levels. Each of these is a separate force; together they describe a cost structure that is collapsing faster than most API pricing strategies can track.

Practitioners are already routing around frontier models where the task permits. Swyx describes a classification model at 800 million parameters, self-hosted, which handles field-of-study routing for queries without touching a frontier API. Swyx also notes that for such tasks, a fine-tuned one-billion-parameter model recovers approximately 95% of frontier performance. Ramin Hasani adds that Liquid AI’s one-billion-parameter model runs fast enough on an iPhone to serve basic use cases like privately searching and classifying local data. At the far edge of the spectrum, Hasani describes liquid neural networks that fit in one to 25 megabytes, capable of running on a CPU or a Raspberry Pi for specialized predictive applications.

I think that majority of inference in 5 years is going to be using a open-source or custom fine-tuned or distilled model, not using a frontier Brendan Foody

Open-source self-training pipelines are pushing capability further than the hardware story alone would suggest. Matei Zaharia describes pipelines built entirely on open-source models, where the same model generates training environments and trains itself, and which beat frontier models at specific tasks. That is not a marginal result. It means that for well-defined problems, the frontier is not a ceiling that only API providers can reach. Zvi Mowshowitz adds a nearer-term data point, predicting that these capabilities will be widely available and open source within a handful of months.

The 18-month horizon that several observers arrive at is worth taking seriously. Jesse Genet predicts that local models will reach a capability level comparable to Claude Opus within that window. Joseph Nelson frames the same gap structurally: there is roughly an 18-month delay between a multimodal cloud-available model’s capability and the point at which that same capability can run on an edge device like a Jetson Orin or an iPhone. Genet also offers a concrete cost signal to watch. When AI spending for a family reaches around $400 per month, she predicts a meaningful uptick in adoption of local inference, driven not by privacy ideology but by the simpler logic of not wanting to pay that bill. She demonstrates the direction already: a fully custom homeschool curriculum, built locally, cost her $8 for the year.

Brendan Foody’s five-year prediction follows from these compounding forces. The majority of inference, he argues, will run on open-source, custom fine-tuned, or distilled models rather than frontier APIs. That view is consistent with the practitioner behavior Swyx and Hasani describe, and with the trajectory Ethan He identifies: compute costs falling roughly 2x every year. The cumulative effect of that compression on what local hardware can run is not speculative. It is already observable in the gap between what shipped on consumer silicon two years ago and what ships today.

What the evidence describes is not a single inflection but several curves arriving at roughly the same moment. Model quality improved. Quantization became reliable. Consumer silicon became a serious inference platform. Distillation costs dropped to a fraction of training costs. The result is that the resource barrier separating frontier capability from local deployment is now primarily a function of time rather than capital. The institutions and pricing structures built around that barrier have, by the most optimistic estimates, 18 months to decide whether to adapt or defend ground that is already being taken.