For a few years the only direction for AI was up. Model sizes swelled, compute budgets grew, and everyone measured progress by the size of the pretraining run. Bigger meant smarter, and smarter was the whole game. Then, in 2024, the industry stepped off that treadmill. Not because the frontier stopped moving, but because a quieter path opened up next to it: small models, built to run on the device already in your pocket.
The turn is easy to date. In February 2024 Google released Gemma, its first open weights models, in 2B and 7B sizes. Two months later Microsoft shipped Phi-3-mini, a 3.8 billion parameter model that its technical report argued was capable enough to run locally on a phone. In September Meta announced Llama 3.2, and the headline sizes at Connect were not the big ones but the compact 1B and 3B variants, tuned and quantized to run on edge hardware. Three labs, within eight months, all betting that useful models could be measured in billions of parameters instead of trillions.
There is an honest reason this happened: most applications do not need a frontier model. A support triage bot, a summarizer, a code autocompleter, a local search index improved with embeddings. These jobs run fine on a few billion parameters, and they run an order of magnitude cheaper. When your inference happens on a phone or a laptop, you also dodge latency, network dependency, and the per-token metering of an API. For a developer shipping a tool, the appeal is not technical nostalgia. It is that the model stops being a recurring cost and becomes a shipped artifact.
The practical enabler is that local inference is now boring in a good way. Ollama bundles model loading and a service interface so running a single command pulls and serves an open model. Tools like it collapsed the distance between "there is a good small model" and "I am using it right now." You no longer need a GPU cluster to play with these ideas; a laptop is fine.
I want to be careful not to oversell it. Small models still struggle with genuinely open-ended or knowledge-heavy tasks, and "capable enough for my use case" requires honest evaluation rather than hope. Edge AI also has a floor: it is only as good as the hardware a user happens to own. What changed in 2024 is not that small models beat the frontier. It is that they stopped being a compromise and started being a design choice. Not everyone needs the biggest model. Most of us, most of the time, need one that is fast, private, offline-capable, and cheap, and that is a smaller problem than the industry spent three years insisting it was.