Memory is the real bottleneck of the AI buildout, and the supply gap will not close until 2028

Dwarkesh Patel puts the capital allocation in blunt terms: hyperscalers are spending 50% of their CapEx this year on memory. That figure alone reframes what the AI buildout actually is. It is not primarily a compute story. It is a memory story, and the supply side is nowhere near ready for it.

Patrick O’Shaughnessy describes the current state of DRAM, NAND, and PCB markets as already running approximately 30% short of demand. Martin Casado adds the broader pressure context, observing that Microsoft, Meta, and Google are each on track to spend over 50% of their revenue on capex in 2026. The scale of that commitment makes the supply gap more consequential, not less, because there is no obvious release valve.

Andrew Feldman offers a concrete read on what the shortage means for pricing: Micron posting gross margins of 80 to 85% on HBM is not a sign of exceptional execution. It is a sign of a market where supply is so constrained that commodity economics have temporarily dissolved. Caitlin Kalinowski expects prices to roughly double, and her practical advice to the companies she works with is to pre-buy and stockpile memory now if they can afford to. Jake Cooper reports that Railway’s servers have actually appreciated in value as RAM prices have risen, which is not a condition that typically describes server hardware.

The inference problem is not really a compute problem because as the models get bigger, you now need to move the weights and of course what we call the KV cache into the compute units. And that is essentially a data movement problem. Kunle Olukotun

Dylan Patel frames the timeline clearly: capacity is growing at 20 to 30% per year, but true incremental supply from new investment will not arrive until 2028. The lag between a capital commitment and meaningful production output is long enough that the shortage will run well past any point where demand might moderate. That asymmetry, a fast-moving demand curve against a structurally slow supply response, is what makes this cycle unusual.

The shortage is not contained to data centers. Dwarkesh Patel notes that smartphone shipment volumes are set to fall 30% because there is not enough memory to go around. Yaroslav Azhnyuk points to a more oblique indicator: optic fiber cable prices jumped from $4 to $32 per kilometer in the early months of 2025, driven by data center demand crowding out other buyers. When a shortage is strong enough to distort a market as far removed as fiber cabling for drones, the constraint is operating at a level that generalist investors and supply chain planners have typically underestimated.

The technical dimension of the constraint is worth separating from the market dimension, because they compound each other. Reiner Pope and Kunle Olukotun, working from different angles, reach the same structural point: the binding limit inside AI systems is not compute but data movement. Olukotun describes inference as essentially a data movement problem, because running larger models requires moving weights and the KV cache into compute units, and that movement is what eats available bandwidth. Pope locates the evidence in the history of context lengths. Models moved from roughly 8,000 to 100,000 to 200,000 tokens in a relatively short period, and then stalled. That stall, Pope argues, reflects a reasonably balanced cost point: going further is not cost-prohibitive because of compute, but because of memory bandwidth. The ceiling on context length is a memory bandwidth ceiling.

Blackwell provides a partial answer. Pope notes that with Blackwell, scale-up memory on the order of 10 to 20 terabytes is now available, enough to hold a roughly 5 trillion parameter model plus KV cache. That matters for what large-scale inference can attempt. But bandwidth is a separate problem from capacity, and the supply constraints that govern HBM production affect both. A market that is already 30% short of demand, with no meaningful relief until 2028, is not going to resolve the bandwidth ceiling through optimism about the next product generation. The institutions that treat memory as a procurement detail rather than a strategic constraint are already behind.