Why AI turned into a compute problem

For most of computing history, getting a program to do more meant writing cleverer code. Modern AI broke that pattern. The largest gains of the last decade came less from new ideas and more from doing the same thing at enormous scale: bigger models, more data, and far more computation.

That shift is why a conversation about artificial intelligence keeps turning into a conversation about power plants, chips, and hundreds of billions of dollars. The intelligence lives in software, but producing it and serving it is now a heavy-industry problem.

The core fact this whole path rests on: an AI model's capability tends to rise with the amount of computation poured into it, and computation is a physical resource with a physical cost, silicon, electricity, cooling, buildings. So a race to build smarter AI became, in practice, a race to build more compute, which is a race to acquire chips and power.

Understanding that translation, from intelligence to infrastructure, is what makes the entire AI buildout legible instead of mysterious.

Compute is not a metaphor. It is a countable quantity of arithmetic: the number of mathematical operations a chip performs. The unit is the FLOP, a floating-point operation, one multiply or add on decimal numbers. Training a large model requires an almost unimaginable number of them.

The headline figures are stated in operations. Training a frontier model can involve on the order of $10^{25}$ FLOPs or more, an amount that would take a single laptop many thousands of years. It is done in weeks only by running tens of thousands of specialized chips in parallel.

one big training run  ~  10^25 FLOPs
one modern AI chip    ~  10^15 FLOPs per second (petaflop-class)
so, very roughly:     10^25 / 10^15 = 10^10 chip-seconds
                      = a huge chip fleet running for weeks

The takeaway is that "compute" reduces to chips multiplied by time multiplied by power. Every headline about AI capability is downstream of that product, and every constraint on the buildout, chips, electricity, cooling, is a constraint on one of those factors.

Spending more compute would be pointless if it stopped helping. The reason the industry poured in ever more is an empirical finding known as scaling laws: across many orders of magnitude, a model's error falls smoothly and predictably as you increase three things together, the model's size, the training data, and the compute.

The crucial word is predictable. Scaling laws describe the improvement as a smooth curve, so a lab can forecast, before spending, roughly how much better a model will get for a given increase in compute. That turned model-building from a gamble into something closer to an investment with a projected return.

That predictability is what unlocked the money. If doubling compute reliably buys a measurable capability gain, then spending billions on chips is a rational bet rather than a hope. The scaling relationship is the economic engine of the entire buildout: it is the reason a company can justify a data center that costs more than a car factory.

Scaling does have limits and diminishing returns, and the exact curves are debated, but the direction, more compute has meant more capability, has held long enough to drive the spending this path describes.

AI compute splits into two very different jobs, and confusing them hides where the demand actually is.

• Training is building the model: the one huge, months-long run over vast data that produces the finished weights. It is enormous but occasional, done a handful of times per model.
• Inference is using the model: every time someone sends a prompt and gets an answer. Each call is small compared to training, but inference happens billions of times a day, continuously, forever, for every deployed model.

Early on, training dominated the story, because that is where the eye-watering single numbers are. But as AI moved into products used by hundreds of millions of people, the aggregate of inference began to rival and in many estimates exceed training as a share of total compute.

The distinction matters for the whole buildout. Training is a capital event, a big spike you can schedule. Inference is an operating load, a permanent, growing, always-on demand for electricity that scales with usage. A world that merely trains big models needs bursts of compute. A world that uses them constantly needs a permanent new tier of always-on infrastructure, which is exactly what is being built.

Put the pieces together and you get a self-reinforcing loop that explains why demand did not just grow but accelerated.

1. More compute produces a more capable model (scaling laws).
2. A more capable model is more useful, so more people use it, for more tasks.
3. More usage means more inference, a permanent operating load.
4. More usage and more revenue justify spending on even more compute for the next, better model.
5. Return to step 1, at a larger scale.

Each turn raises both the one-off training demand and the permanent inference demand, and the two compound. This is why forecasts kept being revised upward: the loop feeds itself, and a capability gain today becomes a usage increase tomorrow, which funds a bigger capability gain after that.

The flywheel also explains the sense of urgency and the enormous bets. In a self-reinforcing race, being behind on compute means being behind on capability, which means losing users, which means less money for the next round. The structure rewards spending first and fastest, which is precisely the behavior the rest of this path examines, its power demands, its economics, and its risks.

This stops being abstract the moment you look at electricity, because compute at this scale is measured in the output of power stations.

The International Energy Agency provides the most-cited figures. Data centers used roughly 415 terawatt-hours of electricity in 2024, about 1.5 percent of global electricity. In the IEA's base case that roughly doubles to around 945 terawatt-hours by 2030, with AI-optimized data centers the largest driver, their demand projected to more than quadruple over the period. US data-center demand alone is projected to pass 250 terawatt-hours by 2026.

To make the scale tangible, roughly a thousand terawatt-hours a year is comparable to the entire electricity consumption of a large industrialized country. That is the new load AI is adding on top of everything else the grid already serves.

The point of the numbers is not the exact figure, which will be revised, but the order of magnitude and the direction. Compute has grown from a rounding error in global electricity to a line item that utilities, governments, and grid operators now plan around. AI stopped being purely a software phenomenon and became an energy phenomenon, which is the subject of a later lesson.

Technology has seen big infrastructure booms before, railways, electrification, the fiber-optic buildout of the internet. The AI compute buildout rhymes with them and differs in ways that matter.

Two differences stand out. First, the binding constraint has shifted from money to physical energy: as later lessons show, the limit on a new data center is often not capital or land but whether the grid can deliver the electricity. Second, the core asset depreciates fast, an AI chip is largely superseded within a few years, unlike a railway that lasts a century, which makes the economics far more demanding.

Those two facts, an energy constraint and a fast-depreciating asset, shape everything that follows: how the data centers are built, where they can go, and whether the enormous spending pays off.

The chain that turns a software goal into a heavy-industry problem: scaling laws make compute pay off, usage turns it into a permanent load, and the flywheel drives demand upward until it hits physical limits.