AI’s Two Power Problems

One scales linearly. The other, exponentially.

September 22, 2024 • Est. Reading Time: 5 minutes

AI’s Two Power Problems

While Friday’s newswire flooded with commentary on the Microsoft-Constellation Energy Three Mile Island power purchase agreement, Carnegie Mellon released an interview with one of Microsoft’s top competitors — Alphabet CEO Sundar Pichai — that passed relatively undetected.

Today’s Energy Shots dissects a short excerpt from Pichai’s discussion that, stripped of technical jargon, indicates a widespread misconception about the primary source of AI’s long-term power demand growth potential.

Question:

“The general consensus is that energy is an issue for AI and emerging technologies… As you look at it, how serious should we be taking this problem?”

Sundar Pichai:

"Look, it's such an important question… On the [power] consumption side, we're going through the early phases where we're inefficiently pre-training these models, but on the inference side, I think we can get dramatically more efficient over time and maybe accomplish a lot of these tasks in a much more efficient way. If you look a decade out, I think we can be optimistic again.”

The underlined points of Pichai’s response indirectly provide a wealth of information for energy market participants.

In summary, AI developers face two distinct power problems.

One for inference. One for training.

The difference:

Inference Power Demand (Operational AI)

  • Demand from data centers

  • Inference power demand scales with user adoption and frequency of use. The more users interact with AI (e.g., querying chatbots, using AI in software), the greater the number of computations the data centers must perform, leading to increased power consumption.

  • Linear demand growth with the number of users

Training Power Demand (AI Advancement)

  • Demand from supercomputers

  • Training requires supercomputing-level resources for a limited time but with massive energy demand. As models advance, the datasets and processing complexity grow, which pushes the power requirement up.

  • Exponential demand growth: energy consumption during training is non-linear, as newer models are larger and more complex, exponentially increasing power use.

To summarize:

  • Inference “compute” refers to the power required for user interactions with processed data in AI models like GPT-4 or the latest multi-step OpenAI o1 system. Inference power consumption scales linearly with user adoption and software deployments with embedded AI.

  • Training compute refers to the power required to create new AI models by ingesting and processing vast amounts of data. Creating more capable AI models requires creating more robust supercomputers. Complexity scales exponentially.

Pichai’s distinction between inference-driven and training-driven power demand reflects a consistently under-appreciated variable in energy markets.

As summarized by mathematician Albert Bartlett (“Forgotten Fundamentals of the Energy Crisis”):

“The greatest shortcoming of the human race is our inability to understand exponential growth.”

Supercomputers’ Exponential Growth

The Department of Energy’s Oak Ridge National Laboratory (ORNL) shattered records in 2022 with the Frontier Supercomputer — the world’s first exascale system — capable of performing 1.7 quintillion calculations per second.

As shown below, Frontier calculates 13 billion times more calculations per second than the world’s fastest supercomputer from 1993.

The scale is difficult to comprehend.

  • 2024: 1,714,810,000,000,000,000 calculations per second

  • 1993: 131,000,000,000 calculations per second

As one likely expects, exponential increases in computing power comes with a price tag in the form of watts.

With instantaneous power consumption of more than 22 MW, systems like Frontier consume more electricity during a one-week cycle at peak than is required for:

  • A full charge on 76,000 Tesla Model 3s

  • The daily power demand of a 50,000-person city

  • The annual power demand of 357 American households

To be sure, the systems are vastly more efficient than their predecessors. However, computing power gains outmatch efficiency gains by approximately 3 million to one.

Furthermore, as the race for AI market share heats up, industry participants are increasingly building their own supercomputers to accelerate AI advancements and reduce their dependence on limited government-operated computing capacity.

In the latest edition of Top500’s world’s fastest supercomputers, systems from firms like Microsoft and Nvidia jumped into the top ten. According to Nvidia execs, the company now produces systems five times as powerful as their official 10th-ranked supercomputer every single month. Estimated power consumption for this 5x’d system exceeds 75 MW — over 3x more than the Frontier’s stated power usage.

Extending this scale across the global network of private and public institutions involved in AI model development adds context to industry commentary like the following:

“I think compute is going to be the currency of the future. I think it’ll be maybe the most precious commodity in the world."

Sam Altman

This commentary contains our views and opinions and is based on information from sources we believe are reliable. This commentary is for informational purposes, should not be considered investment advice, and is not intended as an offer or solicitation concerning the purchase and sale of commodity interests or to serve as the basis for one to decide to execute derivatives or other transactions. This commentary is intended for Mobius clients only and is not considered promotional material.