Nvidia CEO says Inference about to go up by a billion times
Building A.I. models – or “training” them with exabytes of historical data – gets most of the attention. But over time “inference” – or asking the trained models to generate answers – will dominate A.I. workloads
Nvidia CEO Jensen Huang just made this point in an interview with venture capitalist Brad Gerstner.
Today, around 40 percent of Nvidia’s business is for A.I. training, and another 40 percent is for A.I. inference. (The remaining 20 percent of revenue comes from its traditional video, gaming, and automobile products.)
Inference, however, “is about to go up by a billion times,” Huang says. “That’s the part that most people haven’t completely internalized…This is the Industrial Revolution…It’s going to go up a billion times.”
Nvidia’s Big Tech A.I. customers agree, and they are scrambling for both the chips and electricity to make it happen.
On Tuesday, Google inked a deal with Kairos Power, a startup supplier of small modular reactors (SMR), to purchase up to 500 megawatts of nuclear power by 2035.
Meanwhile, Oklo, which is backed by OpenAI, received a design approval this week from the Department of Energy for its Idaho National Laboratory plant, scheduled to go live in 2027.
And Amazon, which several months ago bought a data center situated at the Susquehanna, Pennsylvania, nuclear plant, just announced a separate $500 million investment in SMR developer X-energy.
The biggest move was of course Microsoft’s $15-billion long-term deal to restart Three Mile Island’s 835-megawatt Unit One reactor.
You will notice, however, that the vast majority of capacity from the welcome and long-overdue revival of nuclear power won’t occur in the near term. If they start now, and everything goes according to plan, the nuclear innovators could begin delivering new power later this decade and much more in the 2030s.
A.I. data center builders aren’t so patient. Elon Musk, to Jensen Huang’s total amazement, just locked, loaded, and connected 100,000 H100 chips in his new Memphis, Tennessee, complex in 19 days.
This process, Huang says, might commonly take battalions of normal humans one year. The A.I. race is ON!
Nvidia’s new Blackwell generation of chips, moreover, are so power-hungry they mandate a total rethink of data center design. In fact, Meta (Facebook) just demolished and rebuilt a brand new data center because it didn’t conform to the new power-dense and liquid super-cooled design specs.
A new report from Epoch AI suggests that power is the chief constraint to scaling A.I. through 2030.
Another new report from SemiAnalysis projects power needs for all North American data centers will grow to 45,614 MW in 2025 from 24,646 MW in 2023 – a two year rise of 85 percent.
It projects power needs for data centers dedicated to A.I. will grow to 17,883 MW in 2025 from 2,537 MW in 2023 – or around a 600 percent increase.
Such hyper-growth suggests that despite the exciting but nascent nuclear revival, the A.I. computing boom will, for the next half-decade or more, be powered mostly by traditional and especially by reliable energy sources.
That means baseload electricity generated chiefly by natural gas and back-up generators fueled with diesel.
See more here substack.com
Please Donate Below To Support Our Ongoing Work To Defend The Scientific Method
PRINCIPIA SCIENTIFIC INTERNATIONAL, legally registered in the UK as a company 
incorporated for charitable purposes. Head Office: 27 Old Gloucester Street, London WC1N 3AX.
Trackback from your site.
VOWG
| #
Computers screwing up the world what could possibly be wrong with that. By the way, A I is not sentient.
Reply