As top AI providers scale up their training clusters and inference clusters it brings with it a big hot problem. Air cooling is at its limit in traditional data centres, which were not designed to handle the heat generated by today’s massive AI models.
NVIDIA is turning to liquid cooling in its latest generation AI hardware, believing that the solution is not more fans but plumbing. It’s a step that’s necessary as server racks go from 20kW to 135kW. Air-cooled solutions aren’t able to move heat at this level efficiently.
Liquid cooling is a key solution. Liquid cooling, which reduces reliance on chillers, and allows for more efficient heat rejection is driving the next-generation of high-performance and energy-efficient AI Infrastructure.
NVIDIA’s new Blackwell platform system is a big bet on this.
NVIDIA GB200 NVL72 & GB300 NVL72.
These full rack-scale systems are designed from the ground-up with liquid cooling, to tackle trillion-parameter AI and demanding reasoning tasks. These systems promise to deliver significant improvements in performance and efficiency over older air-cooled setups.
NVIDIA claims that the GB200 system has a 25x higher energy efficiency, and a 300x greater water efficiency compared to traditional air cooling. The newer GB300NVL72 aims to achieve 30x energy savings.
Serious Cost Savings.
The cooling can consume up to 40% of the power bill for a data center. NVIDIA estimates switching to its liquid-cooled GB200 could save a 50MW data center over A$6,000,000 annually (US$4,000,000+) on cooling energy and costs.
The shift isn’t about saving money, it’s about enabling scaling. Air cooling is impractical due to the tight integration required for modern AI. This includes using tech such as NVIDIA NVLink, which allows for rapid GPU communication.
Because liquid is almost 1,000 times more dense than air, it is far better at removing the heat. This allows for more power to be used on actual computing rather than trying to prevent a melting down.
Industry Partners are already jumping onboard, developing reference architectures for these new NVIDIA Systems. Companies like Vertiv Schneider Electric CoolIT Systems and Boyd are launching solutions to handle heat load efficiently.
Even cloud giants like AWS are collaborating on next-gen liquid cooling, reporting significant compute power increases while cutting energy use. It seems clear that cooling is becoming as crucial as the silicon itself for the future of AI.
While specific Australian availability and pricing for the NVIDIA GB200 NVL72 and GB300 NVL72 systems haven’t been detailed yet, expect them to arrive via NVIDIA’s usual enterprise partners and system integrators.
This technology represents a fundamental shift in how high-performance data centres will be built and operated.
For more information, head to the NVIDIA Blackwell Platform page: https://www.nvidia.com/en-us/data-center/blackwell-platform/
