Liquid Cooling in the Data Center is a Good Thing
This launches our article series on liquid cooling design and infrastructure updates and critical considerations in the AI era.
Transforming every data center into an AI data center is not a matter of if but when. The critical updates in the field of liquid cooling since the last paper have accelerated this transformation. New innovations around manageability, ease of service, density, and new applications have propelled liquid cooling into the forefront of data center design. This is not a trend but a necessity.
The exponential growth in computational demands of large-scale (and small-scale) AI systems requires an immediate paradigm shift in data center design.
To that point, we have all become users AI. None of this is a fad. AI is poised to drive a 160% increase in data center power demand and will use 8% of US power by 2030, compared with 3% in 2022. In terms of infrastructure design to support this massive growth, one fact is clear: Our data centers have run out of air. Leaders must turn to some liquid cooling solution to dissipate all of this newly generated heat and create greater density levels. There are also massive cost considerations, and it’s a big reason why many leverage liquid immersion cooling to reduce cooling OPEX by over 90% and reduce CAPEX building costs by more than 50%.
This article series will explore critical updates to our previous special report. Specifically, we will cover the existential situation facing many data center leaders. It’s official; we have entered the Era of Liquid Cooling. The special report will explore what this means; we’ll dispel some new liquid immersion cooling myths and discuss how you can adapt these technologies to your facilities.
Introduction
The second ChatGPT hit the mainstream, a seismic shift occurred in how humanity interacts with data. Even though many in the data center industry didn’t realize it, their world was about to shift as well. To look ahead, we’ll need to look backward just a bit.
Eight years ago, in the first AFCOM State of the Data Center report, we asked the all-important question about rack density. Respondents indicated that their average density was 6.1 kilowatts per rack in that report. Given the typical types of workloads data centers support, this metric is pretty expected. While high-density applications were a thing, most data centers were still running very traditional applications like e-mail servers, databases, and other business- critical services.
Over the past year, everything has changed. In the 2024 report, respondents indicated that the rack’s average density had increased to 12 kilowatts. Most respondents (60%) are actively working to increase density in their racks (58%), primarily by improving airflow, followed by containment (42%) and liquid cooling (40%). What was driving all of this? AI.
Per the report, most respondents (53%) believe new AI workloads (generative AI) will “definitely” increase capacity requirements for the colocation industry.
Here’s the crazy part: Even though density has doubled, at least per the report, it’s still insufficient to support AI and high-density architecture. In perspective, a single node of NVIDIA’s DGX H100 cards (eight cards per node) will consume between 10.2 kilowatts per rack. Based on our findings, a traditional data center could only support one of these high-end units in their rack despite growth in density. This rapid pace of evolution has become the driving force of innovation in our industry. The most significant difference is that it’s happening incredibly fast. While there isn’t current support for immersion cooling of H100s, there are already designs supporting immersion cooling GPUs. We’ll discuss this in a later section.
This is where liquid cooling must come into play. Because of AI, the amount of power our systems require will only continue to grow. In 2023, data centers across the globe consumed 7.4 GW of power, a 55% increase from the 4.9 Gigawatts in 2022, according to Cushman & Wakefield. That number is already forecast to grow significantly through 2030.
Massive strides have been made in terms of efficiency. The PUE trends have indicated that we are indeed using our data center resources more wisely.
However, the amount of power consumed by servers and cooling is still quite a bit.
As we examine emerging use cases (like AI and HPC), we must understand their impacts on cooling systems, networks, storage, and compute. This involves everything from liquid cooling GPUs to deploying a complete, all-in-one compute, storage, and network architecture with liquid cooling built-in.
This next point is critical: Liquid immersion cooling has become a staple design mechanism to support HPC and new AI use cases.
This means new imperatives exist in how we cool, maintain, and optimize our data center facilities.
Liquid in the data center: Yes, it’s a good thing
Liquid immersion cooling isn’t new. However, recent advancements have allowed more data centers to adopt it. Why is this happening? AI and HPC systems require more resources and consume more power. The growing emphasis on reducing data center power consumption is driving the market’s growth.
Increasing concerns about rising power consumption in data centers that use air-cooled technologies have significantly increased the need for liquid immersion cooling and immersion solutions. Data center companies actively invest in new liquid immersion cooling solutions to reduce operating expenses (OPEX). However, they are still struggling to develop data center infrastructure that requires minimal maintenance and less power. Again, working with liquid immersion cooling solutions that are easy to service and have simple maintenance cycles is definitely the way to go.
Remember, immersion cooling technology can reduce upfront data center costs by 60%, cooling power consumption by 95%, server power consumption by 10–20% Also, more racks at data center sites will increase rack density, thus making companies more inclined to use data center liquid immersion solutions over air.
Finally, emerging advanced technologies are a primary trend in the market. Advanced technology like machine learning, artificial intelligence, and blockchains are increasingly being adopted and used. Powerful new hardware for AI workloads can also produce high flux, wherein the power usage in a rack surges rapidly as the hardware begins a new workload, which can be challenging to manage using the traditional air method.
Let’s pause here and focus on you, the data center leader. Demand for new data centers will continue to grow. That part is prominent. However, the use cases tenants and clients ask for will be fundamentally different. Can today’s air-cooled ecosystems keep up? Immersion cooling has pushed boundaries on efficiency and density. Today, these solutions are a bridge to deliver the most advanced workloads to your customers.
Download the full report, Liquid Immersion Cooling Will Save the Data Center: Critical Design and Infrastructure Updates, featuring Hypertec, to learn more. In our next article, we'll examine some of the biggest updates in the liquid immersion cooling industry and how these new innovations will impact your data center.
