Last week we launched our article series on critical liquid cooling design and infrastructure updates and why liquid immersion cooling will save the data center. This week, we’ll examine some of the biggest updates in the liquid immersion cooling industry and how these new innovations will impact your data center.
Between 2005 and 2010, liquid cooling entered the data center with chilled doors. From there, looking into 2010 and beyond, liquid cooling was used in high- performance computing (HPC) environments and designs featuring direct contact and total immersion liquid immersion cooling solutions. But we’re not here to look at the past decade. We’re here to look at what happened alone in the last year. Buckle up.
Life before and after AI/ChatGPT
Introducing AI technologies, particularly generative AI models like ChatGPT, has marked a seismic shift in the data center industry. Before AI became mainstream, data centers were primarily designed to support traditional applications such as email servers, databases, and basic web hosting services. These applications typically required low to moderate levels of computational power, and the corresponding cooling requirements were relatively modest. Air cooling, with average rack densities of around 6.1 kilowatts, was sufficient to maintain the operational integrity of these data centers.
However, the rapid adoption of AI and machine learning applications has dramatically changed this landscape. According to a recent AFCOM State of the Data Center report, the average rack density has doubled to 12 kilowatts, driven primarily by the increased computational demands of AI workloads. The traditional air-cooled infrastructure cannot handle these high-density environments, leading to overheating issues, increased energy consumption, and significant operational costs.
In this new era, liquid cooling has become an option and a necessity. The shift from air to liquid cooling represents a fundamental change in how data centers are designed and operated. Liquid immersion cooling systems, which involve submerging hardware components in non-conductive liquids or directly applying coolant to heat-generating components, offer superior cooling efficiency and energy savings. As AI continues to drive up the computational and cooling demands of data centers, liquid immersion cooling will play an increasingly vital role in maintaining the performance and reliability of these critical infrastructures.
How liquid immersion cooling will save the data center
The limitations of traditional air-cooling systems have become more apparent as data centers strive to meet the demands of modern workloads. AI and HPC applications generate significantly more heat than traditional workloads, and air cooling alone is no longer sufficient to dissipate this heat effectively. This has led to increased energy consumption, higher operational costs, and the need for more sophisticated cooling solutions.
Liquid cooling, particularly immersion cooling, addresses these challenges by offering a more efficient heat dissipation method. Immersion cooling involves submerging servers in a dielectric fluid, which absorbs the heat generated by the components and transfers it away from the hardware. This method reduces the temperature of the components more effectively than air cooling and allows for higher densities within the data center. Liquid immersion cooling significantly reduces the facility’s overall footprint and operational costs by enabling data centers to operate at higher densities without overheating.
Moreover, liquid immersion cooling systems can reduce cooling operational expenditures (OPEX) by up to 95% compared to traditional air-cooling systems. This reduction in OPEX, combined with the potential for increased rack density, makes liquid immersion cooling an essential technology for the future of data centers. As data centers continue to scale up to meet the demands of AI and HPC applications, adopting liquid immersion cooling will be critical to maintaining efficiency, reliability, and sustainability.
Significant trends in data center cooling, power and AI
The shift towards liquid immersion cooling is part of a broader trend in the data center industry, driven by the increasing demands of AI and HPC applications. Several key trends are shaping the future of data center design and operation:
- Cooling: One of the most significant trends in the data center industry is the need for more efficient cooling solutions. As computational power increases, so does the heat generated by data center hardware. Liquid immersion cooling offers a more effective solution than air cooling, enabling data centers to maintain optimal temperatures even as they scale up to meet the demands of AI and HPC workloads.
- Power Consumption: The rise of AI and HPC has also led to a significant increase in power consumption within data centers. According to industry reports, AI workloads are expected to drive a 160% increase in data center power demand by 2030, with AI alone consuming 8% of US power. Liquid immersion cooling helps mitigate this increase in power consumption by improving the efficiency of the cooling process and reducing the overall energy required to maintain data center operations.
- AI and HPC: The growing importance of AI and HPC drives the adoption of new technologies and solutions within data centers. These applications require more computational power and sophisticated cooling solutions to manage the heat generated by high-density hardware. Liquid immersion cooling is becoming increasingly essential for data centers that support AI and HPC workloads, as it can handle high-density environments and reduce energy consumption.
New types of business demands and efficiency requirements
The shift towards AI and HPC drives new business demands within the data center industry. Enterprise customers increasingly seek data center partners who can provide the infrastructure and services needed to support these advanced applications, leading to greater efficiency, scalability, and sustainability requirements.
One of the most significant trends in this area is the growing demand for repatriation—bringing workloads back from public clouds to on-premises or colocation data centers. In the latest AFCOM State of the Data Center Report, most respondents (71%) have seen a migration of workloads from the cloud back to on- premise data centers or colocations. For those data center operators that saw repatriation, 71% have seen an increase in the power load demand, including 22% who report a significant impact. As businesses seek greater control over their data and IT infrastructure, they turn to data centers offering the high-density, efficient cooling solutions needed to support AI and HPC workloads.
In response to these demands, data centers are placing a greater emphasis on efficiency, both in terms of energy consumption and space utilization. Liquid immersion cooling systems enable data centers to meet these efficiency requirements by reducing energy consumption, maximizing rack density, and minimizing the physical footprint of the facility. As a result, component cooling and immersion cooling are becoming essential components of modern data center design, enabling facilities to meet the growing demands of their customers while maintaining operational efficiency and sustainability.
Immersion Liquid Immersion Cooling: The HPC and AI Imperative
Fundamental changes in the data center business
The data center industry is undergoing a significant transformation, driven by the increasing demands of AI and HPC applications. These applications require more computational power, generate more heat, and consume more energy than traditional workloads, necessitating a fundamental shift in how data centers are designed and operated.
One of the industry’s most significant changes is the growing adoption of liquid immersion cooling. Immersion cooling, which involves submerging servers in a dielectric fluid, offers several advantages over traditional air cooling. It provides more efficient heat dissipation, increases densities, and reduces energy consumption, making it an ideal solution for data centers supporting AI and HPC workloads.
Several factors are driving the adoption of immersion cooling. First, the temperature fluctuations caused by AI workloads can be challenging to manage with traditional cooling methods. Immersion cooling provides a more stable and consistent cooling environment, essential for maintaining AI applications’ performance and reliability. Second, HPC applications, which require large computational power, generate significant heat that cannot be effectively dissipated with air cooling alone. Immersion cooling provides a more efficient solution, enabling data centers to support these high- performance workloads without the risk of overheating.
Data center design considerations
The shift towards immersion cooling also requires reevaluating data center design considerations. Traditional air cooling methods, which rely on fans and air circulation to dissipate heat, are no longer sufficient to meet the demands of modern data centers. New design considerations must be considered as data centers scale up to support AI and HPC applications.
One of the most critical considerations is the power implications of liquid immersion cooling. While immersion cooling systems may have a higher initial capital expenditure (CAPEX), they offer significant operational savings over time. Liquid immersion cooling systems reduce energy consumption, lower cooling costs, and enable higher densities, significantly reducing the total cost of ownership (TCO). As a result, data centers must carefully evaluate the long-term benefits of liquid immersion cooling when making design decisions.
Another important consideration is the cooling solution’s scalability. The cooling system must scale with the facility as data centers grow and evolve. Immersion cooling offers a scalable solution that can be adapted to meet the changing needs of the data center, making it an ideal choice for facilities that anticipate future growth.
Challenges in air cooling specific workloads
As the demands on data centers continue to increase, the limitations of air cooling are becoming more apparent. Let’s be clear: there is nothing wrong with air cooling as long as it is used properly. The challenge is that with greater levels of density of AI and HPC, air cooling is less efficient at managing the heat generated by those workloads, leading to several challenges:
- Scale: As data centers scale up to support larger workloads, the limitations of air cooling become more pronounced. Immersion cooling offers a more scalable solution to handle the increased heat generated by high-density deployments.
- Capacity: Air cooling faces a significant challenge in efficiently cooling larger capacities, which immersion cooling effectively addresses. By providing more efficient heat dissipation, immersion cooling enables data centers to support higher capacities without the risk of overheating.
- Efficiency and Performance: Air cooling is less efficient at maintaining optimal operating temperatures, which can negatively impact the performance and reliability of data center hardware. On the other hand, immersion cooling provides more consistent and efficient cooling, improving overall system performance and reducing the risk of hardware failure.
While there may be a few challenges to work through, there are pre-existing liquid immersion cooling myths that have been well covered and reviewed. Understanding how far immersion cooling has come and its capabilities is important.
Mythbusting Immersion Cooling
Despite the growing adoption of immersion cooling, several myths and misconceptions persist about the technology. It is important to address these myths to provide a clear understanding of the benefits and challenges associated with immersion cooling:
- Cable Wicking: A common concern is that liquids could wick up cables and damage the hardware. However, proper design and cable management in immersion systems prevent this issue.
- High-Pressure Systems: Some believe immersion cooling systems require high-pressure environments, making them more complex and difficult to manage. In reality, immersion systems do not require high pressure and are designed to operate safely and efficiently at standard pressure levels.
- Limited Use in Traditional Data Centers: While immersion cooling has traditionally been used in specialized environments, such as HPC and AI data centers, it is also increasingly being adopted in more traditional data centers. As the benefits of immersion cooling become more widely recognized, its use is expected to grow across a broader range of data center environments.
- Water Treatment Requirements: Unlike water cooling systems, which require regular water treatment to prevent corrosion and scale formation, immersion cooling systems use dielectric fluids that do not promote these issues. This reduces maintenance requirements and simplifies the cooling system’s operation.
- Combustible Oils: Another myth is that the oils used in immersion cooling are combustible, posing a safety risk. Modern dielectric fluids used in immersion cooling are non-combustible, making them safe for data centers.
- Material Compatibility: Concerns about material compatibility are also common but advances in materials science have addressed these issues. Most components used in immersion cooling systems are now fully compatible with dielectric fluids.
- Servicing and Maintenance: Some believe immersion cooling systems are difficult to service and maintain. These systems are designed for easy servicing, with components accessible without draining the tank. This reduces downtime and simplifies maintenance operations.
- Adaptability: Immersion cooling systems are highly adaptable and can be customized to meet the specific needs of different data center environments. Whether it is a small-scale deployment or a large data center, immersion cooling can be tailored to fit the facility’s requirements.
- Flammability: Concerns about the flammability of the fluids used in immersion cooling are unfounded. The dielectric fluids used in these systems are non-flammable, ensuring the safety of the data center environment.
- Space Saving: Immersion cooling, often perceived as requiring more space due to its horizontal configuration compared to traditional vertical racks, can actually reduce footprint by as much as 60%. Immersion cooling optimizes floor space utilization by streamlining cooling efficiency and eliminating the need for extensive air-cooling systems. This allows data center operators to reclaim valuable real estate, which can be allocated to other mission-critical infrastructure, enhancing overall data center density and operational efficiency.
Download the full report, Liquid Immersion Cooling Will Save the Data Center: Critical Design and Infrastructure Updates, featuring Hypertec, to learn more. In our next article, we'll outline liquid immersion cooling design considerations and explain the power of immersion-born GPU servers.
