Is Your Data Center Prepared to Support Your Organization’s Growing AI and Digital Transformation Initiatives?

Fred Rebarber, Senior Technical Director, Thermal Solutions, Vertiv

In the roughly 16 months since the start of the pandemic, we’ve undergone a paradigm shift in how we work, shop and communicate. This led many organizations to turn to artificial intelligence for their digital transformation plans to support everything from automated supply chains, to video streaming, to virtual meetings.

According to McKinsey’s Global Survey, “The State of AI in 2020,” 50 percent of the respondents said they have adopted AI in at least one of their business functions. In one Gartner poll, 47% of respondents said that AI investments were unchanged from the start of the pandemic, and 30% of respondents actually planned to increase their investments in AI technology. There is reason to believe this intense ramp up in digital transformation represents an acceleration toward an inevitable future. For organizations looking to be at the forefront of these trends, it’s important to consider how this technology will affect data centers across the globe.

What Increased AI Applications Means for Data Centers

With more and more processing power being packed into servers supporting artificial intelligence and other processing-intensive applications, rack power requirements are exceeding 20 kilowatts (kW) in a growing number of facilities, and many organizations are now looking to deploy racks with requirements of 50 kW or more.

Air cooling systems have continually evolved to address higher densities with greater efficiency, but there is a point at which air simply does not have the thermal transfer properties required to provide sufficient cooling to high-density racks in an efficient manner. This can reduce the performance and reliability of specialized servers and becomes less energy efficient as rack power increases. As more high-power racks are deployed, air cooling becomes untenable from an economic and sustainability perspective.

Forcing the issue are newer generation CPUs and GPUs, which have thermal power densities much higher than previous generation architectures. After years of relatively stable power densities, Intel’s Cannon Lake CPUs introduced in 2020, for example, had thermal power densities double that of the previous generation CPUs introduced just two years earlier.

Simultaneously, server manufacturers are packing more CPUs and GPUs into each rack unit (U). With multiple high-performance servers in a rack, systems that deliver cooling air to racks are unable to provide adequate cooling capacity, even with containment. In addition, the strategy of spreading compute loads out is not feasible in processing-intensive applications because of the latency challenges created by physical distance that exist even within a single server. As a result, components are being compacted within devices, creating ultra-dense 1U servers that are driving rack densities to unprecedented levels.

So, what is the solution to cooling these high-density racks? More organizations are exploring the feasibility of bringing liquid to the rack to increase the capacity and efficiency of data center cooling. Liquid cooling leverages the higher thermal transfer properties of water or other fluids to support efficient and cost-effective cooling of high-density racks.

When Liquid Cooling Becomes Inevitable

In its paper, Data Center Thermal Management Report 2020, technology analyst firm Omdia predicts strong growth in liquid cooling and expects the trend to double between 2020 and 2024. Some of the factors fueling this growth include increased chip and server power consumption as discussed previously, edge growth and changing energy efficiency and sustainability requirements.

Facilities looking to deploy extremely high-density racks (> 30 kW) and adopt large-scale AI and other applications for their transformation will have little choice whether or not to use liquid cooling. No matter how the system is configured or optimized, air cooling will simply be unable to deliver the heat removal capacity required to maintain the reliability of IT systems. This will be true on the edge as well as in core data centers.

Beyond Reliability

For organizations looking to design new data centers, or retrofit existing facilities, to enable digital transformation, liquid cooling can deliver numerous benefits beyond enhanced IT equipment reliability. Here are just a few examples:

• Improved Performance: A liquid cooling system can deliver IT performance benefits. As CPU case temperatures in high-density applications approach the maximum safe operating temperature, as is likely to occur with air cooling, CPU performance is throttled back to avoid thermal runaway.
• Reduction in Acoustical Noise Levels: In very dense compute applications using air-cooled systems, noise levels may get to the point of requiring sound intervention programs to protect data center operators, as prescribed by OSHA. Liquid cooled systems have the potential to dramatically reduce noise levels.
• Improved Energy Efficiency: The higher thermal transfer properties of liquid compared to air, combined with the elimination of fans required to move air across the data center and through servers, can create significant energy savings in liquid-cooled data centers. The pumps required for liquid cooling consume less power than the fans needed to accomplish the same cooling.
• Sustainability: According to ASHRAE’s Technical Committee’s white paper, “Emergence and Expansion of Liquid Cooling in Mainstream Data Centers,” liquid cooling create opportunities to reduce data center energy consumption and drive PUEs down to 1.1. It also provides a more effective approach for re-purposing captured heat to reduce the demand on building heating systems. The return-water temperature from the systems can reach up to 140° F (60° C) and the liquid-to-liquid heat transfer is more efficient than is possible with air-based systems.
• Maximize Space Utilization: The density enabled by liquid cooling allows a facility to better use existing data center space, eliminating the need for expansions or new construction, or to build smaller-footprint facilities. It also enables processing-intensive edge applications to be supported where physical space is limited.
• Lower Cost of Ownership: ASHRAE conducted a detailed cost of ownership analysis of air-cooled data centers versus a hybrid model air- and liquid-cooled data centers and found that, while a number of variables can influence TCO, “liquid cooling creates the possibility for improved TCO through higher density, increased use of free cooling, improved performance and improved performance per watt.” Keeping these factors in mind, the previously mentioned ASHRAE white paper, “Emergence and Expansion of Liquid Cooling in Mainstream Data Centers,” goes on to say, “There is a rack density point, which varies per cooling solution, at which the first cost to build and deploy liquid cooling is actually lower than air-cooling the same IT equipment load, making the payback period instantaneous.”

Organizations looking to incorporate artificial intelligence into their digital transformation need to recognize that the dense concentrations of processing power needed to support this technology exceeds the physical limits of air cooling. For these organizations, liquid cooling is a natural progression. However, organizations should keep in mind that bringing liquid cooling solutions to the rack is a complicated process, and it’s important to find the right partner to ensure the transition is a success. To learn more, read our white paper, “Understanding Data Center Liquid Cooling Options and Infrastructure Requirements.”

This article was written by Fred Rebarber, Technical Business Development Manager, North America, Thermal Management at Vertiv. In his role as Technical Business Development Manager, Fred serves as the corporate technical interface for large end-users and consulting engineers who specialize in mission critical designs. Vertiv brings together hardware, software, analytics and ongoing services to enable continuous and optimal running of vital applications for data centers, communication networks, and commercial and industrial facilities. Its portfolio comprises power, cooling and IT infrastructure solutions and services, extending from the cloud to the edge of the network.