• About Us
  • Partnership Opportunities
  • Privacy Policy

Data Center Frontier

Charting the future of data centers and cloud computing.

  • Cloud
    • Hyperscale
  • Colo
    • Site Selection
    • Interconnection
  • Energy
    • Sustainability
  • Cooling
  • Technology
    • Internet of Things
    • AI & Machine Learning
    • Edge Computing
    • Virtual Reality
    • Autonomous Cars
    • 5G Wireless
    • Satellites
  • Design
    • Servers
    • Storage
    • Network
  • Voices
  • Podcast
  • White Papers
  • Resources
    • COVID-19
    • Events
    • Newsletter
    • Companies
    • Data Center 101
  • Jobs
You are here: Home / Cooling / Roundtable: New Generation of AI Hardware Raises the Bar on Cooling

Roundtable: New Generation of AI Hardware Raises the Bar on Cooling

By Rich Miller - September 22, 2020 Leave a Comment

Roundtable: New Generation of AI Hardware Raises the Bar on Cooling

A high-density rack of NVIDIA GPUs. (Image: NVIDIA)

LinkedinTwitterFacebookSubscribe
Mail

Today we continue our Data Center Executive Roundtable, a quarterly feature showcasing the insights of thought leaders on the state of the data center industry, and where it is headed. In today’s discussion, our panel of experienced data center executives weighs in on the present and future of rack density in the age of artificial intelligence. Our panelists include Phillip Marangella of EdgeConneX, CoreSite’s Juan Font, Angie McMillin from Vertiv, Kevin Facinelli from Nortek, Digital Realty’s Tony Bishop and Jaime Leverton from eStruxture Data Centers and Infrastructure Masons.

The conversation is moderated by Rich Miller, the founder and editor of Data Center Frontier.

Data Center Frontier: Artificial intelligence is bringing more powerful chips into the data center. What’s your take on the present and future of rack density, and how it may influence data center equipment and design?

KEVIN FACINELLI, Nortek Air Solutions

KEVIN FACINELLI, Nortek Air Solutions

Kevin Facinelli: Industry statistics indicate average rack density has increased from 7.2 kW in 2018 to 8.2 kW in 2019. Averages can be misleading because AI and computational rigorous applications can range from 15 to 30 kW.

Data center operators have to realize which technology offers future relevance. Rack densities have climbed exponentially since we entered the liquid cooling market 25 years ago, and our liquid cooling equipment is now in 10 percent of the top 100 supercomputers listed in the International Supercomputer Conference’s Top500 and Green500. More companies are entering the AI and computational market, but they’re also using open hardware or equipment with higher computational abilities in addition to supercomputers.

Therefore, colocations are now at a crossroads between two growth strategies. Some social media companies continue with 5 kW rack densities, but they’re distributing the increasing loads across a large amount of infrastructure as they grow. The other strategy is from the processor level where high density chip clusters are requiring computational density for training AI models. This strategy generates significantly more heat that must be rejected.

Air (hot aisle/cold aisle) and liquid can both provide adequate cooling for a floor of low and medium density racks, especially since the trend is toward higher ambient cooling temperatures without affecting uptime. However, a problem arises when high density 15 to 30-kW server racks are mixed in with mostly low and medium density racks in a data hall. Should the data center hall’s ambient temperature be lowered to accommodate the higher heat generation at a significant operational cost penalty? Or would direct liquid cooling, such as coolant distribution units (CDU) or rear door heat exchangers, efficiently accommodate the high density racks, while leaving the ambient temperature higher for the low and medium density equipment?

While there are plenty of suppliers of equipment such as CDUs, data center operators should instead find turnkey solution providers that offer the CDU, the plant that supplies the CDU and other distribution choices, the additional materials for complete integration into the data hall, the contractor, servicing, warranties and other services leading to single source responsibility.

ANGIE McMILLIN, Vertiv

ANGIE McMILLIN, Vertiv

Angie McMillin: We see the potential for the convergence of two trends to drive growth in rack density in certain applications. One is the introduction of new technologies such as the chips mentioned in the question. The other is the migration of data center capacity closer to users to support private and hybrid cloud applications. These edge deployments will be located in areas where real estate is scarce and more expensive, so there will be an incentive to increase density.

We’re still in the early phases of these trends and what we are seeing today is clusters of high-density racks being integrated into existing data centers. These clusters will serve as proving grounds for the technologies that will enable more widespread deployment of higher density racks. Liquid cooling and hot-scalable power systems will be among the technologies that benefit from these trends.

JUAN FONT, CoreSite

JUAN FONT, CoreSite

Juan Font:  Just like how customer deployments are growing, they are also increasingly becoming more power-dense. Artificial intelligence, machine learning or GUI-hungry gaming applications, to name a few, are use cases requiring high-performance compute, which we see increasingly placed at the edge. Such is the case with autonomous vehicle platforms or 5G applications requiring ultra-low latency.

Our strategy continues to be to support the widest array of use cases and power densities for applications that would benefit from a rich ecosystem of natively deployed fiber networks and cloud on-ramps in close proximity to large population centers. That said, in a multi-tenant data center environment – which is CoreSite’s predominant model – the key is to build enough flexibility in your design to be able to support high-performance computing or even liquid-cooled environments – but not at the expense of overbuilding your facility.

Data center design is impacted principally three ways: you have to augment cooling, increase the weight bearing of the floor space, as the racks are also heavier, and add more power distribution, as most of the new high-power density equipment also require large breaker sizes and 3-phase power.

TONY BISHOP, Digital Realty

TONY BISHOP, Digital Realty

Tony Bishop: The use of AI in the enterprise has accelerated dramatically in the past few years and with it, AI is bringing a significant need to meet new power demands and energy efficiency. Rack density, in particular, will become even more important in meeting power management needs. Colocation providers that can deliver on continuous, reliable energy needs will prove successful as AI applications and use continues to evolve.

But perhaps more important is how we overhaul IT infrastructure to bring infrastructure closer to the data generated by intensive applications like AI. AI workloads have moved to colocation facilities, yet the accrual of data is causing challenges for today’s IT infrastructure. As data accrues, it tends to attract additional services and applications, causing data gravity to emerge, which has the same effect on an enterprise that gravity has on objects on our planet. We often see data gravity as one of the main culprits preventing AI innovation. For example, if enterprises don’t account for data gravity, it can emerge as a big challenge that slows response times, creates information silos and prevents companies from providing excellent customer and employee experiences. To overcome these challenges, enterprises must bring computing resources closer to where data sets lie, which shrinks the time and distance needed to analyze the data that supports AI innovation.

At Digital Realty, we’ve expanded our capabilities in this area. One way we’re doing so is through our partnership with leading chip manufacturer, NVIDIA, to meet the needs of customers by introducing a new AI-ready infrastructure solution that enables them to rapidly deploy AI models in close proximity to their data sets. As a result, enterprises can gain access to AI-ready infrastructure to solve the global coverage, capacity and connectivity needs associated with deploying AI capabilities.

JAIME LEVERTON, iMasons and eStruxture Data Centers.

JAIME LEVERTON, iMasons and eStruxture Data Centers.

Jaime Leverton: The average data center rack density is certainly rising and we expect this trend to continue. There is more computing power being packed in smaller form-factor equipment. While this gives us the ability to concentrate more hardware into a smaller footprint, there’s also a greater need for high-density power and cooling. Designing a high-density data center implies bringing cooling closer to the source of heat through high-efficiency cooling units, as well as employing high-density power distribution solutions. As well, hot-aisle containment is a must for high-density racks as it is equally highly efficient and cost-effective.

We also see incorporating smart IoT systems into our design to better measure temperature and humidity and automate and adjust cooling accordingly in real-time. A positive effect of high-density deployments is that we clearly see a decrease in the PUE numbers.

PHILLIP MARANGELLA, EdgeConneX

PHILLIP MARANGELLA, EdgeConneX

Phillip Marangella: Deployment densities are definitely on the rise as a result of technologies like AI and things like cloud gaming, for example. Requests for rack densities in excess of 20 KWs per rack are certainly increasing. Being able to adapt and utilize technologies like liquid or immersion cooling will be key to support those densities.

At the same time, doing so, while factoring in sustainability, will be a key challenge as customers and providers look to meet their goals of being carbon neutral.

NEXT: How the pandemic is impacting data center networks and interconnection.

Keep pace with the fact-moving world of data centers and cloud computing by following us on Twitter and Facebook, connecting with me on LinkedIn, and signing up for our weekly newspaper using the form below:

LinkedinTwitterFacebookSubscribe
Mail

Tagged With: Artificial Intelligence, CoreSite, Digital Realty, EdgeConneX, Infrastructure Masons, Nortek, Vertiv

Newsletters

Stay informed: Get our weekly updates!

Are you a new reader? Follow Data Center Frontier on Twitter or Facebook.

About Rich Miller

I write about the places where the Internet lives, telling the story of data centers and the people who build them. I founded Data Center Knowledge, the data center industry's leading news site. Now I'm exploring the future of cloud computing at Data Center Frontier.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

  • Facebook
  • Instagram
  • LinkedIn
  • Pinterest
  • Twitter

Voices of the Industry

Water Scarcity: How Data Centers Can Help

Water Scarcity: How Data Centers Can Help Water scarcity is a looming global crisis. Kris Holla, Group Vice President, Channel Sales, Nortek Air Solutions, and Michael Lesniak, AVP Global Corporate Accounts, Global Data Center Solutions at Nalco Water, discuss how data centers can conserve the planet’s diminishing fresh water supply and practice water stewardship.   

DCF Spotlight

The COVID-19 Crisis and the Data Center Industry

The COVID-19 pandemic presents strategic challenges for the data center and cloud computing sectors. Data Center Frontier provides a one-stop resource for the latest news and analysis for decision-makers navigating this complex new landscape.

An aerial view of major facilities in Data Center Alley in Ashburn, Virginia. (Image: Loudoun County)

Northern Virginia Data Center Market: The Focal Point for Cloud Growth

The Northern Virginia data center market is seeing a surge in supply and an even bigger surge in demand. Data Center Frontier explores trends, stats and future expectations for the No. 1 data center market in the country.

See More Spotlight Features

White Papers

London data center capacity

Bringing Data Center Capacity to a Crowded European City

The wholesale adoption of – and reliance on – electronic payments, the inexorable rise of artificial intelligence (AI), the continuing roll-out and acceptance of the Internet of Things (IoT), and more all means the dataverse is expanding exponentially. Get the new white paper from Echelon Data Centres that explores why now is the right time to expand London data center capacity, in particular, as well as outlines the company’s recent projects in the Docklands.

Get this PDF emailed to you.

We always respect your privacy and we never sell or rent our list to third parties. By downloading this White Paper you are agreeing to our terms of service. You can opt out at any time.

Newsletters

Get the Latest News from Data Center Frontier

Job Listings

RSS Job Openings | Peter Kazella and Associates, Inc

  • Generator Technician - East Rutherford, NJ
  • Data Center QA / QC Manager - Huntsville, AL
  • Data Center QA / QC Technician - Omaha, NE
  • Data Center QA / QC Manager - Phoenix, AZ
  • Data Center QA / QC Manager - Bend, OR

See More Jobs

Data Center 101

Data Center 101: Mastering the Basics of the Data Center Industry

Data Center 101: Mastering the Basics of the Data Center Industry

Data Center Frontier, in partnership with Open Spectrum, brings our readers a series that provides an introductory guidebook to the ins and outs of the data center and colocation industry. Think power systems, cooling, solutions, data center contracts and more. The Data Center 101 Special Report series is directed to those new to the industry, or those of our readers who need to brush up on the basics.

  • Data Center Power
  • Data Center Cooling
  • Strategies for Data Center Location
  • Data Center Pricing Negotiating
  • Cloud Computing

See More Data center 101 Topics

About Us

Charting the future of data centers and cloud computing. We write about what’s next for the Internet, and the innovations that will take us there. We tell the story of the digital economy through the data center facilities that power cloud computing and the people who build them. Read more ...
  • Facebook
  • LinkedIn
  • Pinterest
  • Twitter

About Our Founder

Data Center Frontier is edited by Rich Miller, the data center industry’s most experienced journalist. For more than 20 years, Rich has profiled the key role played by data centers in the Internet revolution. Meet the DCF team.

TOPICS

  • 5G Wireless
  • Cloud
  • Colo
  • Connected Cars
  • Cooling
  • Cornerstone
  • Coronavirus
  • Design
  • Edge Computing
  • Energy
  • Executive Roundtable
  • Featured
  • Finance
  • Hyperscale
  • Interconnection
  • Internet of Things
  • Machine Learning
  • Network
  • Podcast
  • Servers
  • Site Selection
  • Social Business
  • Special Reports
  • Storage
  • Sustainability
  • Videos
  • Virtual Reality
  • Voices of the Industry
  • Webinar
  • White Paper

Copyright Data Center Frontier LLC © 2021

X - Close Ad