• About Us
  • Partnership Opportunities
  • Privacy Policy

Data Center Frontier

Charting the future of data centers and cloud computing.

  • Cloud
    • Hyperscale
  • Colo
    • Site Selection
    • Interconnection
  • Energy
    • Sustainability
  • Cooling
  • Technology
    • Internet of Things
    • AI & Machine Learning
    • Edge Computing
    • Virtual Reality
    • Autonomous Cars
    • 5G Wireless
    • Satellites
  • Design
    • Servers
    • Storage
    • Network
  • Voices
  • Podcast
  • White Papers
  • Resources
    • COVID-19
    • Events
    • Newsletter
    • Companies
    • Data Center 101
  • Jobs
You are here: Home / Cooling / Microsoft Tests Immersion Cooling Tech from Bitcoin Mining for its Cloud Servers

Microsoft Tests Immersion Cooling Tech from Bitcoin Mining for its Cloud Servers

By Rich Miller - March 12, 2021

Microsoft Tests Immersion Cooling Tech from Bitcoin Mining for its Cloud Servers

Bubbles rise from servers insider a tank of cooling liquid in the Microsoft lab., where the company is testing two-phase liquid cooling technology (Image: Microsoft)

LinkedinTwitterFacebookSubscribe
Mail

Microsoft is seeking better ways to manage powerful new AI hardware in its cloud computing operation, and focusing on immersion cooling technology used in bitcoin mining as the most promising technology for future high-density data centers.

The company is test-driving a setup in which servers are dunked in tanks of cooling fluid to manage rising heat densities. Citing the need to prepare for more powerful new chips and rising rack densities, the company is test-driving “promising” immersion technologies that are currently used in bitcoin mining operations.

“We’ve been investigating how we can achieve better cooling efficiency, and liquid cooling is what we’ve been focusing on,” said Mark Russinovich, the Chief Technology Officer of Microsoft’s Azure Cloud, said in a presentation at Microsoft Ignite. “What we’ve locked on as likely where we’re headed in our data centers is two-phase liquid cooling, and we’ve made a ton of progress down this path.”

Microsoft showed off a cooling lab where it is cooling servers inside a two-phase immersion system from Allied Control, which is owned by bitcoin mining specialist Bitfury, which uses the design to support power densities of up to 250 kW per rack. Immersion can deliver exceptional power efficiency because it uses sealed tanks that don’t require the raised floors or room-level air cooling found in most commercial data centers.

That’s appealing to the Microsoft team, which likes the potential economic gains from packing more compute into a smaller amount of real estate.

“How do we pack and leverage the floor space in our data centers more efficiently?” Russinovich asked. “If we’re air cooling them, which is the way we’re cooling these data centers today, we’ve got to leave hot air aisles and cool air aisles and have huge HVAC systems that are pumping air in and out of the data center. It’s a large use of space.”

“Liquid cooling affects the whole ecosystem,” said Husam Alissa, a Principal Engineer in Microsoft’s liquid cooling lab. “When you take a look at the data center and the server and the sustainability promise that Microsoft is making, liquid cooling can help us get there faster. With liquid cooling, we can have higher density racks that could lead to smaller data center footprint, and lower their center energy consumption from the mechanical cooling perspective, and also from the server perspective, because we could reduce or remove the fans from the servers.”

Microsoft Moonshots Could Transform the Data Center

The liquid cooling research is one of many ways that Microsoft is testing new technologies that could radically change data centers. As the largest player in the market for leased data center space, Microsoft’s decisions on its core server, power and cooling technologies could have outsized influence in the data center industry. These moonshots and science experiments are in various phases of deployment, but make it clear that Microsoft is preparing to build a very different kind of future cloud as it seeks to meet aggressive carbon reduction goals.

These initiatives include:

  • Underwater data centers for edge computing scenarios, complete with controlled environments that may also extend the life of servers, bring benefits to data center economics.
  • The use of hydrogen fuel cells to power its data center servers, a first step toward adopting new technologies to fulfill its pledge to eliminate the use of diesel fuel in its emergency power systems
  • Data storage using DNA and holograms to house massive amounts of data, and infrastructure that could disrupt how data centers are designed and operate.
  • Containerized data modules that use satellite broadband connectivity, allowing capacity to be deployed in geographies with no fiber infrastructure.

In an era of exciting advances in data center design, Microsoft’s reach seeks to extend the frontiers of computing, with major implications for the broader industry. Thus far, these initiatives have yet to be deployed at scale. But Microsoft’s focus on liquid cooling is being driven by changes in hardware that command a response.

Free Resource from Data Center Frontier White Paper Library

Cooling
Sustainably Meeting High Density Cooling Challenges
As the power densities of IT equipment have increased over the past twenty years, data center designers and operators face a number of challenges when it comes to managing this intensified heat load. A new special report from Data Center Frontier and Nautilus Data Technologies explores issues and potential solutions to supporting high density cooling in an efficient and sustainable way.
We always respect your privacy and we never sell or rent our list to third parties. By downloading this White Paper you are agreeing to our terms of service. You can opt out at any time.

Get this PDF emailed to you.

The rise of artificial intelligence, and the hardware that supports it, is reshaping the data center industry’s relationship with servers. New hardware for AI workloads is packing more computing power into each piece of equipment, boosting the power density – the amount of electricity used by servers and storage in a rack or cabinet – and the accompanying heat. The trend is challenging traditional practices in data center cooling, and prompting data center operators to adapt new strategies to support high-density racks.

More Server Power Requires More Cooling

In his presentation at Ignite, Russinovich noted the heat generated by powerful new hardware like the NVIDIA A100 GPU chips, which feature prominently in new Azure Cloud offerings.

“The A100 really represent kind of trend that we’ve been seeing … for more and more power consumption per server,” said Russinovich, who said general purpose chips were also featuring more cores and drawing more power.

He said Microsoft has experimented with several approaches to liquid cooling, including a cold plate design, which typically contains a tubing system filled with liquid refrigerant. Russinovich tested cold plates using a personal gaming system, but said this approach would be hard to scale.

“That is a great way to cool, but it’s got the downside that every single server has to be custom fitted for the pipes and the cold plates,” he said. “It’s not a one size fits all model.”

Microsoft Azure CTO Mark Russinovich shares a cold plate cooling design he used in a gaming system to text advanced cooling designs. Nice rig, but needs more colored lights. (Image: Microsoft)

Microsoft Azure CTO Mark Russinovich shares a cold plate cooling design he used in a gaming system to text advanced cooling designs. Nice rig, but needs more colored lights. (Image: Microsoft)

He cited similar challenges with a single-phase immersion cooling system in which liquid coolant is introduced into the server chassis.

That’s why Microsoft is focused on two-phase immersion cooling, in which servers are immersed in a coolant fluid that boils off as the chips generate heat, removing the heat as it changes from liquid to vapor. The vapor then condenses into liquid for reuse, all without a pump. Data center designs for this “phase change” cooling were pioneered by 3M as a use case for its Novec engineered fluid, which has a low boiling point. (Flashback: See my initial coverage of this technology in 2012)

Bitcoin Tech Comes to Hyperscale

The video from Ignite showed a two-phase immersion system operating in Microsoft’s liquid cooling lab. The servers were enclosed in a system from Allied Control, which has been a pioneer in adapting extreme density cooling systems using Novec (which is not named as the coolant in the Microsoft video, although 3M-branded boxes are visible in the background). In 2014 it created a high-density immersion cooling system that allowed its client to deploy a bitcoin mining operation on the upper floors of a skyscraper in Hong Kong.

The company was then acquired by bitcoin specialist Bitfury, which brought the technology to scale in containers filled with immersion tanks, which it used to create a 40-megawatt facility that l features power densities of 250kW per enclosure.

That disruptive potential is why the Open Compute Project has been working to enable wider adoption of liquid cooling, citing demand from hyperscale computing providers, as well as new applications in edge computing. Microsoft has been among the participants in the OCP liquid cooling initiatives, which followed Google’s revelation that it has shifted to liquid cooling with its latest hardware for artificial intelligence.

Microsoft signaled its growing interest in liquid cooling in a blog post last year on its OCP initiatives.

“While liquid cooling is a technology that has been used in specific use cases, such as bitcoin mining, we are not only investing in the solutions and technologies that will power new architectures, but also focusing intensely on the challenges that will come into play as we look to extend the reach of these capabilities to a hyperscale cloud,” the company wrote.

LinkedinTwitterFacebookSubscribe
Mail

Tagged With: Bitcoin, BitFury, Immersion, Immersion cooling, Microsoft

Newsletters

Stay informed: Get our weekly updates!

Are you a new reader? Follow Data Center Frontier on Twitter or Facebook.

About Rich Miller

I write about the places where the Internet lives, telling the story of data centers and the people who build them. I founded Data Center Knowledge, the data center industry's leading news site. Now I'm exploring the future of cloud computing at Data Center Frontier.

Comments

  1. ammm@wp.pl'Andrzej says

    March 14, 2021 at 9:11 am

    Cray was first.

    • steve.harrington@chilldyne.com'Steve Harrington says

      March 15, 2021 at 5:45 pm

      And Cray is not using immersion any more.

  2. bwilborn@pagethink.com'B. Wilborn says

    March 25, 2021 at 8:50 am

    The heat still has to go somewhere, so what is the transport mechanism between the coolant bath and the heat-rejection equipment, and is that heat-rejection scheme based also on a new technology?

  3. waynehagelberg@icloud.com'Wayne Hagelberg says

    March 25, 2021 at 2:31 pm

    Excess heat energy can be converted to electricity or to chilled water for building cooling systems or used to heat buildings in cold climates.

  4. david.sundin@engineeredfluids.com'David Sundin says

    April 8, 2021 at 12:51 pm

    I’m old enough to remember the last time that a nonflammable, halogenated dielectric fluid was popular – it was PCBs. That chemical was banned worldwide because it was toxic and bioaccumulative – just like fluorinated fluids. Billions of dollars were spent cleaning up those “forever chemicals”; I expect that the same will happen with these, too.

  • Facebook
  • Instagram
  • LinkedIn
  • Pinterest
  • Twitter

Voices of the Industry

Overcoming Supply Chain Roadblocks: How to Avoid Disruptions in Your Data Center

Overcoming Supply Chain Roadblocks: How to Avoid Disruptions in Your Data Center The data center industry continues to experience significant global supply chain problems. Brett Williams of Service Express, explores the importance of leveraging the secondary hardware market to overcome supply chain roadblocks.

White Papers

Gaming

From Console to Cloud

This white paper from Iron Mountain explores the current challenges, drivers, and opportunities for gaming digital infrastructure.

Get this PDF emailed to you.

We always respect your privacy and we never sell or rent our list to third parties. By downloading this White Paper you are agreeing to our terms of service. You can opt out at any time.

DCF Spotlight

Data center modules on display at the recent Edge Congress conference in Austin, Texas. (Photo: Rich Miller)

Edge Computing is Poised to Remake the Data Center Landscape

Data center leaders are investing in edge computing and edge solutions and actively looking at new ways to deploy edge capacity to support evolving business and user requirements.

An aerial view of major facilities in Data Center Alley in Ashburn, Virginia. (Image: Loudoun County)

Northern Virginia Data Center Market: The Focal Point for Cloud Growth

The Northern Virginia data center market is seeing a surge in supply and an even bigger surge in demand. Data Center Frontier explores trends, stats and future expectations for the No. 1 data center market in the country.

See More Spotlight Features

Newsletters

Get the Latest News from Data Center Frontier

Job Listings

RSS Job Openings | Pkaza Critical Facilities Recruiting

  • Critical Power Energy Manager - Data Center Development - Ashburn, VA
  • Site Development Manager - Data Center - Ashburn, VA
  • Data Center Facility Operations Director - Chicago, IL
  • Electrical Engineer - Senior - Dallas, TX
  • Mechanical Commissioning Engineer - Calgary, Alberta

See More Jobs

Data Center 101

Data Center 101: Mastering the Basics of the Data Center Industry

Data Center 101: Mastering the Basics of the Data Center Industry

Data Center Frontier, in partnership with Open Spectrum, brings our readers a series that provides an introductory guidebook to the ins and outs of the data center and colocation industry. Think power systems, cooling, solutions, data center contracts and more. The Data Center 101 Special Report series is directed to those new to the industry, or those of our readers who need to brush up on the basics.

  • Data Center Power
  • Data Center Cooling
  • Strategies for Data Center Location
  • Data Center Pricing Negotiating
  • Cloud Computing

See More Data center 101 Topics

About Us

Charting the future of data centers and cloud computing. We write about what’s next for the Internet, and the innovations that will take us there. We tell the story of the digital economy through the data center facilities that power cloud computing and the people who build them. Read more ...
  • Facebook
  • LinkedIn
  • Pinterest
  • Twitter

About Our Founder

Data Center Frontier is edited by Rich Miller, the data center industry’s most experienced journalist. For more than 20 years, Rich has profiled the key role played by data centers in the Internet revolution. Meet the DCF team.

TOPICS

  • 5G Wireless
  • Cloud
  • Colo
  • Connected Cars
  • Cooling
  • Cornerstone
  • Coronavirus
  • Design
  • Edge Computing
  • Energy
  • Executive Roundtable
  • Featured
  • Finance
  • Hyperscale
  • Interconnection
  • Internet of Things
  • Machine Learning
  • Network
  • Podcast
  • Servers
  • Site Selection
  • Social Business
  • Special Reports
  • Storage
  • Sustainability
  • Videos
  • Virtual Reality
  • Voices of the Industry
  • Webinar
  • White Paper

Copyright Data Center Frontier LLC © 2022