• About Us
  • Partnership Opportunities
  • Privacy Policy

Data Center Frontier

Charting the future of data centers and cloud computing.

  • Cloud
    • Hyperscale
  • Colo
    • Site Selection
    • Interconnection
  • Energy
    • Sustainability
  • Cooling
  • Technology
    • Internet of Things
    • AI & Machine Learning
    • Edge Computing
    • Virtual Reality
    • Autonomous Cars
    • 5G Wireless
    • Satellites
  • Design
    • Servers
    • Storage
    • Network
  • Voices
  • Podcast
  • White Papers
  • Resources
    • COVID-19
    • Events
    • Newsletter
    • Companies
    • Data Center 101
  • Jobs
You are here: Home / Colo / The Density Debate: Is Cooling Door Adoption a Sign of Coming Shift?

The Density Debate: Is Cooling Door Adoption a Sign of Coming Shift?

By Rich Miller - December 17, 2015

The Density Debate: Is Cooling Door Adoption a Sign of Coming Shift?

The data hall at Colovore in Santa Clara, with a clear tile displaying the water pipes beneath the raised floor. (Photo: Colovore)

LinkedinTwitterFacebookSubscribe
Mail

SANTA CLARA, Calif. – Density is coming to the data center. But thus far, it’s been taking its time.

Over the past decade, there have been numerous predictions of the imminent arrival of higher rack power densities. Yet extreme densities remain limited, primarily seen in high performance computing (HPC) and specialty processing such as bitcoin mining.

The team from Colovore believes data centers will be denser, and the shift will accelerate over the next few years. The colocation specialist believes it is on the front edge of a broader move to denser server cabinets, driven in part by a generational change in IT teams.

Traditional data hall designs will struggle to cool these higher densities. That’s why Colovore is filling its data center in Santa Clara with high-density racks featuring water-chilled rear-door cooling units.

“When we came to market last year, people weren’t buying the density yet,” said Sean Holzknecht, the co-founder and President of Colovore.” This year, everyone’s base requirement is 10kW at a minimum. It’s really flipped.”

Colovore is not alone in adopting rear-door heat exchangers at scale. LinkedIn is implementing a new data center design featuring rear-door cooling units for its new facility near Portland, Oregon. The company said its next-generation design will use “cabinet-level heat rejection” that will double the cabinet densities from its previous data center builds. The new data center will be hosted by Infomart, where LinkedIn has reportedly leased 8 megawatts of space.[clickToTweet tweet=”LinkedIn is implementing water-chilled cooling doors at its new data center near Portland.” quote=”LinkedIn is implementing water-chilled cooling doors at its new data center near Portland.”]

As the rear-door cooling unit gains traction in both the colocation and hyperscale markets, some see it as one of several portents that rack power densities are finally starting to edge higher. It’s a trend that offers both challenges and opportunities for data center operators, both of which are beginning to drive new data center designs like those at Colovore and LinkedIn.

Healthy Appetite for “Headroom”

How long has the data center industry been talking about the arrival of higher densities? My first story on the topic dates to 2002, when cooling vendors demonstrated water-cooled cabinets at a meeting of the 7×24 Exchange, and predicted a “new paradigm” in cooling. Some predictions, such as an industry norm of 150 watts per square foot by 2005, were premature. Others, like the shift to measuring density in watts per cabinet rather than watts per square foot, came to pass fairly quickly.

If density is a long-awaited problem, it’s also one that data center customers have been bracing for in their capacity planning, seeking “headroom” for denser workloads and often provisioning more cooling than they are likely to need.

“The reality is that everyone says they want 200 to 250 watts per square foot, but almost nobody’s using it,” said Jeff Burges, President and founder of colocation specialist DataSite. “There will be some high density users, but also a lot of low density users.”[clickToTweet tweet=”Jeff Burges: Everyone says they want 200 to 250 watts per SF, but almost nobody’s using it.” quote=”Jeff Burges: Everyone says they want 200 to 250 watts per SF, but almost nobody’s using it.”]

Free Resource from Data Center Frontier White Paper Library

Gaming
From Console to Cloud
This white paper from Iron Mountain explores the current challenges, drivers, and opportunities for gaming digital infrastructure.
We always respect your privacy and we never sell or rent our list to third parties. By downloading this White Paper you are agreeing to our terms of service. You can opt out at any time.

Get this PDF emailed to you.

The typical enterprise data center user is probably running densities of 3kW to 5kW per rack, according to Shawn Conaway, Director of Cloud Services at FIS. “I see going on more often is pockets of high-density workloads, especially in internal private cloud, where you can see 10 to 15 kW,” said Conaway. “I think we’ll see more of this.”

Conaway said his firm, which specializes in IT solutions for the financial services industry, runs its own racks at 15kW to 30kW a cabinet. But there are those who are testing the boundaries of even higher densities.

What Density Means for Design

“We are pushing 50kW a rack,” said Richard Donaldson, the Director of Infrastructure Management and operations at eBay. “We’re on our fifth generation server design, which means you have to have a data center that can support that. The technologies are shifting. We’re driving towards density because we’re driving toward the lowest cost.”

This data hall inside the eBay data center in Phoenix uses Motivair cooling doors

This data hall inside the eBay data center in Phoenix uses Motivair cooling doors. And also cool blue lighting. (Photo: eBay)

eBay was among the first companies to use water-chilled rear door cooling units at scale. In 2014, the company switched from in-row air cooling units to water-chilled rear door cooling units in a data hall in its Phoenix data center. The room housed 16 rows of racks, each housing 30kW to 35kW of servers, which required the use of six in-row air handlers in each row. That meant eBay had to sacrifice six rack positions for cooling gear. Switching to rear-door units from Motivair allowed eBay to recapture those six racks and fill them with servers, boosting compute capacity within the same footprint.

“I would argue that given where the technology is headed, we’re going to be seeing more density,” said Donaldson. “We’re now seeing densities shift from 1kW per rack to 5 kW a rack. That trend is coming. We’re already seeing it in Equinix and Digital Realty.”

That trend is being felt by data center developer DuPont Fabros Technology, which is updating its data center design to accommodate a wider range of rack densities.

“In our portfolio, we see growing gap between our 3kW (per cabinet) customers and our 15kW customers,” said Scott Davis, Executive Vice President for Data Center Operations at DuPont Fabros. “In our latest product we have the ability to increase the density up to about 300 watts per square foot.”

DuPont Fabros and its cohorts in the multi-tenant data center space are adapting to increasingly variable requirements in both power density and reliability. For some service providers, higher densities offer the opportunity to specialize and differentiate.

A Look Inside Colovore

As you walk into Colovore’s data center, you get a sense of what a new paradigm in cooling might look like. The noise and air movement felt in many data centers is far more subdued.

The cooling doors allow several significant changes in data center design. They attach to the back of a cabinet, and use the server fans within the rack to provide airflow through the unit, pushing hot air through the door-based coil that cools the air and returns it to the room at close to the same temperature as the air entering the rack.

These units can cool higher densities than air cooling – up to 35kW per rack – and eliminate the need to place CRACs (computer room air conditioners) around the perimeter of the room, making more room for cabinets. It allows users to can run the data hall at a warmer temperature, in this case just below 80 degrees.

“If you’re used to your data center being a meat locker, it’s odd to be in a new experience,” said Holzknecht. “In our facility, there’s no differential between the hot and cold aisle.”

Colovore execs Peter Harrison and Sean Holknecht

Colovore executives Peter Harrison (left) and Sean Holzknecht inside the company’s data center in Santa Clara. (Photo: Rich Miller)

Instead of cold air, the area under the raised floor houses cooling pipes, which connect with a heat exchanger and cooling tower in the equipment yard. Industrial-strength hoses connect the water lines to the cooling door, which is made by Liebert (Emerson Network Power). The supply lines are integrated into the door hinges for protection. Valves are closed for cabinets that are not in use.

The cooling door seemed bound for great things after it prevailed in the “Chill Off,” the groundbreaking 2008 competition between cooling vendors, where it was judged to be the most efficient approach to cooling a 10kW rack. But the design has since been adopted sparingly as the data center industry shifted focus to free cooling using fresh air. Colovore believes the cooling door is an idea whose time has come.

“It’s an incredibly efficient system, and you can turn it on one cabinet at a time,” said Holzknecht. “Because we are so dense, we can put the same amount of gear in fewer cabinets. It’s less expensive to build densely. It’s cheaper for us as a provider as well. Building denser is always less expensive.”

“The revenue per cabinet is so much more than the additional cost to build per cabinet,” said Peter Harrison, the CTO of Colovore and a co-founder along with Holzknecht and CFO Ben Coughlin. “I think we’ll start seeing more colo providers using this technology. We’re definitely early adopters. The acceptance will be accelerating.”

Colovore’s interest in density was tied to its original plan to create multi-use facilities in urban areas that combined high-density data centers with co-working space. That changed when the Colovore team discovered the Santa Clara property, which had 9 megawatts of power capacity ready to go and ease of expansion through an adjacent substation for Silicon Valley Power.

Differentiator in a Competitive Market?

In the crowded Silicon Valley market, designing for density turned out to be a differentiator for Colovore.  “You’ve got a lot of good-looking data centers that can only handle a little density,” said Holzknecht. “Everyone’s migrating out of these telcos. A lot of their facilities are 20 years old. A lot of them are realizing they have to retrofit, and they don’t have the appetite for the investment.”

The ability to cool dense workloads has already won the company some deals. One cloud tenant needed to house a 750kW requirement. “We won the deal because we could do that (load) in 50 cabinets and just 2,500 square feet,” said Holzknecht.

Colovore’s facility was owned by Les Pelio, a veteran developer of data center properties in Silicon Valley, who initially positioned it as a “container colo” center to house data center modules. “The big thing (Pelio) did is that there’s 9 megawatts of power already here,” said Holzknecht. “We can build the next three phases at once if the demand calls for it. We were fortunate to find this place and work with an experienced data center developer.”

Colovore data center in Santa Clara

The Colovore equipment yard features a heat exchanger and cooling tower. (Photo: Rich Miller)

Colovore has built out the first 2 megawatts of data center space, which is rapidly filling, and will soon add another 2 megawatt data hall. As it continues to grow, Colovore can add two more 2.5 megawatt phases to reach its 9 megawatt capacity. Each time it adds a data hall, it will also install skid-mounted modular  power room equipment from Digital Realty.

Digital Realty is both a neighbor and partner. The Colovore facility on Space Park Drive is virtually surrounded by data centers owned by Digital Realty, which is also an investor in Colovore. Scott Peterson, the Chief Investment Office at Digital Realty, is a member of Colovore’s board.

“We’ve been friends with the (Digital Realty) management team forever,” said Holzknecht. “We’re kind of a petri dish for a company the size of Digital Realty. These guys are in every market we want to go into, so we may want to partner with them. They have properties that are not active facilities, and that’s one of the things we’re interested in. But first we’ve got to execute here in Santa Clara.”

Generational Shift Could Accelerate Density

An interesting wrinkle is that Colovore believes the presence of younger engineers in data center teams is changing attitudes about density and water cooling.

“It’s a generational thing in a lot of ways,” said Holzknecht. “Our selling cycle is about engineers. We’re seeing a lot of 200KW to 1 MW deals, and internal infrastructure is often the requirement. At some of these companies, everything is virtualized and cloudified. Instead of having lazy servers being underutilized, they have grids.”[clickToTweet tweet=”Colovore believes younger engineers are driving changing attitudes about water cooling.” quote=”Colovore believes younger engineers are driving changing attitudes about water cooling.”]

Harrison, who worked at Google before teaming with Holzknecht and CFO Ben Coughlin to found Colovore, agrees that the company has done particularly well with “cloud native” companies that are comfortable with DevOps principles.

“A lot of them grew up with PC gaming and water cooling right in their living room,” said Harrison. “A lot of the younger companies are buying new equipment, and new equipment needs density. (Density is) a big deal for many of these customers.”

Does the adoption of cooling doors portend a change in density? At Data Center Frontier we will be tracking the latest trends in cooling and data center design. You can follow us on Twitter and Facebook or get our news by email using the form below.

LinkedinTwitterFacebookSubscribe
Mail

Tagged With: Colovore, Data Center Cooling, eBay, LinkedIn

Newsletters

Stay informed: Get our weekly updates!

Are you a new reader? Follow Data Center Frontier on Twitter or Facebook.

About Rich Miller

I write about the places where the Internet lives, telling the story of data centers and the people who build them. I founded Data Center Knowledge, the data center industry's leading news site. Now I'm exploring the future of cloud computing at Data Center Frontier.

Comments

  1. hahn@mcmaster.ca'Mark Hahn says

    December 21, 2015 at 9:26 am

    as usual, I’m puzzled by this article. in the HPC world, density has been >10 KW/rack, with conventional air cooling, for almost a decade (and I’m talking about COTS 1U clusters, not bespoke HPC.)

  2. kevin.lancaster@aquacooling.co.uk'Kevin Lancaster says

    April 7, 2016 at 10:19 am

    Anyone considering or already using rear door cooling but concerned about cooling water close to their IT hardware should take a look at the Leak Prevention System from QCooling.com

    Energy benefits are amazing with the Usystems door http://www.aquacooling.co.uk/data-centre-cooling/

    • kevinomccann@gmail.com'Kevin O. McCann says

      December 19, 2016 at 6:49 am

      I have been watching aquacooling since they got the queen’s award. Who’s going to buy them and help them take the stage?

  • Facebook
  • Instagram
  • LinkedIn
  • Pinterest
  • Twitter

Voices of the Industry

Six Ways Edge Computing is Changing the Digital Ecosystem Landscape

Six Ways Edge Computing is Changing the Digital Ecosystem Landscape The pandemic exposed a nation-wide digital divide. Brad Alexander of DartPoints, outlines how edge computing is bridging that gap and changing the digital ecosystem landscape.

White Papers

sustainability metrics

Guide to Environmental Sustainability Metrics for Data Centers

As more and more companies are reporting on their Environmental, Social, and Governance (ESG) programs, there’s a need for standardized sustainability metrics, especially in the data center industry. A new white paper from Schneider Electric proposes five categories of metrics for data centers to use when reporting on environmental sustainability.

Get this PDF emailed to you.

We always respect your privacy and we never sell or rent our list to third parties. By downloading this White Paper you are agreeing to our terms of service. You can opt out at any time.

DCF Spotlight

Data center modules on display at the recent Edge Congress conference in Austin, Texas. (Photo: Rich Miller)

Edge Computing is Poised to Remake the Data Center Landscape

Data center leaders are investing in edge computing and edge solutions and actively looking at new ways to deploy edge capacity to support evolving business and user requirements.

An aerial view of major facilities in Data Center Alley in Ashburn, Virginia. (Image: Loudoun County)

Northern Virginia Data Center Market: The Focal Point for Cloud Growth

The Northern Virginia data center market is seeing a surge in supply and an even bigger surge in demand. Data Center Frontier explores trends, stats and future expectations for the No. 1 data center market in the country.

See More Spotlight Features

Newsletters

Get the Latest News from Data Center Frontier

Job Listings

RSS Job Openings | Pkaza Critical Facilities Recruiting

  • Critical Power Energy Manager - Data Center Development - Ashburn, VA
  • Site Development Manager - Data Center - Ashburn, VA
  • Data Center Facility Operations Director - Chicago, IL
  • Electrical Engineer - Senior - Dallas, TX
  • Mechanical Commissioning Engineer - Calgary, Alberta

See More Jobs

Data Center 101

Data Center 101: Mastering the Basics of the Data Center Industry

Data Center 101: Mastering the Basics of the Data Center Industry

Data Center Frontier, in partnership with Open Spectrum, brings our readers a series that provides an introductory guidebook to the ins and outs of the data center and colocation industry. Think power systems, cooling, solutions, data center contracts and more. The Data Center 101 Special Report series is directed to those new to the industry, or those of our readers who need to brush up on the basics.

  • Data Center Power
  • Data Center Cooling
  • Strategies for Data Center Location
  • Data Center Pricing Negotiating
  • Cloud Computing

See More Data center 101 Topics

About Us

Charting the future of data centers and cloud computing. We write about what’s next for the Internet, and the innovations that will take us there. We tell the story of the digital economy through the data center facilities that power cloud computing and the people who build them. Read more ...
  • Facebook
  • LinkedIn
  • Pinterest
  • Twitter

About Our Founder

Data Center Frontier is edited by Rich Miller, the data center industry’s most experienced journalist. For more than 20 years, Rich has profiled the key role played by data centers in the Internet revolution. Meet the DCF team.

TOPICS

  • 5G Wireless
  • Cloud
  • Colo
  • Connected Cars
  • Cooling
  • Cornerstone
  • Coronavirus
  • Design
  • Edge Computing
  • Energy
  • Executive Roundtable
  • Featured
  • Finance
  • Hyperscale
  • Interconnection
  • Internet of Things
  • Machine Learning
  • Network
  • Podcast
  • Servers
  • Site Selection
  • Social Business
  • Special Reports
  • Storage
  • Sustainability
  • Videos
  • Virtual Reality
  • Voices of the Industry
  • Webinar
  • White Paper

Copyright Endeavor Business Media© 2022