The data center of the future will be different than today's facilities. That's why Meta has decided to overhaul its data center design to optimize its facilities for artificial intelligence.
Meta is still working on what this next-generation computing facility will look like. For the moment, it is pausing construction on a number of its data center projects so it can "rescope" its new data centers with the pending design focused on AI, which is the lynchpin of Meta founder Mark Zuckerberg's vision for an immersive metaverse community.
The shift at Meta was first reported by DCD, which noted a construction halt to an expansion at the company's data center campus in Odense, Denmark and learned that the move is part of a larger rethinking of the company's infrastructure.
Industry sources confirm that Meta is making a strategic shift in its data center designs, which will focus on optimizing new facilities to better support artificial intelligence. The expansion in Odense - which would have added a fourth and fifth building on the campus - is the only construction project being halted completely.
But the shift will impact the pace of construction at some campuses around the world. Meta typically builds between five and nine data center buildings on each of its huge campuses. Data centers that are under construction will be completed, but work on future phases may be paused so they can conform to the new design standard, sources say.
Why Meta's Strategy Shift Matters
Meta's decision is significant on a number of fronts, extending well beyond its own infrastructure. Meta operates 18 data center campuses around the globe to support its Facebook, Instagram and Messenger services. These data centers represent an investment of $16 billion and more than that span 40 million square feet of space.
Some possible implications:
- The new Meta design could provide a boost to liquid cooling. Although Meta is not revealing details of its new design, it has recently outlined plans for a gradual shift to a liquid-cooled infrastructure. Accelerating these plans would provide a boost to the evolving commercial ecosystem around liquid cooling.
- Wall Street may welcome a slowdown in Meta's construction timeline, as securities analysts have been wary of the company's infrastructure spending to support its metaverse vision. Shares of Meta plunged 25% in a single day in October after the company reaffirmed its plans to invest heavily in metaverse infrastructure.
- Meta is one of the largest employers of data center construction teams, with multi-phase campus buildouts that keep construction teams busy for years. Any pause in its construction projects would be closely watched by a data center development community that's struggling to find enough workers to meet delivery timelines.
- New designs from Meta could reinforce the relevance of the open hardware movement, as the company shares most of its innovations in hardware and data centers with the Open Compute Project and its growing ecosystem.
How AI Might Shift Meta's Design
Here at DCF the data center has always been our prism for analyzing changes in technology and business. While many headlines are focusing on the construction halt in Odense, we think the upcoming new design merits attention, and likely reflects an ongoing shift in which data centers are taking on more characteristics of high-performance computing (HPC).
In artificial intelligence (AI), computers are assembled into neural networks that emulate the learning process of the human brain to solve new challenges. It’s a process that requires lots of computing horsepower, which is why the leading players in the field have moved beyond traditional CPU-driven servers.
A key question: What elements of an AI-focused data center are prompting Meta to rescope its construction? One possibility is the need to incorporate systems to deliver water to the rack. The current design was developed by Facebook in 2011, and features racks sitting on a slab floor, with cool air being dropped into the data hall from a second story "cooling penthouse."
In October Meta announced a roadmap for a gradual shift to a water-cooled AI infrastructure, using cold plates to provide direct-to-chip cooling for AI workloads on its GPU servers. This would initially be accomplished using a rear-door heat exchanger and air-assisted liquid cooling, followed by a shift to a “facility water” strategy as thermal loads exceed the limits of the rear-door heat exchanger. That next phase will likely require the addition of piping to bring chilled water to the rack.
Here's what we wrote about this strategy at the time:
Meta’s strategy allows it to add higher-density workloads within its current data centers, while working out the details of a next-generation design to transition to facility water supplies and the additional infrastructure that will require.
Earlier this year Meta revealed a new facility to house its Research SuperCluster (RSC), which may become the fastest AI system in the world when it is completed. Much of the GPU-powered infrastructure in that system is air-cooled, but the facility’s InfiniBand network uses a liquid-to-liquid cooling distribution unit.
Photos of the RSC show that racks are deployed on a raised floor, a departure from the traditional Meta slab design. Many data center shells can be adapted to use air cooling in either a slab or raised floor approach, but adding water could involve under-floor piping and connections to exterior water infrastructure.
The RSC design used largely off-the-shelf components, as noted by The Next Platform, leaving open the possibility that a next iteration will feature the kind of custom hardware and design common to Meta and OCP.
In brief public comments thus far, Meta has not shared any timetable on its rescoped design and what it means for its construction schedule. We will keep our readers updated as more details become available.
