• About Us
  • Partnership Opportunities
  • Privacy Policy

Data Center Frontier

Charting the future of data centers and cloud computing.

  • Cloud
    • Hyperscale
  • Colo
    • Site Selection
    • Interconnection
  • Energy
    • Sustainability
  • Cooling
  • Technology
    • Internet of Things
    • AI & Machine Learning
    • Edge Computing
    • Virtual Reality
    • Autonomous Cars
    • 5G Wireless
    • Satellites
  • Design
    • Servers
    • Storage
    • Network
  • Voices
  • Podcast
  • White Papers
  • Resources
    • COVID-19
    • Events
    • Newsletter
    • Companies
    • Data Center 101
  • Jobs
You are here: Home / Design / Oracle’s Data Center Journey

Oracle’s Data Center Journey

By Rich Miller - February 9, 2016

Oracle’s Data Center Journey

Michael Thrift, Oracle's Director of Data Center Facilities, describes the operations of the company's Utah Compute Facility at the 7x24 Exchange Fall Conference in San Antonio. (Photo: Rich Miller)

LinkedinTwitterFacebookSubscribe
Mail

A trip through the Oracle Utah Compute Facility provides an unusual window into the data center industry’s progress on design. The 200,000 square foot data center in South Jordan, Utah delivers the cloud infrastructure to support Oracle’s on-demand computing business.

The first phase of the facility features a raised-floor data hall, battery-based UPS systems, direct evaporative cooling system, and chillers for backup cooling.

The second phase is a different story. Rather than a single data hall, it’s divided into six modules that will be built in phases. There’s a slab floor and a ductless cooling delivery system, which is supported by a new indirect evaporative cooling system and tight airflow containment for the racks. The power infrastructure also has a different look, as flywheel UPS systems have replaced the batteries.

The evolution of the Utah Compute Facility (UCF) illustrates ongoing changes in technology and best practices. It also reflects Oracle’s emphasis on continuous refinement of data center design to strike a balance between sustainability, cost and operational efficiency.

“One of the ways we work at Oracle is to use each project as a way to improve this growth journey,” said Michael Thrift, the Director of Data Center Facilities at Oracle. “Innovation happens when we look at problems holistically, and eliminate organizational boundaries to find creative solutions.”

An aerial view of Oracle's Utah Compute Facility in South Jordan, Utah. (Image: Oracle Corp.)

An aerial view of Oracle’s Utah Compute Facility in South Jordan, Utah. (Image: Oracle Corp.)

The Utah Compute Facility provides a case study of this process, which features three steps – evalute, pivot and adapt. Thrift and Oracle’s team of design and construction vendors shared the details of that process recently at the 7×24 Exchange Fall Conference in San Antonio.

The Backstory: Austin and the Dawn of Containment

Oracle has been a key contributor to advances in data center design, dating to 2004 when the company created one of the first airflow containment systems while expanding the company’s colcoation facility in Austin, Texas.

The racktop chimney containment system in Oracle's Austin data center, circa 2004. (Photo: Oracle)

The racktop chimney containment system in Oracle’s Austin data center, circa 2004. (Photo: Oracle)

The Oracle team improvised a rack-based containment system that vented hot server exhaust air from the top of the rack into a ceiling plenum and back to the CRAC (computer room air conditioner). Oracle then used variable speed fans to adjust the airflow speed, reducing the power required to cool the system.

“At the time it was all hot aisle/cold aisle,” Oracle’s Mukesh Khattar told me in 2010. “This project debunked a lot of common myths associated with variable airflow in data centers and clearly demonstrated its cost effectiveness.”

As on-demand cloud technologies emerged as business driver, Oracle began planning its next phase of growth with a multi-state search for a site for a $200 million data center project. Utah won out over Idaho based on the cost of energy, local workforce and pro-business climate. The new facility in South Jordan, Utah was dubbed “Project Sequoia” to reflect Oracle’s ambition for a sustainable, energy-efficient “evergreen” data center.

Free Resource from Data Center Frontier White Paper Library

The Future of Future Proofing
Venyu provides IT leaders with a framework for future-proofing their systems, networks, and partner ecosystems. 
We always respect your privacy and we never sell or rent our list to third parties. By downloading this White Paper you are agreeing to our terms of service. You can opt out at any time.

Get this PDF emailed to you.

2011: Phase I

Oracle announced Project Sequoia in 2008, but the project was soon put on hold amid the financial crisis and Oracle’s acquisition of Sun Microsystems, which also had a sizable data center portfolio. Construction on the 25,000 square foot first phase resumed in 2010, with several changes from the Austin design. Chief among these was a shift to direct evaporative cooling, using fresh air instead of chilled water to cool the servers. At the time, direct air “free cooling” was in its early phases of adoption.

Oracle’s design team believed it could run with fresh air up to temperatures of 85 degrees due to Utah’s low humidity. Four 1,000 ton chillers were installed to provide cooling on hot days, consistent with Oracle’s focus on efficiency in energy and cost while taking no chances on uptime.

“We’re not ever going to be on the bleeding edge of technology,” said Thrift. “But we’re going to continue to drive smart, sustainable innovation into the industry. We’re not making this commitment to sustainability because it’s a fad. This sustainability has to pencil out for the bottom line – sustainable sustainability.”

oracle-dobson-thrift

Tom Dobson, a vice president at Holder Construction, outlines some of the innovations at Oracle’s Utah Compute Facility during a November 2015 presentation at the 7×24 Exchange Conference in San Antonio. At right is Michael Thrift, Oracle’s Director of Data Center Facilities (Photo: Rich Miller)

The new cooling system was indeed efficient, helping Oracle reduce its PUE (Power Usage Effectiveness) to 1.3, an improvement on the 1.46 PUE in Austin. But those gains came with overhead.

“The direct evaporative unit is difficult to control,” said Tom Dobson, Vice President of Holder Construction. “All of the changes you need to make on the control side turned out to be pretty tricky. It took us three years to optimize.”

There was good news as well. “It turns out the chiller plant was completely unrequired,” said Dobson.

2013: Phase II

As the first phase filled up, Oracle applied its “evaluate, pivot and adapt” process – and found plenty of opportunities to refine the design.

“I was committed to making things different for phase 2,” said Thrift. “We decided we wanted to put all the ideas on the table.”

Oracle organized a design charrette among its vendors, including Holder Construction and Glumac Mission Critical. The challenge was to build in flexibility as they evolved from the original design. The team decided on a number of changes:

  • The adoption of a “build as you grow” strategy, dividing the 30,000 square foot data hall into six modular phases to avoid stranded capacity. This approach also allows Oracle to adapt the design as it goes, adjusting for changes in server technology.
  • Shifting to an indirect evaporative cooling system, which Thrift noted was was not as efficient as the direct cooling but offered other benefits, including a simpler control system requiring less fine-tuning, and no worries about the adverse effect of outside air. The system works well with Oracle’s supply air temperature (72 degrees) and supports a wide range of humidity.
  • The new cooling system also uses less water, which was an important sustainability priority. “Water is a huge resource issue in the western United States,” said Samuel Graves, Associate Principal, Glumac Mission Critical. “We had to help Oracle figure out how we can still use these great direct and indirect evaporative systems but use them smarter.”
  • The updated designs shifts the air handlers to the roof, and adopting a ductless cooling system featuring multiple “decks” housing plenums to separate and transport hot and cold air. Supply air is dropped into the data hall, and from there is distributed to provide cooling for electrical rooms and telecom space. Racks are housed in a containment system, with a chimney system venting exhaust air from the hot aisle into the upper return plenum.
  • A flywheel UPS system replaces the batteries, conserving space and providing a “greener” solution. A flywheel is a spinning cylinder which generates power from kinetic energy, and continues to spin when grid power is interrupted. Although a flywheel offers a shorter ride-through time (about 20 seconds), it eliminates the need to replace batteries, as well as some challenges with “eco-mode” energy efficiency settings on the UPS system in Phase I.
  • In the power system, Oracle ran medium voltage to building, with each modular pod/block having its own generator.

The end result: A PUE of 1.18, with capacity closely matched to demand and simpler operational controls. Here’s a cut-away overview of the facility and cooling airflow.

Oracle-Utah-diagram

The Road Ahead

The Oracle team says the revisions to Phase II provide improved efficiency and sustainability, and the flexibility to adapt the design in future modules. Holder’s Dobson attributed the gains to Oracle’s process and the collaboration between vendors and service providers.

“This type of innovation doesn’t happen just by buying it through a contract,” said Dobson. “When we encountered questions, we accepted that the answer might be ‘I don’t know.’ But we decided that we were going to take the time, do the engineering and figure it out. As a result, we’ve got a pretty incredible project.”

Thrift said Oracle was pleased with the outcome, and believes the experience positions it for future design refinements in a changing data center landscape.

“These projects demonstrate the ability to learn and stay ahead of the industry.” said Thrift. “The data center of 10 years from now will be different in ways we can’t possibly imagine.”

LinkedinTwitterFacebookSubscribe
Mail

Tagged With: IaaS, Oracle

Newsletters

Stay informed: Get our weekly updates!

Are you a new reader? Follow Data Center Frontier on Twitter or Facebook.

About Rich Miller

I write about the places where the Internet lives, telling the story of data centers and the people who build them. I founded Data Center Knowledge, the data center industry's leading news site. Now I'm exploring the future of cloud computing at Data Center Frontier.

  • Facebook
  • Instagram
  • LinkedIn
  • Pinterest
  • Twitter

Voices of the Industry

Building an Enduring World Begins with Accountability and Defining an Approach for Long-term Success

Building an Enduring World Begins with Accountability and Defining an Approach for Long-term Success TJ Faze, Head of ESG Strategy and Engagement at Vertiv, outlines the company's learnings around its recent ESG efforts.

White Papers

data center services

Data Center Services: A Story of Build Versus Buy

Waffle House opted to implement a disaster recovery solution in QTS’ 385,000 square foot facility in Suwanee, Georgia, replicating its financial, inventory management and supply chain management services to the site. Download the new report that explores how choosing a colocation provider helped Waffle House reduce its risk and provide a higher level of performance.

Get this PDF emailed to you.

We always respect your privacy and we never sell or rent our list to third parties. By downloading this White Paper you are agreeing to our terms of service. You can opt out at any time.

DCF Spotlight

Data center modules on display at the recent Edge Congress conference in Austin, Texas. (Photo: Rich Miller)

Edge Computing is Poised to Remake the Data Center Landscape

Data center leaders are investing in edge computing and edge solutions and actively looking at new ways to deploy edge capacity to support evolving business and user requirements.

An aerial view of major facilities in Data Center Alley in Ashburn, Virginia. (Image: Loudoun County)

Northern Virginia Data Center Market: The Focal Point for Cloud Growth

The Northern Virginia data center market is seeing a surge in supply and an even bigger surge in demand. Data Center Frontier explores trends, stats and future expectations for the No. 1 data center market in the country.

See More Spotlight Features

Newsletters

Get the Latest News from Data Center Frontier

Job Listings

RSS Job Openings | Pkaza Critical Facilities Recruiting

  • MEP Coordinator - Data Center Construction - Ashburn, VA
  • Data Center Facility Engineer - Chantilly, VA
  • Data Center Site Operations VP - Seattle, WA
  • Senior Electrical Engineer - Data Center - Denver, CO
  • Senior Estimator - Data Center Construction - Denver, CO

See More Jobs

Data Center 101

Data Center 101: Mastering the Basics of the Data Center Industry

Data Center 101: Mastering the Basics of the Data Center Industry

Data Center Frontier, in partnership with Open Spectrum, brings our readers a series that provides an introductory guidebook to the ins and outs of the data center and colocation industry. Think power systems, cooling, solutions, data center contracts and more. The Data Center 101 Special Report series is directed to those new to the industry, or those of our readers who need to brush up on the basics.

  • Data Center Power
  • Data Center Cooling
  • Strategies for Data Center Location
  • Data Center Pricing Negotiating
  • Cloud Computing

See More Data center 101 Topics

About Us

Charting the future of data centers and cloud computing. We write about what’s next for the Internet, and the innovations that will take us there. We tell the story of the digital economy through the data center facilities that power cloud computing and the people who build them. Read more ...
  • Facebook
  • LinkedIn
  • Pinterest
  • Twitter

About Our Founder

Data Center Frontier is edited by Rich Miller, the data center industry’s most experienced journalist. For more than 20 years, Rich has profiled the key role played by data centers in the Internet revolution. Meet the DCF team.

TOPICS

  • 5G Wireless
  • Cloud
  • Colo
  • Connected Cars
  • Cooling
  • Cornerstone
  • Coronavirus
  • Design
  • Edge Computing
  • Energy
  • Executive Roundtable
  • Featured
  • Finance
  • Hyperscale
  • Interconnection
  • Internet of Things
  • Machine Learning
  • Network
  • Podcast
  • Servers
  • Site Selection
  • Social Business
  • Special Reports
  • Storage
  • Sustainability
  • Videos
  • Virtual Reality
  • Voices of the Industry
  • Webinar
  • White Paper

Copyright Endeavor Business Media© 2022