ASHBURN, Va. – The Open Compute Project set out to start a revolution in data center hardware. Hatched inside Facebook’s highly-customized data centers, the open source project’s lofty ambition is to democratize data center infrastructure, making cutting-edge designs available to companies that can’t afford their own design team.
For the movement to grow beyond its roots in the hyperscale world, it required a bridge – a customer that could adapt the pioneering hardware designs to commercial data centers, making them accessible to a large pool of customers.
For Open Compute, that path to a larger market ran through a Rackspace data center in northern Virginia.
It is here, on a huge cloud campus operated by DuPont Fabros Technology, that the Open Compute Project (OCP) made the transition to “plug-and-play” data halls – first in space leased by Facebook, and then in a Rackspace facility that houses part of its managed cloud.
Several halls of the Rackspace data center are filled with servers using Open Compute designs, which power the company’s flagship Cloud Servers offering, as well as its onMetal service for single-customer “bare metal” clouds. The huge data hall is filled with rows of Open Rack enclosures, including extra-tall 56U racks that help Rackspace get more cloud computing capacity out of each rack. About 60 “Rackers” work at the facility, maintaining IT gear for the company’s managed hosting and cloud customers.
Bringing OCP to the Wholesale Market
In Rackspace, the open hardware movement found a champion who could bring scale to the emerging ecosystem, prompting vendors to develop Open Compute designs that will work in wholesale data centers from DuPont Fabros and Digital Realty – and not just the custom facilities built by Facebook.
Rackspace is a long-time supporter of open source projects, but its motivation is driven by the bottom line. The managed cloud company spent $324.9 million on customer hardware in 2015, and says the shift to Open Compute designs has helped it get better bang for the buck on purchases of servers and storage.[clickToTweet tweet=”Aaron Sullivan of Rackspace: Open Compute continues to be a considerably lower cost option.” quote=”Aaron Sullivan of Rackspace: Open Compute continues to be a considerably lower cost option.”]
“Open Compute continues to be a considerably lower cost option” said Aaron Sullivan, Distinguished Engineer at Rackspace Hosting. “We really have a lot more control over elements of the server. We can pick components (SSDs, RAM and CPUs). The opportunity to get customized OCP server packages is much better than what we’ve traditionally had. These things make a pretty big difference.”
The front of Open Compute servers in a Rackspace data center. The managed cloud company was a key early adopter of OCP hardware designs. (Photo: Rackspace)
After years of working with OEM server vendors HP and Dell, Rackspace began collaborating with firms specializing in Open Compute designs, including Quanta Cloud Technology, WiWynn (Wistron), Delta Group, and Cloudline, a joint venture between HP Enterprise and FoxConn. These companies created products for Rackspace to implement OCP in its data center, but also sell to other companies for use in standard data halls.
“We were the anchor customer driving this, but we wanted them to develop products that others could use as well,” said Sullivan.
The Roots of the Revolution
The growth of the open hardware movement will be showcased at next week’s Open Compute Summit in San Jose, Calif. The event will mark the fifth anniversary of the project’s launch, and will feature more than 60 sponsors, including many of the biggest names in the data center business: Facebook, Microsoft, Intel, Hewlett Packard Enterprise, Panasonic, Sony, Schneider Electric and Emerson Network Power.
The growing reach of open hardware will be on display in a keynote session featuring adoption in the telecom industry, with panelists from AT&T, Verizon and Deutsche Telekom.
The corporate embrace of open hardware seemed unlikely in 2011, when the OCP was launched to create a development community to offer Facebook’s designs to other companies. The movement started with servers and data center, but has since expanded to storage, network switches and telecom gear.
A key question was whether the OCP designs could make their way into conventional data centers. This was not a trivial issue. Facebook’s design includes features not seen in multi-tenant facilities, especially a streamlined power distribution with a decentralized UPS design.
Infrastructure at Rackspace
In 2012, Rackspace announced plans to adopt Open Compute hardware to power its fast-growing cloud computing operation. The move was consistent with the open ethos at Rackspace, which was a driving force in the creation of the OpenStack cloud platform, and is also active in IBM’s OpenPOWER server initiative.
Rackspace deploys infrastructure using a wholesale model, leasing its data center space from providers like DuPont Fabros Technology and Digital Realty. The company has 13 data centers around the world, with 54 megawatts of power capacity. At the start of this year, it was using 32.2 megawatts of electricity to power 118,177 servers in its data centers.
A Rackspace employee (Racker) working on a server in one of the company’s data centers. (Photo: Rackspace)
Rackspace has seen spectacular growth in its infrastructure since it was founded in 1998 by three classmates at Trinity University. The young company’s philosophy of offering “fanatical support” to customers soon made it one of the leading players in managed hosting. After riding out the dot-com bust, Rackspace created a division called Mosso to focus on white label hosting. Mosso soon shifted focus to cloud computing, an emerging hosting model that would guide the future for Rackspace.
After its IPO in 2008, Rackspace acquired several cloud companies (SliceHost and JungleDisk) and launched the Rackspace Cloud to compete in the fast-growing public cloud sector. By 2009, the company was managing 50,000 servers in its data centers, and would add 45,000 more over the next four years.
Cloudy Competition
As Amazon Web Services grew into a public cloud juggernaut, Rackspace boosted its focus on hybrid cloud, a strategy that aligned with its mix of managed hosting and cloud customers. Last year Rackspace introduced managed cloud deployments on Amazon, leveraging its Fanatical Support to support the appetite for capacity AWS.
As its business has evolved, Rackspace’s rate of infrastructure growth has moderated slightly, but the company remains a major purchaser of IT gear. The company’s server count grew by 5,500 in 2015. When factoring in the decommissioning of 2,400 older servers in the fourth quarter, Rackspace added servers at a rate of about 650 per month.
Some of the early-generation high-density cloud servers in a Rackspace cloud data center. (Photo: Rackspace)
The company’s transition to Open Compute accelerated in early 2013, when Rackspace chief operating officer Mark Roegnik said the company would soon begin deploying Open Compute hardware in the company’s data center in northern Virginia.
Design flaxibility was important. Rackspace continues to operate a significant managed hosting business, so the company’s data centers must support a diverse array of customer hardware in addition to its cloud infrastructure. It helped that the transition would take place in a facility operated by DuPont Fabros, which was also working to implement Open Compute designs in nearby Ashburn data halls leased by Facebook.
Rackspace has always had an active in-house R&D team, which began working with other OCP members on modifications. “There were challenges to taking the original design and putting them into our data center,” said Sullivan. “We got really geared up for this discussion about how to design and what had to change.”
The Power of Adaptation
One of those challenges was modifying the OCP designs to work with standard electrical power distribution schemes. The electrical cabling, which is typically routed below a raised-floor, was shifted to overhead trays. Running power overhead gives Rackspace more flexibility in where it places equipment in its data center. Overhead bus bars can provide redundant A and B power connections for each server.
Most data centers use a centralized UPS (uninterruptible power supply) to support an entire data hall, while Facebook’s Open Compute design feature a distributed system featuring in-row UPS units for every three racks. Each OCP rack features a “power shelf” housing the power supplies for all the servers in the rack, rather than having a power supply on each server.
A view of the overhead cabling trays in a Rackspace data center. (Photo: Rackspace)
“We worked on modifications to the power shelf with companies like Delta, Quanta and Wistron,” said Sullivan. “This allows us to put Open Compute equipment into our data center that’s compatible with traditional designs. It still has a central UPS. We don’t run a distributed battery system.”
Quanta’s design work adapting OCP gear for the northern Virginia data halls operated by Facebook and Rackspace resulted in a commercial product, the Quanta RackGo, a 21-inch solution inspired by the OCP Open Rack design, which offers centralized power supplies and more room for improved thermal management and cabling.
The Road Ahead
One modification was guided by maintenance issues.
“We reworked the three-node (Winterfell) form factor so the fans could be hot-swappable,” said Sullivan. “We never want to take a server down because a fan died.” Rackspace also made some servieability-related design changes to the OCP storage chassis, known as Knox, to relocate circuit boards and improve the cable management.
After developing the design during 2013, the OCP Cloud Servers and onMetal servers began to go live in 2014. “We spent most of the rest of that year working through refinements,” said Sullivan. “At this point, 100 percent of these two applications are deployed on Open Compute.”[clickToTweet tweet=”Rackspace has monthly revenue of $1,472 per server, up $60 per server from the close of 2014.” quote=”Rackspace has monthly revenue of $1,472 per server, up $60 per server from the close of 2014.”]
The ability to save money on the procurement and operation of the OCP servers goes to the bottom line. Rackspace now sees monthly revenue of $1,472 per server, up $60 per server from the close of 2014.
As it continues to add servers, Rackspace sees the benefits of having vndors from the Open Compute ecosystem who are used to deploying gear at hyperscale.
“We are transitioning to an environment where we increasingly add capacity by rolling in racks of servers,” said Sullivan. ” There’s less and less of people installing servers into racks by hand.”
“There were parts of the (Open Compute) system that initially took more time to implement,” said Sullivan. “At this point, it’s turned into business as usual.”
