Data center staff, installing servers around the clock and across the world. All while staying six feet apart.
That’s how Microsoft boosted the capacity for its Microsoft Azure cloud computing service as demand surged during the COVID-19 pandemic, which the company detailed this week in a blog post from the Azure team.
“The scope and the scale of the response to COVID-19 was completely unprecedented, in terms of how much of the world went digital inside a month,” said Mark Simms, a Microsoft software architect who helped manage the COVID-19 response across Azure. “So the work that we had to do to get through the initial surge in demand and free up capacity for our customers to run critical health and safety workloads was also unprecedented.”
While scalability relies on hardware, Microsoft’s account of its network expansion highlights the human component of the company’s pandemic response, as Azure staff worked to scale its cloud operations and help customers on the front lines of the pandemic response address their most urgent needs.
Needed: Bigger Pipes Across the Atlantic
As one of the largest technology companies, Microsoft operates more than 60 data center regions around the globe, and keeps computing hardware stockpiled in warehouses to support rapid deployments.
In March the company began adding new servers to the hardest hit regions and installing new hardware racks 24 hours a day. To protect the health of critical data center employees, Microsoft also quickly established social distancing requirements, provided protective equipment and implemented strict disinfectant policies.
The dramatic increase in data traffic required larger pipes between continents. Microsoft says it doubled capacity on one of its own undersea cables carrying data across the Atlantic and negotiated with owners of another to open up additional capacity. Network engineers installed new hardware and tripled the deployed capacity on the America Europe Connect cable in just two weeks, according to Microsoft.
Microsoft prioritized capacity for existing customers while also reserving capacity for first responders who needed to quickly scale life and safety services.
Shifting Capacity to Pandemic Priorities
Use of Microsoft’s cloud services soared amid the COVID-19 pandemic, especially in regions implementing social distancing and shelter-in-place orders. The traffic surge has been seen across a number of services, but nowhere more than Teams, Microsoft’s video conferencing tool.
At the end of March, Microsoft Teams set a new daily record of 2.7 billion meeting minutes in one day, up from 900 million minutes just two weeks earlier. In April, that number climbed to 4.1 billion meeting minutes in a single day. To handle the Teams growth, Microsoft’s Azure Wide Area Network team added 110 terabits of capacity in just two months to the fiber optic network that carries Microsoft’s own data around the globe. It added 12 new edge sites that connect the network to infrastructure owned by local internet providers.
The COVID-19 stay-at-home orders also drove increased use of Microsoft’s Xbox gaming platform, including a 50% increase in multiplayer gameplay and a 30% increase in peak concurrent usage. As they sought to prioritize capacity for COVID-19 related workloads, the Xbox and Azure teams moved gaming workloads out of high demand data centers in the United Kingdom and Asia.
“There’s no question that in those regions the people who were on the front lines of the COVID-19 efforts really needed that capacity more than us,” said Casey Jacobs, who manages reliability for Xbox operations. “And our telemetry gave us confidence that we could make these tradeoffs while protecting our customer experience.”
It was all part of a unified, company-wide mobilization to respond to the pandemic.
“We made some pretty profound changes in order to do the right thing, and we did them under a very short time frame,” Simms said.
