Pandemic Prompts New Urgency for AI, Software Automation in Data Centers

Today we begin a series of stories examining innovation amid the pandemic. Perhaps the most prominent development will be increased adoption of software-powered automation and artificial intelligence to remotely manage data centers.

Data centers have always been highly-automated environments. The growing use of environmental sensors in racks and servers has enabled companies to use software to track and control many aspects of data centers. This movement has expended beyond the initial wave of data center infrastructure management (DCIM) software to embrace broader platforms that integrate automation and AI.

“If ever there was a time for data center owner-operators and their customers to embrace automation and remote management and monitoring of their infrastructure, that time is now,” said Phillip Marangella of EdgeConneX.

Protecting Essential Operations and Workers

Since the COVID-19 outbreak hit the US in late February, major data center operators have been limiting access to their facilities, which house servers and network equipment that power many of the world’s leading social media, video platforms, and cloud computing services. Data centers also play a key behind-the-scenes role, providing mission-critical technology to support emergency communication for public agencies, emergency services, hospitals and 911 systems.

A Coronavirus outbreak inside a data center could, over time, create serious challenges in maintaining customer uptime. The first line of response has been access control, with data centers allowing some employees to work remotely and limit site visits by vendors.

In March, Equinix said the growing outbreaks in some countries required it to lock down facilities, with only Equinix staff allowed inside and providing “remote hands” support for customers who can no longer access their equipment.

Software has provided the next level of pandemic-driven enablement. Access restrictions have prompted many customers to take advantage of management software offerings from their data center provider, many of which offer remote management options.

“Keeping our critical data centers running while ensuring employee, partner and customer safety is a front-and-center concern across the industry,” noted Alexis Romero of ServerFarm, which offers a cloud-based management portal called InCommand. “If the current crisis had hit even five years ago, enterprise IT could be facing serious downtime issues.  Automated data center management has contributed massively to the smooth running of IT at a time of huge disruption.”

Staffers at NTT Global Data Centers demonstrate temperature sensors the company is using to screen visitors to its data centers. (Image; NTT)

“We are leveraging our data center infrastructure management capabilities to extend remote monitoring and management capabilities to enable our customers, partners and team members to work remotely in a fully productive way,” said Erich Sanchack, Executive Vice President of Operations for Digital Realty.

QTS Data Centers says it has seen a big increase in usage of its Service Delivery Platform (SDP), a software orchestration platform that allows customers to remotely manage data center deployments. Logins to SDP jumped 30 percent in late March and early April, with users spending nearly twice the amount of time on the platform – 62 minutes compared to 36 minutes.

“QTS customers are benefiting from greater control over their IT environments and in the current environment, decreasing risk at the same time through a digitized orchestration capability that largely has not existed in the data center industry previously,” said Jon Greaves, Chief Technology Officer of QTS Data Centers.

Regional data center provider vXchange reported a 60 percent increase in use of its Insight Business Intelligence platform during the pandemic. “As people are not coming to the sites, they are able to leverage this platform to provide their view into their mission-critical deployments,” said vXchange CEO Keith Olsen, who said the platform is available on any device.

Commissioning and Certification

Elements of deploying new data center capacity have also adjusted to the pandemic by shifting some of the work to remote management. Compass Datacenters recently used a virtual commissioning process to deliver a new data center in Raleigh, using live-streaming cameras and wireless sensors to manage the process remotely, with on-site tasks completed by a small group of staff separated by safe distances.

“It is conceivable that this commissioning model could become prevalent for data center facilities even after the COVID pandemic subsides,” said Compass Chief Innovation Officer Nancy Novak.

Meanwhile, the Uptime Institute has developed remote versions of several of its Tier Standard Certification. Uptime said it has revised the protocols in place to deliver Certification of Design, Construction and Operations of data center facilities. These revisions eliminate a significant portion of the on-site work previously required.

The Next Level: Lights-Out Operations

The COVID-19 pandemic has become an extended event, with no clear end date and no timeline for a vaccine or treatment. The near-term future will place a premium on contactless, socially distant operations. Software and sensors can provide additional options to automate operations and refine how staff are used.

“Complete automation of the data center means faster response to issues not requiring human interventions,” EdgeConneX’s Marangella wrote recently. “Monitoring and automation also mean that a data center can run with minimal or no staffing required during emergencies or crises like the one we find ourselves in now. For many years, there have been discussions of lights-out data centers, where everything is fully automated, and humans never have to enter the premises. Fortunately, we’ve now moved beyond mere talk into actualizing this capability.”

EdgeConneX has built a series of regional facilities that can operate unmanned, using its EdgeOS data center operating system to track more than 500,000 sensors used in the company’s sites, monitored every 15 milliseconds. Facilities are managed from a central network operations center in Santa Clara, with remote hands support when on-site maintenance is required. The company has the ability to remotely control the generators and UPS systems at each data center.

Automation is a particular focus for pioneers in edge computing, a model which benefits from the ability to run unmanned or lightly-staffed facilities.

Vapor IO’s Kinetic Edge uses software and high-speed connectivity to bring multiple micro data center facilities into a single virtual facility with multiple availability zones. Modules are meshed together with high-speed fiber across a group of data centers. The design builds upon the “availability zone” approach popularized by Amazon Web Services, creating a cluster of local data centers to easily hand off data and traffic.

Even as states and regions “reopen” from their COVID-19 lockdowns, the industry will likely look for new ways to use remote management to handle more data center management tasks. The ability to fully automate data center operations also provides a backstop in scenarios where facility staff fall ill or are unable to be on-site.

Artificial Intelligence

Artificial intelligence has emerged as a key tool in facility automation, and holds tremendous promise for automating infrastructure so that it requires fewer humans to manage.

At Google, AI has been used to improve energy efficiency by fine-tuning its data center cooling systems to respond to changes in weather or workloads. ROOT Data Center (now part of Compass) has used artificial intelligence (AI) to monitor emergency backup generators, analyzing sound and video of the equipment at a granular level and sending alerts when a change is detected.

Creating AI agents to perform employees’ job functions makes that worker extendable, but also replaceable. Researchers and sociologists are grappling with the workforce implications of these technologies, and their impact on employment and the job market.

“Digital labor will be a very disruptive thing,” said data center veteran Dean Nelson, who is a strategic advisor to IPSoft, which develops cognitive AI to develop digital agents that can automate IT helpdesk operations, integrating natural language conversations and process automation. IPSoft offers a catalog of downloadable skills for the call center and telecom industries.

Nelson sees this as an enabler of hybrid workforces that more deeply integrate humans with technology automation – a capability that may prove attractive in being social distancing into workplaces.

The Road Ahead

Many of these concepts featured in a presentation last year about the future of mission-critical design and operations by researcher Suvojit Ghosh (The Data Center of 20330 will be Faster, Cooler and Automated).

Suvojit Ghosh, Managing Director of the Computing Infrastructure Research Center in Toronto, discusses the future of data center design at DCD Enterprise New York. (Photo: Rich Miller)

“In the next 10 years, we will see a situation where orchestration allows data centers to be operated without a maintenance person,” Ghosh predicted. “From a single pane of glass, you should be able to maintain 1,200 to 2,000 data centers.”

Will the COVID-19 pandemic act as a disruptive accelerant of some of these changes?

“The biggest challenge of the current crisis is managing uncertainty – which is one part of crisis, but another part opportunity,” said Ken Forster, Executive Director of Momenta Partners. “When events force rapid changes in behavior, people are more amenable to adopt new innovations, and this provides a unique opening for creative entrepreneurs to break through the inertias that characterize status quo.

“Looking back on prior eras of recession, conflict and natural disasters, each period gives rise to new and better ways of doing things – providing fertile ground for enterprising new ventures to gain a solid foothold in their journey to success.”