Mitigate Risk, Improve Performance and Decrease Operating Expenses through Data Center Self-Performance
In the ad, masked robbers smash their way into a bank with baseball bats. An alarm sounds. Everyone drops to the ground in fear except for a stationary security officer. One of the customers looks up at the uniformed officer asking, “Why don’t you do something?” The officer calmly replies, “Oh, I’m not a security guard, I’m a security monitor. I’m only here to inform people if there is a robbery.” The officer looks around the room, glancing across the dramatic scene and announces, “There’s a robbery.” Cut to confused and bewildered faces of the customers.
If it’s broke, why not fix it?
This advertisement perfectly illustrates a data center industry standard practice that is as troubling as it is common. Most data center and critical infrastructure facility management providers utilize multiple vendors and subcontractors to perform daily and ongoing facility management, base IT services and periodic equipment maintenance. This establishes an outside ‘caretaker’ or monitor role as the dominant industry standard; in fact, a reliance on outside or third-party suppliers to ‘turn the wrench’ or provide real ‘boots on ground’ work is only increasing.
With this operating model in mind, imagine a data center version of the LifeLock ad:
A large data hall with hundreds of servers is operating with a perpetual hum. Suddenly, power and lights go out, plunging the room into total darkness and eerie silence. In the resulting chaos, people hurry through the darkness, a single worried voice rising above the clamor to ask, “What is going on?” The speaker flips an unresponsive light switch on and off again and directs the next question to the data center operator: “What are you going to do?” The data center operator responds, “Oh, I only monitor your data center. I see the power has gone off – I’ll notify our vendor and get back to you.”
A scenario like this, while dramatic, illustrates the negative implications of this standard practice: operational risks associated with long business continuity and vendor response times; higher operating costs due to unnecessary vendor utilizations, not to mention mark-up costs on their services; and, simply, a lack of true accountability on the part of the provider.
Too often mission critical infrastructure is managed by providers whose business and operating model is one of facility caretaker or monitor, only notifying customers and owners of problems and bringing in vendors and subcontractors when real work needs to be done.
There are more progressive data center operations providers who perform more than 80% of data center operations services themselves. More traditional providers self-perform less than half of services.
Why it matters
Today’s data center is an increasingly complex and sophisticated environment comprised of interconnected systems. These mission critical facilities house the applications and workloads on which American business and our digital world depends. There are a growing number of factors that make self-performance an increasingly critical aspect of facility operations and management.
According to the 2021 Uptime Institute Global Data Center Survey, roughly half of all data center outages cause significant revenue, time and reputational damage, and 69% of all data center owners or operators report a data center outage in the past three years. The survey attributes a majority of outages to onsite issues: 43% power failures, 14% cooling failures, 14% network issues, 14% software / IT systems errors. Together, these outages reinforce the criticality of an ability to identify, triage and respond to onsite incidents in an effective and timely fashion. While Uptime Institute doesn’t calculate average cost of outages due to the wide varying degrees of impact, more than 6 out of every 10 outages cost the owner or operator over $100,000.
However, outage costs can be significantly mitigated through self-performance of data center operations – and that’s just the beginning. For example, preventative and corrective maintenance of equipment that supports mechanical, electrical and plumbing (MEP) critical infrastructure – including cooling, primary and backup power, life safety systems and monitoring controls – can eliminate outages before they happen. The ability of your onsite data center operations team to handle monthly, semi-annual and some annual maintenance workloads can result in a decrease of overall operating costs by over 30%, which for many single-site data centers provides hundreds of thousands of dollars in annual savings.
A simple test
Most, if not all, data center owners and operators suggest they conduct some level of self-performance, however, as with everything in life (and business), the devil is in the details. Here are three questions tied to base data center operations that may allow you to gauge where you or your data center is on a self-performance scale:
- Who replaces back-up power or UPS batteries at your facility?
- Who repairs the motor on a chiller?
- Who performs quarterly or semi-annual preventative maintenance on the electrical, HVAC or plumbing systems?
If the answer to one or more of these questions is “a vendor conducts the actual work”, then you or your operator aren’t maximizing your current operating resources and are experiencing incremental cost and risk.
What’s the right approach?
Rethink preventative and most corrective equipment maintenance. Original Equipment Manufacturers (OEMs) complement any good mission-critical program and add value for the proprietary knowledge, firmware updates and annual required preventative maintenance. However, OEMs are not invested in the day-to-day success or long-term performance of your facility. You can (and should) expect your data center operator to have the skills and expertise to conduct monthly and semi-annual preventative maintenance, plus a large portion of corrective maintenance work.
Hire stewards not caretakers. Consider a mindset and paradigm shift in your data center operational model. Place onsite subject matter experts (SMEs) as first responders that self-perform the majority of daily operations. These services need to be provided by a single partner provider using their own trained and certified employees, eliminating a high percentage of third-party vendors and outside service providers.
These actions – and many others associated with a high degree of self-performance on the part of your data center operator(s) – will mitigate risk, improve operational performance, and substantially decrease operational expenses.
Chad Giddings is the Executive Vice President, Chief Marketing & Sales Officer for BCS Data Center Operations. He has more than 30 years of marketing, communications and strategic planning experience growing business-to-business and direct-to-consumer brands, marketing agencies and Fortune 500 companies. Contact BCS Data Center Operations to learn more about their integrated self-performance model.
