AI Driving Data Centers to the Edge

In today’s rapidly advancing technology landscape, data centers are the cornerstone of our digitally-interconnected world. At the same time, the quick growth and application of artificial intelligence (AI) and Machine Learning (ML) is profoundly shaping the design and operation of data centers. Predictive algorithms are impacting industry sectors ranging from healthcare and finance to manufacturing, transportation, and entertainment. Once in production, AI models will be placed at the edge to reduce latency. 

AI workloads are estimated to represent 15% to 20% of total data center energy by 2028, based on internal Schneider Electric growth projections. These AI training workloads are expected to consistently operate at very high densities, which causes challenges for power, cooling, racks, and software management design in data centers. 

The distinction between training and inference is critical when designing AI systems. Training workloads are used to train AI models like large language models (LLMs). These models require massive amounts of data to be fed to specialized IT Clusters (concentrated high-performance compute) and network infrastructure – allowing them to operate in unison. Inference models could be deployed on edge devices or cloud servers, depending on the application's needs, and take the previously trained AI model into production to predict the output of new queries (inputs). The demand for high-performance computing has led to increased requirements for powerful servers with GPUs (Graphics Processing Units) that increase the IT Rack density up towards 100+ kw/rack.  This has changed the technology and architecture of power, cooling and network infrastructure in order to support this higher density requirement.  

Increased AI and ML usage is driving record demand for data centers, but power constraints and old infrastructure grids are inhibiting growth as the data processing requirements are heavily reliant on power-intensive technologies. The data center industry will need to carefully plan and manage environmental and regulatory concerns as companies continue to invest in renewable and energy-efficient fuel sources to mitigate power constraints to support market growth. 

The energy consumption of a generative AI query can vary greatly depending on the size of the inference model, the specific hardware it is running on, and where it is located.  

The Rapid Growth in Data Traffic 

It’s no secret that digital demand will continue to grow. We aren’t going back to beepers and pay phones. From the increasing demand for digital services, cloud computing, and data storage, (and don’t forget the advancements in smart end devices), all of this contributes to the rapid growth in data traffic, especially with AI. This, in turn, drives the need for more data centers, network, aggregation points and investments in digital infrastructure. This also means a greater demand for electricity to power this growth. We can see the future will be digital and with AI-driven automation in many verticals and use cases. Facility owners and operators from data centers, manufacturing, buildings will use this technology to automate routine tasks, reduce manual workloads and enhance overall efficiency in operations, risk, and energy.   

AI and Data Center Evolution 

The demand for AI and the evolution of the data center are interconnected elements shaping the digital landscape. Increased workloads, especially high-performance compute models, require significant computing resources to execute. This requires data centers that can support the performance requirements of AI workloads. As AI technology advances, it will continue to influence the design and operation of data centers. While these advancements bring efficiency and innovation, they also pose challenges related to energy consumption, monitoring, and control, and will require advancements in power and cooling systems. More sustainable approaches to cooling (moving to liquid cooling) and heat capture systems, as well as renewable and green sources of energy becoming available at scale, will help balance this demand vs capacity. 

This relentless advancement of AI is only going to continue, and to meet these evolving needs, the data center industry needs to adapt. Schneider Electric offers guidance on best practices for embracing scalable and flexible infrastructure design to support intensive AI workloads. Strategies go from grid to plug with energy management and controls systems (back of meter power plants & micro grids); deploying high-efficiency and high-capacity power systems; to chiller to chip with mechanical cooling/liquid cooling systems and building automation systems, as well as full suite of sustainability services and global software platforms.