AI’s Impact on Data Center Networking

Last week we launched our article series on future-proofing data center networking in the era of AI. This week, we’ll examine AI’s impact on data center networking and key pain points as the industry shifts to adapt to AI workloads.

The rise of AI has fundamentally transformed the landscape of data center networks, introducing so- called AI backend networks with demands that far exceed those of traditional IT workloads. Unlike conventional applications, which primarily generate north-south traffic between clients and servers, AI workloads, especially those involving large language model (LLM) training and real-time inference, produce massive east-west traffic between GPU clusters within the data center. This shift requires networks to deliver not only vastly greater bandwidth but also ultra-low latency and dynamic scalability to keep pace with the parallel, distributed nature of AI computation.

AI-driven tasks such as LLM training involve splitting data and computation across dozens or even hundreds of GPUs, each of which must synchronize frequently to complete a single iteration. As a result, the backend network connecting these GPUs must support bandwidths of 800 Gbps to 1.6 Tbps per link, with sub-microsecond latency to prevent costly idle time and ensure efficient model convergence. Inference workloads, particularly those serving real- time applications, demand similarly high throughput and near-instantaneous response, further straining traditional network architectures. Research indicates that the explosive growth of generative AI and multi- modal models will accelerate network traffic inside data centers, with AI workloads projected to account for nearly 30% of all data center traffic by the end of 2025.

Meeting these requirements has driven a wave of innovation in data center networking. Operators are rapidly upgrading to high-capacity switches, routers, and fiber optic cabling, while adopting new protocols and technologies such as 1.6T Ethernet, co-packaged optics, and advanced network automation. The unpredictability of AI traffic patterns, driven by dynamic scaling, bursty training jobs, and edge inference, necessitates adaptive, software-defined networks capable of real-time optimization and congestion management. Additionally, the need for lossless transmission and network resilience is paramount, as even brief interruptions or latency spikes can derail distributed AI training.

Ultimately, the impact of AI on data center networking is both transformative and ongoing. As AI models and datasets continue to grow in size and complexity, data centers must evolve into highly intelligent, flexible, and robust environments where network performance is a critical differentiator. This transformation is reshaping every layer of the data center, from physical cabling and cooling systems to protocol stacks and automation frameworks, ensuring that the infrastructure can sustain the relentless pace and scale of AI innovation.

Key Pain Points as the Industry Shifts

As AI workloads continue to expand, data centers face mounting challenges in managing the rapid growth of data volume and network traffic. The backend networks supporting AI training and inference require significantly higher bandwidth, often four to eight times that of traditional cloud environments. This translates into a massive increase in fiber cabling and connectivity complexity. This surge not only stresses physical infrastructure but also amplifies installation timelines and labor demands, creating bottlenecks in deployment and scaling efforts.