AI Hardware is Disrupting the Data Center. Here’s Who to Watch

The market for data center hardware is undergoing major disruption. NVIDIA’s plan to acquire ARM is the latest in a series of shifting tides, with new processors and servers providing many options for users yearning for more power to tame artificial intelligence (AI) workloads. That includes new offerings from resurgent players, as well as a herd of startup offering specialized chips for AI computing.

The rise of specialized computing has the potential to bring change in the data center industry, which must adapt to new form factors and higher rack densities. But for all the excitement in the chip and server sector, the racks and rows of most data halls continue to be populated by Intel chips, particularly in the enterprise sector. Is this about to change? And if so, when?

Those were the big questions on the table at the AI Hardware Summit, a virtual event from Kisaco Research that offered updates from and insights from leading lights in high-performance hardware.

The bottom line: There will be powerful new chips arriving from several startups in coming months. It’s the arrival of what analyst Karl Freund of Moor Insights calls a “Cambrian Explosion” of new chips optimized for AI data-crunching.

“We’re going to see a lot of new competitors later this year,” said Freund. “I know I said that last year. Now we’re starting to see chips come out, and it should be pretty exciting. These chips take longer to develop than anyone wants. But there’s no way faster chips can come close to keeping up with the growth in models.”

The Rise of Domain-Specific Architectures

The development of AI algorithms is accelerating, said Freund, with new models incorporating billions of data points to make recommendations and decisions. As models incorporate more data, they also require more computing horsepower, which in turn is driving an AI hardware arms race.  The competition to leverage AI is led by the industry’s marquee names – including Google, Facebook, Amazon and Microsoft – who are seeking to add intelligence to a wide range of services and applications.

The future of AI hardware will be specialized, according to David Patterson, a Distinguished Engineer at Google.

“Moore’s law is slowing down,” said Patterson, one of the keynote speakers at the summit. “There’s people that want to argue about it, but it’s a fact that you can measure. As a result, general-purpose microprocessor performance is improving slowly.

“The domain specific architecture is the future of computing,” said Patterson, a pioneer in reduced instruction set computing (RISC) and vice-chair of the RISC-V initiative. “You can tailor your work to that domain and ignore other domains.”

That realization led to big changes in how Google operates its massive global computing platform. In 2015 Patterson led Google’s development of Tensor Processing Units (TPUs), a specialized chip architecture that dramatically boosted Google’s processing power.

The TPU is a custom ASIC tailored for TensorFlow, an open source software library for machine learning that was developed by Google. An ASIC (Application Specific Integrated Circuit) is a chip that can be customized to perform a specific task. Adopting a domain-specific approach allowed Google to drop general purpose features to save on space and energy in the processor.