Project Brainwave: Intel FPGAs Accelerate Microsoft’s AI Hardware

The artificial intelligence arms race continues, as the largest tech companies explore new ways accelerate AI workloads for cloud platforms. The appetite for more computing horsepower is following several tracks, with major investment in graphics processors (GPUs) as well as custom ASIC chips.

Microsoft has been a leader in using FPGAs (Field Programmable Gate Arrays) to accelerate its cloud and AI workloads. This week Microsoft unveiled Project Brainwave, a deep learning acceleration platform based on its collaboration with Intel on FPGA computing.

Microsoft says Project Brainwave represents a “major leap forward” in cloud-based deep learning performance, and intends to bring the technology to its Windows Azure cloud computing platform.

“We designed the system for real-time AI, which means the system processes requests as fast as it receives them, with ultra-low latency,” writes Doug Burger, a Microsoft Distinguished Engineer, in a blog post. “Real-time AI is becoming increasingly important as cloud infrastructures process live data streams, whether they be search queries, videos, sensor streams, or interactions with users.

Real-Time Deep Learning

Tuesday’s announcement Microsoft at the Hot Chips 2017 conference fleshed out the details on an approach that Microsoft described in broad terms at its Build user event in April.  Microsoft says its new approach, which it calls Hardware Microservices, will allow deep neural networks (DNNs) to run in the cloud without any software required, resulting in large advances in speed and efficiency.

FPGAs are semiconductors that can be reprogrammed to perform specialized computing tasks, allowing users to tailor compute power to specific workloads or applications. FPGAs can serve as coprocessors to accelerate CPU workloads, an approach that is used in supercomputing and HPC. Intel acquired new FPGA technology in its $16 billion acquisition of Altera in 2016.

“We exploit the flexibility of Intel FPGAs to incorporate new innovations rapidly, while offering performance comparable to, or greater than, many ASIC-based deep learning processing units,” said Burger.

Microsoft is using Intel Stratix 10 FPGAs as the hardware accelerator in its Brainwave platform. Microsoft describes its approach as using a “soft” DNN processing unit (or DPU), synthesized onto commercially available FPGAs. Microsoft says this approach provides flexibility and the ability to rapidly implement changes as AI technology advances.