At this week's start of the Open Compute Project (OCP) Global Summit 2024, taking place at the San Jose McEnery Convention Center from October 15-17, it is clear that artificial intelligence (AI) and liquid cooling are two of the dominant topics on attendees’ minds.
As far back as 2018 in reporting by DCF Founder Rich Miller on OCP's then newly-launched Advanced Cooling Solutions project, Bill Carter, Chief Technology Officer of the Open Compute Foundation explained:
“OCP envisions a supply chain offering a variety of IT devices (servers, storage, networking etc.) that can work with a variety of liquid-enabled racks from many solutions providers. . . Direct contact, immersion, and other advanced cooling options are within the scope of this project.”
As correspondingly noted by nVent Senior Product Manager Kevin Roof in a recent DCF 'Voices of the Industry' guest article, earlier this year OCP announced the initiative's reformulation as its Advanced Liquid Cooling solutions project to drive collaboration throughout the industry to meet liquid cooling demand in the AI era.
OCP AI Innovation Spotlight
To say the least, it's fair to observe that the OCP Global Summit is traditionally known for its focus on innovative data center hardware and software designs, and that's a trend on steroids in this second year of the industry's AI tsunami, what with numerous partners and customers of global AI technology front-runner NVIDIA on-hand to display and demo the latest NVIDIA accelerated computing platforms, including its long-awaited Blackwell AI servers, AI compute racks, liquid-cooled hardware, DPUs and SuperNICs, network switches and network operating systems.
Additionally showcased at this year's OCP Global Summit will be a new NVIDIA Grace CPU C1 configuration — a single-socket Grace CPU Arm-based server geared especially for hyperscale cloud, high-performance edge, and telco applications — as well as the NVIDIA GH200 Grace Hopper and Grace Blackwell Superchips.
Partners showcasing NVIDIA technology and products at this year's OCP Summit include Aivres, Arm, ASRock Rack, ASUS, Dell Technologies, GIGABYTE, Hewlett Packard Enterprise, Hyve Solutions, Ingrasys, Inventec, Lenovo, Mitac, MSI, Pegatron, QCT, Supermicro, Wistron and Wiwynn.
The volume of press and product announcements from companies not named NVIDIA surrounding this year's OCP Global Summit also reflects the heavy groundswell of AI technology demand in data centers.
For instance, to support the substantial increased power requirements of high-density AI accelerated servers, Advanced Energy will be showcasing its ORv3 5.5kW HPR PSU [Power Supply Unit] and power shelf, optimized for peak efficiency nearing 98%. The company says this PSU provides a higher power factor to handle dynamic loading, easing the burden on data centers' AC power infrastructure for GPU applications.
And in the arena of chipscale interconnect, Credo will showcase its data center AI, compute and CXL capabilities with XConn PCIe and Compute Express Link (CXL) switches at OCP Global Summit 2024, in live demos including newly introduced PCI Express 6.0 devices and CXL interconnect technology.
Specifically at the event, Credo will conduct live demos of Toucan, its newly introduced PCI Express (PCIe) 6.0 retimer, and 1Tb OSFP-XD PCIe6 (16x64Gb) Active Electrical Cable (AEC). In addition, Credo will display its 800G sub-10W OSFP optical modules with Linear Receive Optics (LRO) capability interoperating with 51T switches and standard DSP modules.
Meanwhile in this year's OCP Innovation Village, as referred to in the video at the top of this page, Credo will pair with AMD, GIGABYTE, MemVerge, MSI, Penguin Solutions, Rittal, SMART Modular Technologies, and XConn for live demos of PCIe and CXL interconnect.
Additionally, the solution providers will showcase how rack power/density increases as liquid cooling technology penetrates the data center. Notably, the first live demonstration of a rack-scale shared H100 GPU will consist of an AMD EPYC server connected to an XConn PCIe 5 switch via Credo OSFP-XD PCIe AECs, with the XConn switch further driving two chassis of NVIDIA H100 GPUs.
Also in the sphere of connectivity at this year's OCP Global Summit, Broadcom will showcase a range of technology innovations that power its far-ranging offering of Ethernet, Ethernet NIC Adapters, Co-Packaged Optics (CPO), PCIe switches and retimers, and Sian2 optical networking products. Broadcom’s AI architects and engineers will also deliver key talks and technical panels at the event covering topics across the company's broad AI infrastructure footprint.
Notable data center connectivity advancements on the cutting-edge of AI from Broadcom featured at this year's OCP include:
- Ethernet networking switches including the Broadcom Tomahawk 5 and Jericho3-AI, designed to accelerate AI/ML workloads.
- The Trident4-X11 Ethernet switch, engineered to create the front-end fabric that interfaces with the back-end AI fabric.
- The Tomahawk 5 - Bailly, the company's 51.2 Tbps CPO [co-packaged optics] Ethernet switch developed in partnership with Micas Networks, which combines advanced silicon photonics CPO technology with Broadcom's Tomahawk 5 switch chip, setting a new benchmark for power efficiency and performance in AI infrastructure.
- High-performance, low-power 400G PCIe Gen 5.0 Ethernet adapters, developed as open, standards-based solutions to address connectivity challenges as XPU bandwidth increases and AI data center clusters expand.
- PCIe Gen 5.0 switches, billed as "the open, standards-based fabric of choice for AI connectivity; drawing half the power of alternatives, with industry leading SerDes, telemetry and diagnostics."
Is Direct to Chip Leading Present Liquid Cooling Prospects?
With a search for technology that will enable adequate cooling to allow continuous uptime and to make energy use sustainable and efficient, as exemplified by NVIDIA at this year's OCP Global Summit, companies are looking for partners to deliver suitable technologies for their data centers to address these issues.
As our previous coverage is likely to have shown you, direct-to-chip liquid cooling systems seem to be taking a market leadership role, and hardware manufacturers are looking to integrate such systems into their deliverables in order to be able to support the 50 kW+ rack data center designs that are becoming standard with the demand for support for AI and HPC in the data center.
As featured at this year's OCP Global Summit, Flex and JetCool are two of the leading-edge companies that provide innovative solutions to support the demand for high-density computing. Today they announced a partnership to bring a series of new reference platforms for servers and racks that will directly address those density demands. Flex and JetCool are building rack-level solutions, including a new line of co-designed liquid cooling-ready servers that are compliant with the OCP specification.
Flex: Innovating with Design and Manufacturing
Flex Ltd. is an innovator in design, engineering, and manufacturing services. The company provides end-to-end solutions that combine design, engineering, manufacturing, global supply chain solutions, and logistics into a focused technology and manufacturing powerhouse for a range of industries and end markets, including automotive, healthcare, computers and networking, data storage, industrial and home control, consumer, and portable device and displays.
As Michael Hartung, president and chief commercial officer, Flex, puts it:
“Flex delivers integrated data center IT and power infrastructure solutions that address the growing power and compute demands in the AI era. We are expanding our unique portfolio of advanced manufacturing capabilities, innovative products, and lifecycle services, enabling customers to deploy IT and power infrastructure at scale and drive AI data center expansion.”
At the OCP Summit, Flex is highlighting a number of technologies and solutions, including:
- Reference platforms for liquid-cooled servers based on the company's new partnership with JetCool Technologies.
- Flex’s ORv3-compatible rack integrated with single-phase liquid cooling, enabled for two-phase liquid cooling.
- Intermediate bus converter products designed for AI workloads.
For data center customers, Flex customizes hardware design and manufacturing to meet their specific needs. For example, it creates server racks, power distribution units and cooling systems for data centers. The company's clients can save time-to-market and enjoy cost advantages in manufacturing this sort of equipment for data center infrastructure by working with Flex.
The partnership that was announced with JetCool is to deliver these liquid cooling technologies at scale. According to Rob Campbell, president of Communications, Enterprise and Cloud at Flex:
“Advanced cooling systems are essential for supporting AI-enabled applications, high-performance computing, and increased rack power densities,” said JetCool extends the value of single-phase, direct-to-chip liquid cooling deployments to meet the escalating power demands of AI servers. Flex’s expertise across IT and power infrastructure, global manufacturing, and vertical integration makes these solutions easier to deploy at scale.”
And as mentioned, the OCP standard ORv3 racks, which are wider, taller, and allow for more airflow than the familiar 19” rack, will support both single and dual-phase liquid cooling technology. Flex is already in volume production of the ORv3 rack, delivering in quantity to their hyperscaler customers.
JetCool: Pioneering Liquid Cooling Solutions
JetCool Technologies is a company developing advanced cooling technologies for high-performance computing (HPC) and data centers and uses microconvective liquid, focusing on direct-to-chip cooling applications.
Its proprietary technology process leverages microjets to deliver liquid coolant directly to the surface of a microprocessor, or another electronic component or device, or other component generating heat, such as a power converter.
This method is inherently more effective at transferring away heat from the surface of the component that is being cooled, greatly enhancing the capability to cool components more effectively and at lower temperatures than the more old-fashioned air cooling approach.
JetCool’s patented microconvective cooling technology extends the value of single-phase, direct-to-chip liquid cooling deployments by cooling over 1,500W per socket. Dr. Bernie Malouin, CEO of JetCool, tell us:
“As AI and other high-density workloads push the limits of conventional cooling methods, the need for advanced solutions becomes critical. Partnering with Flex allows us to combine our cutting-edge liquid cooling solutions with Flex’s unparalleled manufacturing capabilities. Together, we are poised to deliver servers for AI at scale that not only meet but exceed the performance and efficiency requirements of hyperscalers and enterprise customers.”
More JetCool Liquid Cooling Moves
Cooling typically represents the highest component of energy usage for a data center, often representing over 40 per cent of operational costs. Therefore, exploring alternative cooling methods has been an important focus for the data center and for operators who are striving to be environmentally aware. JetCool knows that one of the most in-demand technologies in this area is direct-to-chip liquid cooling.
As proof of this, also being announced this at the OCP Summit is JetCool’s partnership with Start Campus, the operator of the Sines DC in Portugal, a facility known for its innovated use of seawater cooling. As previously reported by DCF this year, Start Campus offers up to 1.2 GW of renewable energy and is adding JetCool to their portfolio of liquid cooling partners.
Denis Browne, Head of R&D for Start Campus said, of the new partnership:
“We believe in pushing the boundaries of performance and sustainability in data centers. By partnering with JetCool, we’re expanding our suite of cooling options for customers, while maximizing the benefits of SINES DC’s existing sea water cooling infrastructure and AI-readiness. Offering our customers access to a state-of-the-art cooling option that allows them to use very high inlet coolant temperatures enables a data center environment that is not only more efficient and adaptable to a variety of high-density workloads already supported by our facility.”
Stateside, last year JetCool also announced a notable liquid cooling partnership with colocation leader Sabey Data Centers.
Also, JetCool this week said it is launching a 6U in-rack Coolant Distribution Unit (CDU) that is capable of cooling 300 kW and scalable to 2.1 MW at a row level. At the 2024 OCP Global Summit, JetCool will also demonstrate its fully-sealed SmartPlate cold plate, which cools superchips over 3 kW, showcasing the significant headroom potential for single-phase direct-to-chip liquid cooling for high-performance applications, using sustainable non-toxic fluids.
Endgame: Moving DTC Liquid Cooling Into Data Center AI Rack Systems for Improved Energy Efficiency
The new partnership between Flex and JetCool as announced at OCP Global Summit 2024 is premised on the reality that improved energy efficiency represents arguably the principle advantage of direct-to-chip liquid cooling, inasmuch as the technology can accomplish with less energy input the same primary result sought vai air cooling, i.e. heat dissipation.
Because liquid is a far more efficient medium for heat transfer than air, every other thing being equal, direct-to-chip cooling could keep component temperatures lower — and therefore result in much smaller PUEs.
Typically, a small increase in the exothermic characteristics of a dynamic system (such as a data center) translates to greater savings, both for the energy consumed and for the infrastructure required for cooling it.
And because direct-to-chip liquid cooling enables much higher power densities over the same physical footprint, it allows data center operators to run higher server workloads without thermal disruption.
Flex’s Role in Integrating DTC Liquid Cooling
As data center hardware manufacturer, Flex is in positioned to integrate JetCool’s direct-to- liquid cooling solutions in its product portfolio, and further collaborate with a range of cloud computing partners for custom data center rack systems.
When combined with Flex’s industrial design and contract manufacturing expertise, JetCool’s liquid cooling ingenuity facilitates the build of power-optimized hardware for all tiers of data center facilities with the capacity for carbon-positive computing efficiencies.
By enabling faster and more sophisticated rollouts of next-generation cooling, this partnership can provide a competitive edge to operators striving to scale their operations with minimal environmental footprint.
Demand Ongoing for Data Center Cooling Innovation
As it stands, there is no real end in sight for the growth of modern computing workloads. As such, the cooling method of the future with the most potential to transform the thermal management of modern computing in the both near and far terms could be direct-to-chip liquid cooling.
By announcing their partnership on liquid cooling reference designs for data centers, Flex and JetCool, like many others in the industry, are making strides to break down the barriers to liquid cooling adoption -- which, writ large, is clearly where the industry must go to meet the needs of the future data growth sustainably with the goal of energy efficient, sustainable and cooler data centers, as reflected at the OCP Global Summit 2024.
