Skip to the shelves

Meta Unveils Advanced Networking Innovations for AI at OCP 2026

Date due card
Shelf AI & ML
Date in
Readers882

Meta's latest networking technologies, launched at OCP 2026, aim to enhance AI infrastructure with innovations like liquid cooling, expansive switching capabilities, and open-source protocols.

Meta Unveils Advanced Networking Innovations for AI at OCP 2026

At the Open Compute Project (OCP) Global Summit 2026, Meta showcased a new generation of open networking infrastructure aimed at empowering the rapidly evolving landscape of artificial intelligence (AI). This announcement marks a significant advancement in high-performance networking, aiming to meet the demands of AI workloads that require unprecedented speed, scalability, and efficiency.

Among the key innovations unveiled are:

  • IcePack: Meta’s first liquid-cooled network rack.
  • IceCube: A 102.4T switch platform designed for high-density environments.
  • Flattened Planar Fabric (FPF): A scalable backend infrastructure that optimizes GPU connectivity.
  • MetaRoCE: An open-source AI transport protocol developed to enhance data transfer efficiency.
  • Open Switch Platform Architecture (OSPA): A new initiative to standardize switch platforms within the OCP ecosystem.

These developments respond to the growing need for a flexible and open networking environment suitable for AI's scale-up and scale-out requirements. As Meta's Co-founder of OCP, the company has been a proponent of open networking since its inception, and this year's contributions reflect the latest strategies aimed at creating sustainable and efficient infrastructures.

Breaking Ground: Liquid-Cooled Networking Solutions

The centerpiece of Meta's hardware unveiling is the IcePack rack, featuring a liquid-cooling design tailored to support the power density required by modern AI data centers. Each IcePack includes IceCube switch blades, boasting a non-blocking switching capacity of 128×800G and a total bandwidth of 102.4 Tbps. This marks a 2x increase in bandwidth compared to their previous 51.2T platforms while effectively managing thermal demands.

The IcePack rack complies with the OCP's ORV3-HPRv1 rack standard, which ensures compatibility and interoperability within the open community. With IceCube switch blades based on either Broadcom's Tomahawk6C or NVIDIA's Spectrum-6 ASICs, Meta’s approach emphasizes open-source software, leveraging the FBOSS, a network operating system that integrates the Open Compute Project Switch Abstraction Interface (SAI).

The IcePack rack — Meta’s first liquid-cooled network rack, populated with IceCube 102T switch blades.

Scaling AI Fabrics: From Buildings to Regions

The operational needs for AI clusters are evolving, necessitating fabric architectures that not only support enormous scales but also maintain resilience and flexibility across multiple geographic sites. The newly introduced Flattened Planar Fabric (FPF) enhances how GPUs connect to the overall network, with a design that employs eight independent 100G links for each GPU, connecting to various switches.

This scalable architecture offers several advantages:

  • Larger non-blocking GPU clusters allow for the high demands of substantial AI workloads.
  • A flattened topology using a 2-tier Clos fabric reduces latency and infrastructure costs.
  • Load balancing at the NIC level begins early in the hardware stack, minimizing congestion while maximizing throughput.
  • Combined control plane for enhanced fault tolerance enables efficient traffic steering away from failing paths.

Collaboratively developed with NVIDIA, the FPF architecture ensures performance remains high, even during adverse conditions. Meta is also collaborating with cloud partners to apply these innovations across shared infrastructures, which will address the challenges of mixed regional and public cloud deployments.

Backend Aggregation: Supporting Gigawatt-Scale Operations

The backend aggregation (BAG) layer plays a critical role in interconnecting massive AI clusters that span conventional data center structures, outdoor components, and colocation spaces. As Meta plans for AI capacity to increase to roughly 1 gigawatt with the Prometheus project and up to 5 gigawatts with Hyperion, such scalability is essential.

Inter-BAG capacity operates at petabit scale, supporting the substantial bandwidth requirements to accommodate operational clusters located across extensive distances. Lossless Ethernet protocols have been adapted for this context, taking physical considerations like cable lengths into account to prevent congestion and ensure robustness in delivery.

Moreover, when shifting workloads across vast geographic locations, BAG leverages eBGP link-bandwidth signaling to optimally route traffic based on real-time conditions, maintaining resilience even in scenarios of partial failure.

Backend aggregation (BAG) connects building-scale AI fabrics across sub-regions into a unified training network.

Advancing Scale-Up Networking with Ethernet

Another significant breakthrough at OCP 2026 revolves around demonstrating that Ethernet can effectively serve the scale-up domain—a requirement for low-latency, high-bandwidth interconnects among accelerators within racks. The newly announced Tahan switch, built on the 51.2 Tbps Tomahawk 5 ASIC, reinforces this commitment. By providing scale-up connectivity to accelerators through a liquid-cooled, non-blocking infrastructure, Tahan successfully integrates the bandwidth elements necessary for advanced AI architecture.

Importantly, Tahan utilizes the same lossless RoCE transport that is employed across Meta's scale-out network, unifying the two operation paradigms under a single layer of software management.

Tahan, Meta’s first open, Ethernet-based scale-up switch, liquid-cooled and running FBOSS.

Looking forward, Meta previewed the next iteration of their scale-up blades, indicating a configuration of 72 accelerators per rack using their latest 102.4T silicon technology—a significant leap forward in system density and efficiency, which is poised to further strengthen their open standards approach.

OSPA and MetaRoCE: Unifying Open Networking Standards

The introduction of the Open Switch Platform Architecture (OSPA) within OCP aims to unify the way switch platforms are built and maintained. This multi-company collaboration, which includes Meta, Microsoft, and Alibaba, focuses on standardizing interfaces for power management, environmental controls, and firmware updates across switches, thereby lowering long-term maintenance costs and increasing operational efficiency.

In addition, Meta's MetaRoCE transport protocol, optimized for AI workloads over Ethernet, offers an open-source solution that tackles the unique challenges of data transmission in large AI clusters. The open-source nature of MetaRoCE enables a wider community to adopt and build on proven transport methodologies, crucial for distributed AI training environments.

Join the Open Future of AI Networking

As demand for AI infrastructure continues to surge, Meta emphasizes the necessity of collaboration across the industry to solve complex networking challenges. They invite engineers, silicon vendors, system integrators, and operators to engage in advancing this initiative within the OCP community.

Through involvement at the OCP Summit and concerted efforts in the development and adoption of tools like FBOSS and MetaRoCE, the goal remains to establish a flexible, efficient, and open networking framework that meets the evolving needs of AI and beyond.

The authors are members of the Network Infrastructure team at Meta.

For further details, refer to the original post on the Engineering at Meta blog.

Source: James Rodriguez · engineering.fb.com

Readers also borrowed