Optimizing AI Data Center Fabrics for Massive GPU Clusters

Switching architectures are being redesigned around latency, congestion control, telemetry and multi-site connectivity, while AMD’s Helios shows that the competitive unit is shifting from the individual server to the integrated rack. For operators, the practical issue is no longer simply how many GPUs can be installed, but whether the fabric can keep those GPUs supplied with predictable bandwidth.

The network is now part of the accelerator budget

Adding thousands of GPUs without a fabric capable of sustaining their traffic can leave expensive compute underutilized. Data Center Knowledge describes the bottleneck in terms of throughput and latency: as AI jobs generate larger and more irregular communication bursts, network performance directly affects GPU utilization and job-completion times.

The architecture is developing along three dimensions. Scale-up concentrates compute within a server or rack using high-bandwidth links. Scale-out connects additional racks so that larger models and concurrent workloads can share resources. Scale-across extends the cluster between data centers through optical networks, introducing wide-area latency, buffering and traffic-management requirements.

These layers cannot be evaluated with a single bandwidth figure. Intra-rack traffic is dominated by latency and link density; rack-to-rack communication requires congestion avoidance and rapid load balancing; inter-data-center traffic depends on sustained throughput, deep buffering and telemetry that can expose bottlenecks across a much larger physical path.

Cisco is targeting the scale-out layer with its Silicon One G300. The company says the chip is designed for backend networking between GPU racks and supports aggregate bandwidth of 102.4 Tbps through 512 lanes operating at 200 Gbps each. The design also includes real-time telemetry, identity-aware forwarding, traffic visibility and mechanisms intended to redirect packets during congestion.

Cisco outlined availability for the new Silicon One devices at its June Cisco Live event, after the family was initially announced in February 2026. Broad shipments are expected before the end of the year, according to the report.

For scale-across connectivity, Cisco’s Silicon One P200 is positioned for optical links between data centers. The chip provides 51.2 Tbps through 512 100-Gbps links and is paired with external high-bandwidth memory for deeper buffers and traffic management. The objective is to keep optical capacity in use for longer periods rather than allowing bursty traffic and queueing behavior to reduce effective throughput.

Rack-scale systems make networking a product decision

Data Center Frontier’s report on AMD Helios points to the same structural shift from the system side. Helios is described as a rack-scale AI platform combining 72 Instinct MI455X GPUs, sixth-generation EPYC Venice CPUs, Pensando networking and the ROCm software stack.

AMD said Helios is in production, with deployments beginning in the second half of 2026. The system uses liquid-cooled MI455X accelerators based on the CDNA 5 architecture and equipped with HBM4 memory. Networking is included for front-end, scale-up and scale-out traffic rather than treated as a separate layer assembled after the compute design.

AMD is positioning Helios for both training and inference. The company says the platform can deliver up to 30% more inference tokens per dollar than a competing system, while also claiming higher peak AI compute and substantially greater memory capacity than Nvidia’s Rubin GPU. Those comparisons are AMD’s own benchmarks, not independent results, so they should be treated as vendor performance claims rather than settled market measurements.

The more consequential change is architectural. Helios uses Open Compute Project Open Rack Wide, UALink and Ethernet-based networking, presenting rack-scale integration without requiring customers to adopt every element of a single proprietary stack. For hyperscalers and other large operators, that may affect sourcing strategy as much as raw FLOPs or accelerator memory capacity.

The physical implications are equally significant. Helios uses direct liquid cooling, and its Open Rack Wide form factor moves beyond the conventional 19-inch rack. As AI clusters become denser, rack design is increasingly tied to cooling, power delivery, memory topology and network cabling. Nvidia’s roadmap toward Vera Rubin and rack systems operating at several hundred kilowatts illustrates the same direction.

What infrastructure teams should measure

The relevant evaluation is therefore broader than switch throughput. Teams planning AI capacity should map scale-up, scale-out and scale-across traffic separately, then test latency under burst conditions rather than relying only on peak line rate. Buffer depth, telemetry coverage, congestion response and the ability to balance traffic across links become operational variables alongside accelerator count and parameter throughput.

The central constraint is utilization. A cluster with more GPUs but unpredictable bandwidth can produce less usable compute than a smaller system with a better-matched fabric. As rack-scale systems such as Helios enter production and switching silicon moves into the 51.2–102.4-Tbps class, networking is becoming an architectural dependency that must be specified at the same time as FLOPs, HBM capacity, quantization strategy and cooling.