Introduction

The rapid expansion in the size and capacity of AI workloads is significantly impacting both compute and network technologies used in modern data centers. Although limited to around 100k-200k GPUs today, AI clusters are expected to increase to 1 million GPUs and beyond during the next two to three years. This will have a significant impact on the design and bandwidth requirements of the back-end scale-out network, which connects each GPU to every other GPU in a cluster.

This report provides an overview of the main components of an AI data center’s scale-out network and highlights the key players - both big tech and start-ups - that the provide the crucial connectivity chips to enable high-performance rack-scale based AI computing across hundreds or thousands of racks.