We propose a novel high performance computing (HPC) network architecture$\mathrm {HFOS}_{L}$based on$L$parallel levels distributed low radix fast optical switches (FOS). We provide a detailed description of the blade, FOS and the operation of the$\mathrm {HFOS}_{L}$network. The$\mathrm {HFOS}_{L}$HPC network is highly scalable, and HFOS4 architecture can support an extremely large HPC network of 65,536 blades under distributed FOS with the same radix of 16 in each level. In principle, the$\mathrm {HFOS}_{L}$HPC network can be built by FOSes with different radices at each level. To find out the best configuration of FOS at each level and solve the energy and cost optimization problem in$\mathrm {HFOS}_{L}$network, we break down all the components in the FOS and develop the energy and cost models for the FOS. We verify that the energy and cost per radix functions of FOS are convex functions. Given this foundation, the theoretical investigation of the energy and cost optimization problem shows that the$\mathrm {HFOS}_{L}$network could achieve the minimum energy and cost only when the FOS radices of all levels in$\mathrm {HFOS}_{L}$network are the same. Besides, the cost and power consumption of$\mathrm {HFOS}_{L}$networks are compared with a widely used Leaf-Spine network.

Due to widespread applications and rapid development of cloud computing and Internet services, hyperscale data centers have experienced an unprecedented demand for networking. Data center networks are required not only to handle fast-growing traffic, which doubles almost every one to two years, but also to provide high flexibility and availability to support rapidly changing businesses as well. Therefore, hyperscale data center networks have become one of the main drivers for optical interconnect technologies in recent years. In this article, technologies for scaling optical interconnects for inter-data center interconnects (DCIs) are presented. We first describe hyperscale data center network architectures and requirements, as well as the differences between carrier’s optical transport networks and DCI optical networks, with the main focus on metro-DCI and campus-DCI networks. The scale and fast growth rates of DCI networks require innovations not only in data plane technologies but also in control and management planes as well. In data planes, high-capacity flexible coherent technology, pluggable wavelength-division-multiplexing (WDM) optical transponders, including 100G direct-detection four-level pulse-amplitude modulation (PAM4) and coherent 400ZR/800ZR, and flex-grid and disaggregated reconfigurable optical add-drop multiplexer (ROADM) optical networks are described. Control and management planes are an integral part of optical transport networks and crucial for effectively operating DCI networks. We show that standard protocols, data models, and modular design of a software platform are essential to building a scalable open and disaggregated optical network, which is fundamental to enabling high automation and intelligence in an optical network. An example control and management platform is discussed in detail. With current fiber capacity approaching the nonlinear Shannon limit, challenges to further scale DCI networks and some potential technologies to support ever-increasing traffic growth are discussed at the end.