The arrival of 100K+ GPU clusters marks a new frontier in AI infrastructure. Standard communication stack meets new challenges as physical topologies span multiple datacenter buildings, introducing high bandwidth-delay product links where latency increases by up to 30× compared to intra-rack traffic. Furthermore, the transition toward Mixture-of-Experts architectures generating bursty all-to-all patterns that create transient congestion hotspots. These constraints, combined with an operational environment where hardware failures shift from anomalies to frequent occurrences, renders traditionally lightweight operations like initialization and resource management challenging.

The rapid growth in both computational density and scale in AI models in recent years motivates the construction of an efficient and reliable dedicated network infrastructure. This paper presents the design, implementation, and operation of Meta's Remote Direct Memory Access over Converged Ethernet (RoCE) networks for distributed AI training.

Building large AI fleets to support the rapidly growing DL workloads is an active research topic for modern cloud providers. Generating accurate benchmarks plays an essential role in designing the fast-paced software and hardware solutions in this space. Two fundamental challenges to make this scalable are (i) workload representativeness and (ii) the ability to quickly incorporate changes to the fleet into the benchmarks.

Mobile crowd sensing has been a very important paradigm for collecting sensing data from a large number of mobile nodes dispersed over a wide area. Although it provides a powerful means for sensing data collection, mobile nodes are subject to privacy leakage risks since the sensing data from a mobile node may contain sensitive information about the sensor node such as physical locations. Therefore, it is essential for mobile crowd sensing to have a privacy preserving scheme to protect the privacy of mobile nodes. A number of approaches have been proposed for preserving node privacy in mobile crowd sensing. Many of the existing approaches manipulate the sensing data so that attackers could not obtain the privacy‐sensitive data. The main drawback of these approaches is that the manipulated data have a lower utility in real‐world applications. In this paper, we propose an approach calledP3to preserve the privacy of the mobile nodes in a mobile crowd sensing system, leveraging node mobility. In essence, a mobile node determines a routing path that consists of a sequence of intermediate mobile nodes and then forwards the sensing data along the routing path. By using asymmetric encryptions, it is ensured that a malicious node is not able to determine the source nodes by tracing back along the path. With our approach, upper‐layer applications are able to access the original sensing data from mobile nodes, while the privacy of the mobile node is not compromised. Our theoretical analysis shows that the proposed approach achieves a high level of privacy preserving capability. The simulation results also show that the proposed approach incurs only modest overhead.

Participatory sensing has increasingly become a new paradigm of data collection from a wide physical area and a large population. One of the major challenges in participatory sensing is the privacy issue. Sensing data from smartphones may contain sensitive information such as user locations. Thus, it is of great importance to preserve privacy throughout the data collection process in participatory sensing. It is however very challenging because of the distributed nature of the network, many potential malicious attackers and the convergecast model of data collection. In this paper, we present a data collection approach which preserves user privacy in participatory sensing. In this approach, a smartphone node utilizes other smartphones as intermediate nodes to transfer its sensing data. In addition, asymmetric encryption is used to prevent malicious reverse tracking along the data forwarding route, hence anonymizing the originator of the data. We analyze the security of the approach and show that it achieves a high level of security. Extensive simulations demonstrate that the proposed approach has a low overhead.