Chiplet-based architectures emerge as a promising approach to overcoming the physical and manufacturing constraints faced by monolithic chips, enabling the scalable integration of computing resources to meet the growing demands of AI workloads. However, efficient inter-chiplet communication still faces significant bottlenecks, especially under the fine-grained and bursty Direct Memory Access (DMA) request patterns generated by processing elements in modern AI tasks. Existing communication models and simulators fail to capture these characteristics, which limits the accuracy of performance analysis and the effectiveness of optimization strategies. These limitations hinder DMA-communication inefficiencies in chiplet-based AI systems and pose challenges for designing HPC architectures. To address these challenges, we present the first comprehensive chiplet communication model that explicitly incorporates finegrained DMA traffic observed in realistic AI workloads. Building on this model, we propose COMET, a novel framework that intelligently searches for optimal DMA request aggregation and memory address mapping strategies tailored to chiplet environments. COMET dynamically consolidates small DMA transfers to improve bandwidth utilization and reduce communication latency, while also adapting on-chip memory mapping to align with workload-specific dataflows. This mitigates synchronization overhead across diverse AI tasks. Compared with inference on conventional chiplet communication schemes, COMET achieves$1.1 \times-2.6 \times$speedup and$1.5 \times-4.4 \times$higher bandwidth utilization across different DNN and LLM workloads.

Conventional heterogeneous computing systems built on PCIe interconnects suffer from inefficient fine-grained host-device interactions and complex programming models. In recent years, many proprietary and open cache-coherent interconnect standards have emerged, among which compute express link (CXL) prevails in the open-standard domain after acquiring several competing solutions. Although CXL-based coherent heterogeneous computing holds the potential to fundamentally transform the collaborative computing mode of CPUs and XPUs, research in this direction remains hampered by the scarcity of available CXL-supported platforms, immature software/hardware ecosystems, and unclear application prospects. This paper presents Cohet, the first CXL-driven coherent heterogeneous computing framework. Cohet decouples the compute and memory resources to form unbiased CPU and XPU pools which share a single unified and coherent memory pool. It exposes a standard malloc/mmap interface to both CPU and XPU compute threads, which share a single per-process page table for user applications, leaving the OS dealing with smart memory allocation, page auto-migration, and management of heterogeneous resources. This design significantly simplifies heterogeneous parallel programming to a level comparable to homogeneous programming. To facilitate Cohet research, we also present a fullsystem cycle-level simulator named SimCXL, which is capable of modeling all CXL sub-protocols and device types. SimCXL has been rigorously calibrated against a real CXL testbed with various CXL memory and accelerators, showing an average simulation error of 3 %. Our evaluation reveals that CXL.cache reduces latency by 68 % and increases bandwidth by$14.4 \times$compared to DMA transfers at cacheline granularity. Building upon these insights, we demonstrate the benefits of Cohet with two killer apps, which are remote atomic operation (RAO) and remote procedure call (RPC). Compared to PCIe-NIC design, CXL-NIC achieves a 5.5 to$40.2 \times$speedup for RAO offloading and an average speedup of$\mathbf{1. 8 6} \times$for$\mathbf{R P C}$(de)serialization offloading.

Mixture-of-Experts (MoE) models face memory and PCIe latency bottlenecks when deployed on commodity hardware. Offloading expert weights to CPU memory results in PCIe transfer latency that exceeds GPU computation by several folds. We present PreScope, a prediction-driven expert scheduling system that addresses three key challenges: inaccurate activation prediction, PCIe bandwidth competition, and cross-device scheduling complexity. Our solution includes: 1) Learnable Layer-Aware Predictor (LLaPor) that captures layer-specific expert activation patterns; 2) Prefetch-Aware Cross-Layer Scheduling (PreSched) that generates globally optimal plans balancing prefetching costs and loading overhead; 3) Asynchronous I/O Optimizer (AsyncIO) that decouples I/O from computation, eliminating waiting bubbles. PreScope achieves 141% higher throughput and 74.6% lower latency than state-of-the-art solutions.

Krylov subspace methods are widely used in scientific computing to solve large sparse linear systems and eigenvalue problems. Their performance bottleneck is often dominated by high-order matrix-power kernels (MPK), especially in polynomial preconditioners that must scale to millions or billions of variables. We present Diagonal Block MPK (DBMPK), a lightweight and parallel-friendly optimization that partitions the input matrix into diagonal blocks and off-diagonal regions. This design enables efficient intra-block data reuse and eliminates inter-block dependencies. It improves cache locality, parallelism, and reduces preprocessing overheads, compared to existing techniques. Our evaluation on x86 and Arm HPC platforms shows that DBMPK improves MPK performance by 26.6%-38.4%. When applied to polynomial preconditioners for linear systems and eigenvalue problems, it achieves consistent end-to-end speedups of 18.6%-34.0%, including in weak scaling tests on 128 nodes, demonstrating strong scalability and practical impact.

Sparse matrix–matrix multiplication (SpMM) is a fundamental operation in scientific computing with broad applications across numerous domains. Tiling is a key optimization technique for improving data locality and is widely adopted in high-performance computing. However, the irregular data access patterns inherent to SpMM make it challenging to exploit tiling effectively for data reuse. In this article, we propose MaSpMM , a memory-aware SpMM framework that integrates cache-aware tiling with a segment-oriented data layout. MaSpMM stores matrices as continuous segments to enhance data locality within each tile. Moreover, since many sparse matrices in real-world applications exhibit symmetry, we further develop MaSpMM-Sym, an extension that recursively partitions symmetric matrices to eliminate write conflicts and further improve locality. To adapt to diverse scenarios, we finally introduce MaSpMM-Adap, which adaptively selects the most suitable approach for each input matrix. Comprehensive evaluations on both x86 and ARM CPUs demonstrate that MaSpMM-Adap achieves average speedups of up to 1.86× over Intel oneMKL, 1.84× over ASpT, and 1.75× over J-Stream.

Compute eXpress Link (CXL) has emerged as a key enabler of memory disaggregation for future heterogeneous computing systems to expand memory on-demand and improve resource utilization. However, CXL is still in its infancy stage and lacks commodity products on the market, thus necessitating a reliable system-level simulation tool for research and development. In this paper, we propose CXL-DMSim1, an open-source full-system simulator to simulate CXL disaggregated memory systems with high fidelity at a gem5-comparable simulation speed. CXL-DMSim incorporates a flexible CXL memory expander model along with its associated device driver, and CXL protocol support with CXL.io and CXL.mem. It can operate in both app-managed mode and kernel-managed mode, with the latter using a dedicated NUMA-compatible mechanism. The simulator has been rigorously verified against a real hardware testbed with both FPGA- and ASIC-based CXL memory devices, which demonstrates the qualification of CXL-DMSim in simulating the characteristics of various CXL memory devices at an average simulation error of 3.4%. The experimental results using LMbench and STREAM benchmarks suggest that the CXL-FPGA memory exhibits a ~2.88× higher latency than local DDR while the CXL-ASIC latency is ~2.18×; CXL-FPGA achieves 45-69% of local DDR memory bandwidth, whereas the number for CXL-ASIC is 82-83%. The study also reveals that CXL memory can significantly enhance the performance of memory-intensive applications, improved by 23× at most with limited local memory for Viper key–value database and approximately 60% in memory-bandwidth-sensitive scenarios such as MERCI. Moreover, the simulator’s observability and expandability are showcased with detailed case-studies, highlighting its great potential for research on future CXL-interconnected hybrid memory pool.

The General Matrix Multiplication (GEMM) is a crucial subprogram in high-performance computing (HPC). With the increasing importance of power and energy consumption, modern Digital Signal Processors (DSPs) are being integrated into general-purpose HPC systems. However, due to architecture disparities, traditional optimizations for CPUs and GPUs are not easily applicable to modern DSPs. This paper shares our experience of optimizing the GEMM operation using a CPU-DSP platform as a case study. Our work employs a set of strategies to improve the performance and scalability of GEMM. These strategies focus on developing micro-kernels based on heterogeneous on-chip memory, addressing the memory access bottleneck in multi-core parallelism, and facilitating efficient transpose-GEMM. These approaches, collectively referred to as an efficient and practical library (a.k.a.mtGEMM), maximize computational capabilities and bandwidth utilization of multi-core DSPs, while achieving high performance for variously-shaped GEMMs. Our experimental results demonstrate thatmtGEMMcan attain between 92% and 96% of the hardware peak, with the multi-core scalability being almost linear.

Online zero-shot 3D instance segmentation of a progressively reconstructed scene is both a critical and challenging task for embodied applications. With the success of visual foundation models (VFMs) in the image domain, leveraging 2D priors to address 3D online segmentation has become a prominent research focus. Since segmentation results provided by 2D priors often require spatial consistency to be lifted into final 3D segmentation, an efficient method for identifying spatial overlap among 2D masks is essential—yet existing methods rarely achieve this in real time, mainly limiting its use to offline approaches. To address this, we propose an efficient method that lifts 2D masks generated by VFMs into a unified 3D instance using a hashing technique. By employing voxel hashing for efficient 3D scene querying, our approach reduces the time complexity of costly spatial overlap queries from O(n2) to O(n). Accurate spatial associations further enable 3D merging of 2D masks through simple similarity-based filtering in a zero-shot manner, making our approach more robust to incomplete and noisy data. Evaluated on the ScanNet200 and SceneNN benchmarks, our approach achieves state-of-the-art performance in online, zero-shot 3D instance segmentation with leading efficiency. The project page is at https://yjtang249.github.io/OnlineAnySeg.

Data prefetchers play a crucial role in alleviating the memory wall by predicting future memory accesses. First-level cache prefetchers can observe all memory instructions but often rely on simpler strategies due to limited resources. While emerging machine learning-based approaches cover more memory access patterns, they typically require higher computational and storage resources and are usually deployed in the last-level cache. Other intelligent solutions for the first-level cache show only modest performance gains. To address this, we propose Amphi, the first practical and intelligent data prefetcher specifically designed for the first-level cache. Applying a binarized temporal convolutional network, Amphi significantly reduces storage overhead while maintaining performance comparable to the SOTA intelligent prefetcher. With a storage overhead of only 3.4 KB, Amphi requires only one-eighth of Pythia's storage needs. Amphi paves the way for the broader adoption of intelligence-driven prefetching solutions.

As network simulations scale to hundreds of thousands of nodes, parallel discrete event simulation (PDES) has become indispensable for sustaining performance—yet its efficacy hinges on frequent time synchronization steps. Existing synchronization algorithms suffer from the inter-machine communication overhead in distributed environments, eroding the benefits of parallelism. We observe that time synchronization in PDES often involves frequent allreduce operations, and that in-network computing has the potential to substantially accelerate such collectives. In this work, we present SINA (Synchronization using In-Network Allreduce)—the first integration of in-network computing into time synchronization to address the demands of large-scale simulations. We implemented SINA in a testbed with Mellanox SHArP-enabled switches and ConnectX-5 network cards, offloading allreduce to the network hardware while preserving software‑level correctness. Our evaluation shows that SINA achieves up to 88.6% acceleration compared to state‑of‑the‑art methods and achieves up to 67.5% optimization in topologies of tens of thousands of nodes, demonstrating its suitability for high‑performance, large‑scale parallel simulations.

Algebraic multigrid (AMG) is widely used to accelerate largescale sparse linear solvers.In distributed environments, neighboring communication overhead in AMG significantly impacts overall solution time.We propose Communication-Reduced Algebraic Multigrid (CRAMG) methods to minimize inter-process data exchange and message count by fusing interpolation/restriction operators with residual computations.This reduces communication frequency from four per level to as few as two.Experiments show up to 45% reduction in data exchange and 35% fewer messages.Performance evaluations on an Intel platform demonstrate significant improvements

Interconnection networks are playing an increasingly crucial role in achieving scalability and throughput for post-exascale and zettascale computing systems.Resource consumption characteristics of AI and HPC workloads are essential to designing effective network infrastructure.The operation practice of production supercomputing systems reveals that small and medium-sized jobs consume most compute-core hours, generating traffic that mainly utilizes a size-restricted portion of the network.Unfortunately, current topologies lack sufficient support for flexible partitioning of distinct concurrent jobs.This study bridges this gap by exploring a new tradeoff among scalability, throughput, and non-blocking regions.We present Zettafly, a family of low-diameter topologies with large-scale non-blocking sub-networks, allowing the majority of jobs to be isolated within a single sub-network.Zettafly is a two-layer structure, consisting of non-blocking groups and global routers, such that each minimally routed packet between groups traverses at most one global router.We also introduce simple and efficient adaptive routing algorithms for Zettafly.We conduct extensive simulations and analysis to evaluate the performance and cost of Zettafly against state-of-the-art topologies.The results show that Zettafly achieves great performance under typical HPC workloads, offers cost-effectiveness under multi-task mixed traffic, supports flexible configurations and incremental deployments, and is costeffective to deploy for post-exascale and zettascale systems.

Driven by the rapidly growing demands of modern applications, RDMA technology has been widely adopted in cluster networks, including high-performance computing (HPC) systems and datacenters. However, the high-speed, kernel-bypass nature of RDMA presents significant challenges for congestion control (CC). On one hand, since most RDMA operations are offloaded to the RDMA network interface card (RNIC), software-based CC struggles to regulate packet transmission effectively. On the other hand, RDMA workloads are dominated by short messages, making it difficult for both reactive and proactive CC protocols to manage congestion in a timely and effective manner, posing risks to network application QoS. Despite these challenges, proactive and reactive CC protocols each have distinct strengths, and combining them offers a promising research direction. This paper proposes an endpoint-network collaborative congestion control protocol (HNCC) that integrates proactive endpoint congestion control with reactive network monitoring and adjustment to achieve precise and rapid congestion management. HNCC introduces a novel reservation mechanism embedded within the RDMA protocol to prevent endpoint congestion while efficiently mitigating in-network congestion by dynamically adjusting the data injection rate based on network trend analysis. HNCC is offloaded to the RNIC, ensuring that its traffic scheduling is timely and effective. Extensive load testing across various scales shows that HNCC achieves performance comparable to state-of-the-art CC protocols in datacenter workloads and reduces the average flow completion time (FCT) by 5 − 56% under HPC workloads.

Transformer-based large models, with trillions of parameters and massive datasets, have driven breakthroughs in NLP, vision, and multimodal tasks. However, their rapid growth poses substantial challenges for training within limited GPU resources, making distributed training indispensable.

This paper presents DAS-ILU, a Distributed Asynchronous parallel Incomplete LU factorization method based on domain decomposition. DAS-ILU partitions the computational domain into independently processed interior nodes and asynchronously updated separator nodes, thereby reducing cross-processor dependencies and halving the separator size compared to conventional methods. To further improve performance, it employs optimized data exchange patterns to minimize communication overhead and extends support to block-structured sparse matrices via exact block inversions. Comprehensive evaluations on a range of problem types—including structural mechanics, computational fluid dynamics, and reservoir simulation demonstrate the superior performance of DAS-ILU. Compared to state-of-the-art ILU implementations, DAS-ILU achieves solve time speedups of up to 2.07 × over Chow-Patel’s fine-grained parallel ILU and up to 4.11 × over HYPRE’s ILU. Moreover, DAS-ILU exhibits strong robustness when applied to challenging nonsymmetric and indefinite systems.

Graph processing underpins a vast array of data-centric applications, serving as a crucial component in fields such as social network analysis, recommendation systems, bio-informatics, and search engines. As graph data grows in scale and complexity, high-performance graph processing is increasingly essential. Many graph processing tasks depend on efficient data structures to manage the sparsity typical of real-world graphs, where most vertices have limited connectivity. This sparsity poses challenges for memory and computational efficiency in large-scale graph processing, and conventional sparse formats like Compressed Sparse Row (CSR) often struggle with memory and computation inefficiencies when handling massive graphs. To address these challenges, we introduce GraphCSR, a degree-equalized CSR format specifically tailored to enhance the spatio-temporal efficiency of distributed graph processing across various tasks. GraphCSR aggregates low-degree vertices into synthetic high-degree ones and applies group-wise compression to reduce storage overhead by recording only the starting index for each aggregated group. This reduces memory usage and supports batch-memory access to improve performance. Our extensive evaluations in various graph processing algorithms and datasets demonstrate that GraphCSR not only reduces the memory footprint required for large-scale graphs, but also improves performance across multiple types of graph processing tasks, outperforming popular sparse storage formats. Furthermore, when deployed on a production-scale supercomputer with 79,024 nodes, GraphCSR achieved a graph processing throughput that exceeded the top-ranked system on the Graph500 benchmark.

Convolution kernels are widely seen in high-performance computing (HPC) and deep learning (DL) workloads and are often responsible for performance bottlenecks. Prior works have demonstrated that the direct convolution approach can outperform the conventional convolution implementation. Although well-studied, the existing approaches for direct convolution are either incompatible with the mainstream DL data layouts or lead to suboptimal performance. We designnDirect2, a novel direct convolution approach that targets multi-core CPUs commonly found in smartphones and HPC systems.nDirect2is compatible with the data layout formats used by mainstream DL frameworks and offers new optimizations for the computational kernel, data packing, advanced operator fusion, and parallelization. We evaluatenDirect2by applying it to representative convolution kernels and demonstrating how well it performs on four distinct ARM-based CPUs and an X86-based CPU. Experimental results show thatnDirect2outperforms four state-of-the-art convolution approaches across most evaluation cases and hardware architectures.

Tiny object detection is increasingly crucial in the fields such as remote sensing, traffic monitoring, and robotics. Inspired by human visual perception, attention mechanism has become a widely used method for enhancing object detection performance. While existing attention mechanisms have significantly advanced general object detection performance, they often fall short in adapting to the characteristics in tiny object datasets, including huge object size variations and concentrated distributions. In detailed, most current attention mechanisms rely on convolutional or linear layers with fixed receptive fields to compute attention vectors. Some methods attempt to enlarge the receptive fields by using multiscale structures, but they often simply sum feature maps, leading to information interference and increased computational costs. To address these issues, we propose a novel Multiscale Gaussian Attention Mechanism (MGAM). This mechanism integrates multiscale receptive fields with dynamic feature weighting and a Gaussian attention module, replacing traditional convolutional layers to reduce training and inference overhead. In additional, our mechanism can be easily embedded into various detectors without any hyperparameters. Extensive experiments on six object detection datasets demonstrate the effectiveness and robustness of our method. Code is available at: https://github.com/cszzshi/MGAM.

Data prefetching is an essential technique in contemporary high-performance processors for mitigating the effects of long-latency memory accesses. With the increasing demand for prefetcher to learn complex memory access patterns, many state-of-the-art prefetchers adopt methods such as using the program counter or access delta to separate memory access streams. This allows them to learn detailed memory access features and thereby improve memory system performance. However, the separation-based approach is prone to missing global correlations, leading to miss prefetching opportunities. In this paper, we propose Chimera, a hybrid offsets prefetcher that captures multiple offset features from the overall stream of memory access instructions, thus overcoming the drawbacks of traditional prefetchers that tend to lose memory access information when learning from one-sided features. We evaluated Chimera using SPEC CPU 2006 and 2017 through simulation, and the results show that Chimera improves system performance by 39.5% over a baseline with no data prefetcher and by 6.6% over the state-of-the-art data prefetcher.

Cell-cell communications (CCCs) mediated by ligand-receptor interactions (LRIs) play a pivotal role in coordination and function of biological systems. The two primary steps of CCC inference methods are usually filtering significant LRIs and measuring the intercellular communication strength. Biological language models have demonstrated notable progress in bioinformatics across several biological problems, inspiring by the success of language models in the field of natural language processing. Here, we proposed a computational approach for CCC inference called SeqCCC. First, SeqCCC employed two sophisticated protein language models and the protein sequences of ligands and receptors to predict potential LRIs with the help of existing LRIs. Then, SeqCCC employed permutation test to filter significantly expressed LRIs from the single-cell expression matrix based on these known and predicted LRIs. Finally, SeqCCC calculated communication strength by applying the molecular diffusion and law of mass action in chemistry, and then it improves this by eliminating non-specific CCCs using a permutation test. In the results, SeqCCC yielded an average AUC of 0.93 (mean of the five folds, with a standard deviation of 0.02). The validity and reliability of SeqCCC are further supported by a comparative analysis with common CCC inference methods, which reveals high concordance in inference results. Additionally, SeqCCC provided different visualization methods for CCC results, including Circos Plot, Heatmap Plot, Dot Plot and Heatmap for LRIs. In conclusion, SeqCCC provides a new option for CCC inference by fusing biological language models with sequence information to improve our comprehension of intercellular communications.

The limitations of reactive congestion control protocols in scheduling buffer become more apparent due to its long feedback delay. As a result, the switch buffer management (BM) policy respond directly to complex traffic within the network is critical to optimizing network performance because it determines the buffer occupancy pattern. Nevertheless, the existing BMs exhibit an inadequate consideration of the unique traffic character of contention scenarios, thereby overlooking the potential problem of threshold truncation. After analysis, we find that the root cause of this problem is that existing BMs are sensitive to contention. Therefore, we blunt BM's perceptibility to contention changes (BPC) to avoid the possible threshold truncation. Experiments show that BPC can avoid packet loss and retransmission triggered by the threshold truncation.

The sparse matrix-vector multiplication (SpMV) is widely used in large-scale scientific computing and engineering. However, optimizing SpMV for high-performance digital signal processors (DSPs) has received limited attention. We present HaLAV, a method to accelerate SpMV on CPU-DSP heterogeneous platforms, using the FT-M7032 DSP platform as a case study. HaLAV partitions the input matrix into ‘dense’ and ‘sparse’ parts through column reordering. For the dense part, HaLAV automatically selects storage formats optimized for vectorization to run on the DSP. At the same time, it offloads the sparse component to be processed by the CPU using the standard CSR algorithm. We evaluate our approach on the FT-M7032 platform and an Intel Xeon CPU. Experimental results show that our techniques achieve average speedups of 2.09 × and 1.66 × over the competing baselines on the FT-M7032 and the Xeon platform, respectively.

Transformers reign supreme in natural language processing, representing a milestone innovation in deep learning. For high-performance model inference, optimizing the time-consuming attention module is crucial. Owing to the irregular-shaped matrix workloads and intricate data access patterns, the attention operator is bounded by memory bandwidth. Existing works utilize kernel fusion to reduce memory access overhead, resulting in promising performance enhancements. However, these efforts primarily focus on GPU or X86 architectures, leaving ARM multi-cores, commonly encountered in emerging HPC systems, insufficiently explored. We present MEATTEN, a memory-efficient attention fusion scheme and batched approach to exploit ARM multi-core CPUs effectively. It builds on fused micro-kernels and a new data layout suitable for SIMD vectorization. An analytic model is used to guide loop permutation, tiling, and batched parallelization according to the on-chip hierarchical memory architecture and workload characterization. We apply MEATTEN to three representative ARM multi-cores against state-of-the-art libraries and compilers. Experimental results demonstrate that our approach consistently outperforms prior approaches across various evaluation scenarios and platforms.

Code comment generation aims at generating natural language descriptions for a code snippet to facilitate developers' program comprehension activities. Despite being studied for a long time, a bottleneck for existing approaches is that given a code snippet, they can only generate one comment while developers usually need to know information from diverse perspectives such as what is the functionality of this code snippet and how to use it. To tackle this limitation, this study empirically investigates the feasibility of utilizing large language models (LLMs) to generate comments that can fulfill developers' diverse intents. Our intuition is based on the facts that (1) the code and its pairwise comment are used during the pre-training process of LLMs to build the semantic connection between the natural language and programming language, and (2) comments in the real-world projects, which are collected for the pre-training, usually contain different developers' intents. We thus postulate that the LLMs can already understand the code from different perspectives after the pre-training. Indeed, experiments on two large-scale datasets demonstrate the rationale of our insights: by adopting the in-context learning paradigm and giving adequate prompts to the LLM (e.g., providing it with ten or more examples), the LLM can significantly outperform a state-of-the-art supervised learning approach on generating comments with multiple intents. Results also show that customized strategies for constructing the prompts and post-processing strategies for reranking the results can both boost the LLM's performances, which shed light on future research directions for using LLMs to achieve comment generation.

General Matrix Multiplication (GEMM) is a key subprogram in high-performance computing (HPC) and deep learning workloads. With the rising significance of power and energy consumption in HPC systems, accelerators based on Digital Signal Processors (DSPs) have been integrated into general-purpose HPC systems. Due to the architecture disparities, the GEMM optimization techniques used on conventional multi-core CPUs and GPGPUs are not always applicable to DSPs. This paper shares our experience in optimizing GEMM on multi-core GPDSPs, using a CPU-DSP processor as a case study. Our approach employs a range of techniques to optimize performance for DSP architectures. These include data partitioning, three-level pipelining, dedicated micro-kernel design, and improved vector reduction. These optimizations maximize the overlap between computation and communication while fully exploiting the capabilities of floating-point arithmetic units to achieve high performance. Our experimental results demonstrate that the performance attained by our optimization is up to 96% of the theoretical peak performance of the hardware.

Processing large-scale graphs with billions to trillions of edges requires efficiently utilizing parallel systems. However, current graph processing engines do not scale well beyond a few tens of computing nodes because they are oblivious to the communication cost variations across the interconnection hierarchy. We introduce GraphCube, a better approach to optimizing graph processing on large-scale parallel systems with complex interconnections. GraphCube features a new graph partitioning approach to achieve better load balancing and minimize communication overhead across multiple levels of the interconnection hierarchy. We evaluate GraphCube by applying it to fundamental graph operations performed on synthetic and real-world graph datasets. Our evaluation used up to 79,024 computing nodes and 1.2+ million processor cores. Our large-scale experiments show that GraphCube outperforms state-of-the-art parallel graph processing methods in throughput and scalability. Furthermore, GraphCube outperformed the top-ranked systems on the Graph 500 list.

Remote Memory Access (RMA) enables direct access to remote memory to achieve high performance for HPC applications. However, most modern parallel programming models lack schemes for the remote process to detect the completion of RMA operations. Many previous works have proposed programming models and extensions to notify the communication peer, but they did not solve the multi-NIC aggregation, portability, hardware-software co-design, and usability problems. In this work, we proposed a Unified Notifiable RMA (UNR) library for HPC to address these challenges. In addition, we demonstrate the best practice of utilizing UNR within a real-world scientific application, PowerLLEL. We deployed UNR across four HPC systems, each with a different interconnect. The results show that PowerLLEL powered by UNR achieves up to a 36% acceleration on 1728 nodes of the Tianhe-Xingyi supercomputing system.

The Sparse Triangular Solver (SPTRSV) plays a critical role in solving structured grid problems. Yet, the commonly used sparse matrix storage formats for structured grid methods do not efficiently support SPTRSV in utilizing the instruction parallelism offered by modern multi-core CPUs. We introduce DBSR, a new sparse storage format to enable SPTRSV to take advantage of the SIMD instructions. DBSR promotes contiguous memory access and vectorized computation, while also optimizing memory usage. We evaluate DBSR by applying it within multigrid algorithms and the zero fill-in incomplete $\mathbf{L U}$ preconditioner. Our evaluation, conducted on four architectures - three ARMv8 systems and one x86 system - demonstrates that DBSR consistently outperforms mainstreamed storage formats across evaluation workloads and platforms.

RDMA (Remote Direct Memory Access) networks require efficient congestion control to maintain their high throughput and low latency characteristics. However, congestion control protocols deployed at the software layer suffer from slow response times due to the communication overhead between host hardware and software. This limitation has hindered their ability to meet the demands of high-speed networks and applications. Harnessing the capabilities of rapidly advancing Network Interface Cards (NICs) can drive progress in congestion control. Some simple congestion control protocols have been offloaded to RDMA NICs to enable faster detection and processing of congestion. However, offloading congestion control to the RDMA NIC faces a significant challenge in integrating the RDMA transport protocol with advanced congestion control protocols that involve complex mechanisms. We have observed that reservation-based proactive congestion control protocols share strong similarities with RDMA transport protocols, allowing them to integrate seamlessly and combine the functionalities of the transport layer and network layer. In this article, we present COER, an RDMA NIC architecture that leverages the functional components of RDMA to perform reservations and completes the scheduling of congestion control during the scheduling process of the RDMA protocol. COER facilitates the streamlined development of offload strategies for congestion control techniques —specifically, proactive congestion control —on RDMA NICs. We use COER to design offloading schemes for 11 congestion control protocols, which we implement and evaluate using a network emulator with a cycle-accurate RDMA NIC model that can load Message Passing Interface (MPI) programs. The evaluation results demonstrate that the architecture of COER does not compromise the original characteristics of the congestion control protocols. Compared with a layered protocol stack approach, COER enables the performance of RDMA networks to reach new heights.

General Matrix Multiplication (GEMM) is a key subroutine in high-performance computing. While the mainstream Basic Linear Algebra Subprograms (BLAS) libraries can deliver good performance on large and regular-shaped GEMMs, they are inadequate for optimizing small and irregular-shaped GEMMs, which are commonly seen in emerging HPC applications. Recent research has focused on improving GEMM performance on GPUs, but there is still significant room for improvement on emerging HPC hardware based on multi-core CPUs. We presentLibShalom2, an open-source library to optimize full-spectrum GEMMs, taking small, irregular-shaped, and large-scale regular-shaped matrices.LibShalom2explicitly targets the ARMv8 architecture, which is becoming common in HPC systems.LibShalom2is designed to minimize the expensive memory accessing overhead for data packing and processing small matrices. It uses analytic methods to determine GEMM kernel optimization parameters, enhancing the computation and parallelization efficiency of the GEMM kernels. We evaluateLibShalom2by applying it to three ARMv8 multi-core architectures and comparing it against five mainstream linear algebra libraries. Experimental results show thatLibShalom2consistently outperforms existing solutions across full-spectrum GEMM workloads and hardware architectures. We also show thatLibShalom2delivers an average speedup of 2.2x for real-life neural network workloads.

Multigrid preconditioned conjugate gradient (MGPCG) is commonly used in high-performance computing (HPC) workloads. However, MGPCG is notoriously challenging to optimize since most of its computation kernels are memory-bounded with low arithmetic intensity and non-trivial communication patterns among parallel processes. This article presents new techniques to improve the data locality and reduce the communication overhead of MGPCG by first merging the kernels of multigrid (MG). We then develop an asynchronous neighboring communication algorithm to reduce the data communications across parallel processes. We demonstrated the benefits of our approach by applying it to the high-performance conjugate gradient (HPCG) benchmark and integrating it with a real-life algebraic multigrid package. We test the resulting software implementations on three ARMv8 and one Intel Xeon system. Experimental results show that our approach leads to a 1.62x-2.54x speedup over the engineer- and vendor-tuned HPCG implementations across various workloads and platforms.

Sparse Matrix-Vector Multiplication (SpMV) is a fundamental operation in sparse computations. Although many techniques have been developed to speed up SpMV, optimizing this process on low-power multicore digital signal processors (DSPs) has often been neglected. This paper presents the FT-M7032, a cutting-edge CPU-DSP hybrid multi-core processor. We assess the data transfer efficiency among various units to identify performance constraints of SpMV on multicore DSPs. Based on our evaluation, we develop a method for block sparse matrices, namely SpMV_BLOCK, which can break the bandwidth bottleneck of SpMV and achieve significant performance gains. We then propose a load-balancing strategy for each thread by using binary search and devise a pipeline that overlaps data transfers and computations to improve SpMV performance. To measure our method’s effectiveness, we compared its performance against a baseline on the FT-M7032’s general-purpose CPU cores. Our experiments show that our approach delivers a notable 5.80 × speedup over the baseline.

The dense linear solver is an essential subroutine in high-performance computing. Typical parallel implementations either adopt the fork-join or task parallel programming models. Blocked algorithms built upon the fork-join paradigm focus on optimizing cache locality, leaving significant synchronization overhead. Following the data-driven execution model, tile-based algorithms formed on the task parallel paradigm effectively relieve the pain and exhibit superior load balancing. Nevertheless, they introduce redundant memory access expenses, plaguing the CPU execution. In this paper, we first characterize and quantify the impact of the performance bottlenecks in-depth and then propose a series of optimizations. Specifically, we reduce the idle time of threads by merging LU factorization with the subsequent lower triangular solver to improve parallelism. Moreover, we eliminate tile-based matrix format transformation and diminish duplicated data packing operations to lower memory access overhead. Performance evaluation is conducted on two modern multi-core systems, Intel Xeon Gold(R) 6252N and HiSilicon Kunpeng 920. The evaluation results demonstrate the superiority of our proposed solver over state-of-the-art open-source implementations, achieving performance gains of up to 11.5% and 12.2% on the respective platforms.

The routing algorithm is important for the design of high-performance interconnection networks. With the increasing size of high-performance computing (HPC) systems, the possibility of network component failures increases simultaneously. Fault tolerance becomes a more critical consideration for the routing algorithms, as failed network devices, mostly network links, corrupt the regularity of the topology. However, existing routing algorithms focus on load balancing, ignoring that when the network is faulty. Similarly, current fault-tolerant routing algorithms ensure the correct functionality when failures exist without considering the more challenging post-failure load balancing. In this paper, we co-design the load balancing and fault tolerance for adaptive routing algorithms in fat tree networks and propose DFAR, Dynamic-threshold Fault-tolerant Adaptive Routing. DFAR prioritizes D-Mod-K deterministic output ports by adding thresholds to other available candidates. We adopt a simulated gradient descent algorithm to dynamically update the thresholds according to the network state changes. More state information other than local queue occupancies is used in the thresholds optimization. Experiments with synthetic load show that DFAR improves throughput by up to 25% for network with considerable failures. For realistic MPI workloads, DFAR also improves performance by up to 22%.

Delay-based congestion control (CC) protocols have been widely used in data centers, where senders monitor the real-time delay and adjust the sending rate or congestion window accordingly. We propose Rately, a CC algorithm that utilizes accurate one-way delay (OWD) measurements and a compatible rate regulation mechanism. Experiments show that Rately greatly improves performance under realistic workload.

Dragonfly is a highly scalable, low-diameter, and cost-efficient network topology, which has been adopted in new exascale High Performance Computing (HPC) systems. However, Dragonfly topology suffers from the limited direct links between groups. The reconfigurable network can solve this problem by reconfiguring topology to adjust the number of direct links between groups. While the performance improvement of a single job on reconfigurable HPC network has been evaluated in previous works, the performance of HPC workloads has not been studied because of the lack of an appropriate resource allocation policy.

The allreduce operation is the most commonly used collective operation in distributed or parallel applications. It aggregates data collected from distributed hosts and broadcasts the aggregated result back to them. In-network computing can accelerate allreduce by offloading this operation into network devices. However, existing in-network solutions face the challenge of high throughput, performance of aggregating large message and producing repeatable results. In this work, we propose a simple and effective router microarchitecture for in-network allreduce, which uses an RDMA protocol to improve its throughput. We further discuss strategies to tackle the aforementioned challenges. Our approach not only shows advantages in comparison with the state-of-the-art in-network solutions, but also accelerates allreduce at a near-optimal level compared to host-based algorithms, as demonstrated through experiments.

Multigrid algorithms are widely used to solve large-scale sparse linear systems, which is essential for many high-performance workloads. The symmetric Gauss-Seidel (SYMGS) method is often responsible for the performance bottleneck of MG. This paper presents new methods to parallelize and enhance the computation and parallelization efficiency of the SYMGS and MG algorithms on multi-core CPUs. Our solution employs a matrix splitting strategy and a revised computation formula to decrease the computation operations and memory accesses in SYMGS. With this new SYMGS strategy, we can then merge the two most time-consuming components of MG. On top of these, we propose a new asynchronous parallelization scheme to reduce the synchronization overhead when parallelizing SYMGS. We demonstrate the benefit of our techniques by integrating them with the HPCG benchmark and two real-life applications. Evaluation conducted on four architectures, including three ARMv8 and one x86, shows that our techniques greatly surpass the performance of engineer- and vendor-tuned implementations across various workloads and platforms.

Applications such as cloud computing, big data processing, and artificial intelligence, demand high bandwidth and low latency in datacenter networks. Existing congestion control and flow control schemes at switches have limitations in granularity, fairness, and signal transmission delay. This paper proposes a fast notified per-hop per-flow backpressure congestion management, called BCN. BCN dynamically allocates queues for each flow at each switch hop, precisely pauses upstream queues based on queuing conditions, and enables non-paused queues to continue transmission. The switch sends network status notifications, collaborating with sender-side speed up and deceleration strategies to achieve rapid traffic rate adjustment. To the best of our knowledge, this is the first work focusing on fast rate adjustment of per-hop per-flow traffic. We evaluate BCN in real traffic scenarios and observe significant reductions in flow completion time of 53% and 60% compared to BFC and DCQCN, respectively. Under high workload conditions, BCN also improves network throughput. In high incast scenarios, BCN outperforms BFC and DCQCN, reducing flow completion time by 67.7% and 74.2%, respectively, while maintaining throughput comparable to BFC. Additionally, BCN minimizes transmission delay, ensuring nearly lossless transmission and optimizing overall throughput.

This paper presents a novel approach to optimize multiple invocations of a sparse matrix-vector multiplication (SpMV) kernel performed on the same sparse matrix A and dense vector x, like Ax, A2x, ⋯, Akx, and their linear combinations such as Ax + A2x. Such computations are frequently used in scientific applications for solving linear equations and in multi-grid methods. Existing SpMV optimization techniques typically focus on a single SpMV invocation and do not consider opportunities for optimization across a sequence of SpMV operations (SSpMV), leaving much room for performance improvement. Our work aims to bridge this performance gap. It achieve this by partitioning the sparse matrix into submatrices and devising a new computation pipeline that reduces memory access to the sparse matrix and exploits the data locality of the dense vector of SpMV. Additionally, we demonstrate how our approach can be integrated with parallelization schemes to further improve performance. We evaluate our approach on four distinct multi-core systems, including three ARM and one Intel platform. Experimental results show that our techniques improve the standard implementation and the highly-optimized Intel math kernel library (MKL) by a large margin.

The routing algorithm is crucial for network performance in network-on-chips (NoCs). With emerging applications bring new features to NoCs with more complex and time-varying traffic, which turns the routing computation process into multi-objective optimization. However, We found that the existing routing algorithms cannot effectively achieve load-balanced between different traffic due to the static method of routing design. The routing algorithm design space will increase if all factors affecting routing are considered. Reinforcement learning (RL) methods have demonstrated promising opportunities applied to architecture design exploration. In this paper, we proposed a novel RL framework for adaptive routing design in NoCs. This method uses network information to select the best path to achieve load balance and lower communication latency at the same time. Unfortunately, with this method, the implementation overhead of RL increases rapidly as the network scale increases. Therefore, we introduce a linear function of approximate RL-based adaptive routing (LARE) to reduce implementation over-head. We conduct extensive experiments against state-of-the-art routing algorithms to evaluate our design. Simulation results demonstrate the benefits of our design under synthetic traffic workloads and real applications. In addition, LARE can achieve similar network performance with traditional RL implementation with a much lower hardware overhead.

Code comment, i.e., the natural language text to describe the semantic of a code snippet, is an important way for developers to comprehend the code. Recently, a number of approaches have been proposed to automatically generate the comment given a code snippet, aiming at facilitating the comprehension activities of developers. Despite that state-of-the-art approaches have already utilized advanced machine learning techniques such as the Transformer model, they often ignore critical information of the source code, leading to the inaccuracy of the generated summarization. In this paper, to boost the effectiveness of code summarization, we propose a two-stage paradigm, where in the first stage, we train an off-the-shelf model and then identify its focuses when generating the initial summarization, through a model interpretation approach, and in the second stage, we reinforce the model to generate more qualified summarization based on the source code and its focuses. Our intuition is that in such a manner the model could learn to identify what critical information in the code has been captured and what has been missed in its initial summarization, and thus revise its initial summarization accordingly, just like how a human student learns to write high-quality summarization for a natural language text. Extensive experiments on two large-scale datasets show that our approach can boost the effectiveness of five state-of-the-art code summarization approaches significantly. Specifically, for the well-known code summarizer, DeepCom, utilizing our two-stage paradigm can increase its BLEU-4 values by around 30% and 25% on the two datasets, respectively.

Convolution kernels are widely seen in deep learning workloads and are often responsible for performance bottlenecks. Recent research has demonstrated that a direct convolution approach can outperform the traditional convolution implementation based on tensor-to-matrix conversions. However, existing approaches for direct convolution still have room for performance improvement. We present nDirect, a new direct convolution approach that targets ARM-based multi-core CPUs commonly found in smartphones and HPC systems. nDirect is designed to be compatible with the data layout formats used by mainstream deep learning frameworks but offers new optimizations for the computational kernel, data packing, and parallelization. We evaluate nDirect by applying it to representative convolution kernels and demonstrating its performance on four distinct ARM multi-core CPU platforms. We compare nDirect against state-of-the-art convolution optimization techniques. Experimental results show that nDirect gives the best overall performance across evaluation scenarios and platforms.

Intensive communication and synchronization cost for gradients and parameters is the well-known bottleneck of distributed deep learning training. Based on the observations that Synchronous SGD (SSGD) obtains good convergence accuracy while asynchronous SGD (ASGD) delivers a faster raw training speed, we propose Several Steps Delay SGD (SSD-SGD) to combine their merits, aiming at tackling the communication bottleneck via communication sparsification. SSD-SGD explores both global synchronous updates in the parameter servers and asynchronous local updates in the workers in each periodic iteration. The periodic and flexible synchronization makes SSD-SGD achieve good convergence accuracy and fast training speed. To the best of our knowledge, we strike the new balance between synchronization quality and communication sparsification, and improve the tradeoff between accuracy and training speed. Specifically, the core components of SSD-SGD include proper warm-up stage, steps delay stage, and the novel algorithm of global gradient for local update (GLU). GLU is critical for local update operations by using global gradient information to effectively compensate for the delayed local weights. Furthermore, we implement SSD-SGD on MXNet framework and comprehensively evaluate its performance with CIFAR-10 and ImageNet datasets. Experimental results show that SSD-SGD can accelerate distributed training speed under different experimental configurations, by up to 110% (or 2.1× of the original speed), while achieving good convergence accuracy.

Deep learning's widespread adoption in various fields has made distributed training across multiple computing nodes essential. However, frequent communication between nodes can significantly slow down training speed, creating a bottleneck in distributed training. To address this issue, researchers are focusing on communication optimization algorithms for distributed deep learning systems. In this paper, we propose a standard that systematically classifies all communication optimization algorithms based on mathematical modeling, which is not achieved by existing surveys in the field. We categorize existing works into four categories based on the optimization strategies of communication: communication masking, communication compression, communication frequency reduction, and hybrid optimization. Finally, we discuss potential future challenges and research directions in the field of communication optimization algorithms for distributed deep learning systems.

The widespread deployment of Remote Direct Memory Access (RDMA) in datacenter networks increases the stringency for convergence speed when congestion occurs. Fast convergence significantly reduces buffer occupancy, which in turn lessens the probability of triggering Priority-based Flow Control (PFC). Besides, the propagation delay becomes shorter with rapidly growing link speed, which correspondingly makes the queueing delay a major part of end-to-end latency in datacenter networks. Fast convergence and low buffer occupancy become more essential for lowering queue delay and flow complete time. In this paper, we present ERA, an ecn-ratio-based congestion control scheme, which contributes to fast convergence for datacenter networks. ERA consists of two fundamental components: (i) an ECN-marking-ratio-based queue buffer occupancy estimating (QBOE) solution and (ii) a queue-building-rate driven rate adjustment (QDRA) mechanism to achieve fast convergence in several control periods. We conduct extensive experiments to evaluate the performance of ERA, and the results show that ERA greatly accelerates the convergence process compared to other solutions. ERA achieves low tail latency and low buffer occupancy simultaneously.

NVMeoF is the latest extension of NVMe for remote storage access which allows remote access to NVMe controllers through high-speed RDMA, FC, and TCP networks. NVMe over TCP (NoT) can build on the basis of large-scale common network infrastructure in datacenters and standard TCP/IP software protocol stack, enabling a wide availability compared with RDMA-enabled specific network infrastructure for NVMe-overRDMA. However, the processing of read/write I/O at the host and target prominently shows significantly different characteristics and requirements, where one side sends the NVMeoF instruction of the request, while the other side sends the requested data. The existing NoT implementation can not meet the different characteristics of requests in the datacenter, which eventually results in the I/O performance being limited by the common processing pipeline and sending strategy. In this paper, we propose RNoT, a transformable queue that can meet the differentiated processing scheme of read or write request characteristics respectively in NoT implementation. Specifically, RNoT defines a switchable working attribute and separates resources for read and write I/O to achieve intra-queue long-term exclusivity, delivers read and write requests into other RNoT queue pairs to achieve inter-queue I/O scheduling, and transfers request command and data with targeted approaches to achieve short and long flow optimization. We implemented RNoT in Linux Kernel and evaluated it using realistic benchmarks and applications. Our experimental results demonstrate that RNoT can achieve 30.39% and 29.27% lower latency than i10 and NoT respectively, increase IOPS by up to 41.34% than NoT on average, thus RNoT can effectively optimize the read and write I/O performance in NoT with dedicated processing scheme.

As intelligent transportation systems (ITS) are now being integrated into our everyday lives, it has been widely accepted that forecasting road networks is a promising killer engine for ITS with high social and economic benefits. However, current solutions ignore the heterogeneity of spatial-temporal traffic data and fail to capture hidden spatial-temporal correlations. This paper presents STEGNN: a novel spatial-temporal embedding graph neural network for road network forecasting. The key idea of STEGNN is utilizing Cosine Similarity to generate a high-quality temporal graph and thus fills the gap between the temporal-spatial correlations for traffic graph, which includes (i) a novel approach to construct temporal graph based on temporal-spatial similarity from traffic graphs, which is much more accurate on measured similarity of time series claimed by previous methods; (ii) an advanced spatial-temporal embedding model to exploit spatial-temporal dependencies by leveraging specific arrangements of temporal and spatial graphs; and (iii) an effective framework that gasps extensive spatial-temporal dependencies in the long-term by mixing multi-layer graph convolution with dilated convolution to understand wide-range spatial-temporal features. Extensive evaluations validate STEGNN by applying it to real-world traffic graphs and indicate that STEGNN outperforms state-of-the-art solutions with much more accurate forecasting of road networks.

Congestion control is crucial for the overall performance of data center networks and still faces considerable challenges. Recently, credit-driven congestion control has been emerging to enable precise flow control for current high-speed and highly dynamic data centers. However, existing credit-driven methods essentially separate credit and data packets, i.e., credits can fully regulate data packets, but they receive little feedback from the data packets. Accordingly, these approaches inevitably struggle with lossy credits and impaired throughput. To address the issue, we present data-credit-coupling congestion control named DC4. For a better understanding of the relationship between data and credit, we revisit the principle of credit-based congestion control and make the first attempt to explore the art of presenting the data-credit plane architecture. Based on the proposed data-credit framework, DC4 transforms the interaction between credit and data packets from one-way control to two-way coordination to achieve mutual benefits and dynamic balances between the credit and data packets. We conduct extensive experiments to evaluate the performance of our design and compare it with state-of-the-art protocols, including HPCC, ExpressPass, and Aeolus. Experimental results show that DC4 outperforms data-credit-separated approaches in terms of the flow completion time, throughput, and credit waste.

The Message Passing Interface (MPI) is the most prominent and dominant programming model for scientific computing in super-computing systems today. Although many general and efficient algorithms have been proposed for MPI collective operations, there is still room for topology-aware optimization. Dragonfly is a high-scalability, low-diameter, and cost-efficient network topology adopted in more and more supercomputing networks. However, Dragonfly topology limits the performance of some MPI collective operations. In this paper, our analysis shows that the bottlenecks of collective algorithms in Dragonfly topology are intra-job interference, inter-job interference, and topology mismatch. We propose 5 different optimizations, i.e., Pseudo-random Pairwise, Tree-based Shuffle, Reversed Recursive Doubling, Reordered Bruck, and Matched Rabenseifner, for MPI collective operations including All-Gather, All-to-All, All-Reduce, and Reduce-Scatter. We evaluate each optimization through CODES network simulation framework with minimal, non-minimal, and adaptive routing. The simulation results demonstrate that the performance of All-to-All, All-Gather, All-Reduce, and Reduce-Scatter can be improved by 4.7X, 3.4X, 12.7%, and 4.1X, respectively, for 32768-node jobs with adaptive routing.

Understanding the I/O characteristics of applications on supercomputers is crucial to paving the path for application optimization and system resource allocation. We collect and analyze I/O traces of applications on a production supercomputer and reconfirm that I/O bursts exist in most applications. What's more, we find that the I/O bursts not only occur in short periods of time but also originate from a minority of adjacent compute nodes allocated to the applications, which we call spatiotemporal I/O burstiness. The concentration of I/O traffic in both time and space dimension will make applications experience poor I/O performance and incur I/O inefficiency of the storage system. Although there are some solutions, such as burst buffer, can help alleviate such inefficiency, there is still no work that measures, analyzes and further predicts the application I/O characteristic in terms of spatiotemporal burstiness, which we think is vital for application-aware optimizations, including but not limited to burst buffer allocation and job scheduling. In this paper, we first propose a mathematical model to measure the spatiotemporal I/O burstiness. Then a thorough analysis on the spatiotemporal I/O characteristic of all applications on the system is elaborated. We further make use of the job's submitting path to explore the I/O characteristic similarity among jobs, based on which a machine learning classification algorithm is proposed to accurately predict the job spatiotemporal I/O burstiness in advance. With accurate job I/O characteristic at hand, some useful suggestions are put forward to hedge the impacts of the spatiotemporal I/O burstiness.

The widespread deployment of Remote Direct Memory Access (RDMA) in datacenter networks increases the stringency for convergence speed when congestion occurs. Fast convergence significantly reduces buffer occupancy, which in turn lessens the probability of triggering Priority-based Flow Control (PFC). Besides, the propagation delay becomes shorter with rapidly growing link speed, which correspondingly makes the queueing delay a major part of end-to-end latency. Fast convergence and low buffer occupancy become more essential for lowering queue delay and flow complete time. We present DQCC (Double-Q Congestion Control), a fast-converging congestion control scheme, which consists of two fundamental components: (i) an ECN-marking-ratio-based queue buffer occupancy estimating (QBOE) solution and (ii) a queue-building-rate driven rate adjustment (QDRA) mechanism to achieve fast convergence. We conduct extensive experiments to evaluate the performance of DQCC, and the results show that DQCC greatly accelerates the convergence process. DQCC achieves low tail latency and low buffer occupancy simultaneously.

Code comment, i.e., the natural language text to describe code, is considered as a killer for program comprehension. Current literature approaches mainly focus on comment generation or comment update, and thus fall short on explaining which part of the code leads to a specific content in the comment. In this paper, we propose that addressing such a challenge can better facilitate code understanding. We propose Fosterer, which can build fine-grained semantic interactions between code statements and comment tokens. It not only leverages the advanced deep learning techniques like cross-modal learning and contrastive learning, but also borrows the weapon of pre-trained vision models. Specifically, it mimics the comprehension practice of developers, treating code statements as image patches and comments as texts, and uses contrastive learning to match the semantically-related part between the visual and textual information. Experiments on a large-scale manually-labelled dataset show that our approach can achieve an F1-score around 80%, and such a performance exceeds a heuristic-based baseline to a large extent. We also find that Fosterer can work with a high efficiency, i.e., it only needs 1.5 seconds for inferring the results for a code-comment pair. Furthermore, a user study demonstrates its usability: for 65% cases, its prediction results are considered as useful for improving code understanding. Therefore, our research sheds light on a promising direction for program comprehension.

As an important pipeline stage in the router of Network-on-Chips, switch allocation assigns output ports to input ports and allows flits to transit through the switch without conflicts. Previous work designed efficient switch allocation strategies by maximizing the matching efficiency in time series. However, those works neglected the interaction between different router pipeline stages. In this article, we propose the concept of Utility-of-Allocation (UoA) to indicate the quality of allocation to be practically used in on-chip routers. We demonstrate that router pipelines can interact with each other, and the UoA can be maximized if the interaction between router pipelines is taken into consideration. Based on these observations, a novel class of routers, MUA-Router, is proposed to maximize the UoA through the collaborative design (co-design) between router pipelines. MUA-Router achieves this goal in two ways and accordingly implements two novel instance router architectures. In the first, MUA-Router improves the UoA by mitigating the impact of endpoint congestion in the switch allocation, and thus Eca-Router is proposed. Eca-Router achieves an endpoint-congestion-aware switch allocation through the co-design between routing computation and switch allocation. Based on Eca-Router, CoD-Router is proposed to feed back switch allocation information to routing computation stage to provide switch allocator with more conflict-free requests. Through the co-design between pipelines, MUA-Router significantly improves the efficiency of switch allocation and the performance of the entire network. Evaluation results show that our design can achieve significant performance improvement with moderate overheads.

Hierarchical high-radix router microarchitecture consisting of small SRAM-based intermediate buffers has been used in large-scale supercomputers interconnection networks. While hierarchical organization enables efficient scaling to higher switch port count, it requires intermediate buffers which can cause performance bottleneck. Shallow intermediate buffers can cause head-of-line blocking to create backpressure towards input buffers and reduce overall performance. Increasing intermediate buffer size overcomes this problem but becomes infeasible due to the large overhead. In this work, we propose to organise decentralized intermediate buffers as a centralized buffer and leverage alternate memory technology to increase its capacity. In particular, we exploit the high-density nature of Spin-Torque Transfer Magnetic RAM (STT-MRAM) to increase intermediate buffer depth while also providing near-zero leakage power. STT-MRAM has disadvantages such as higher write latency and higher write energy. To overcome these disadvantages, we propose DeepHiR, a novel deep hybrid buffer organization (STT-MRAM and SRAM) combined with a centralized buffer organization to provide high performance with minimal cost. Although the deep intermediate buffer provided by DeepHiR can effectively improve router performance, a large amount of input buffer will still cause a lot of hardware overhead. At the same time, deeper intermediate buffers also makes it take longer for the backpressure to propagate to the source node, thereby reducing the performance of DeepHiR. Therefore, we further propose ElasHiR, which leverages elastic input buffer design in the centralized row buffer to allow a part of the centralized row buffer to act as input buffer. ElasHiR adopts reduced input buffers and automatically determines the length of input buffer in the centralized row buffer. This design minimizes the buffer resource while achieving excellent efficiency. Evaluation results show that DeepHiR can achieve 56.7 percent performance improvement in packet latency under synthetic traffic, and the cost of energy and area is moderate. ElasHiR can reduce the input buffer by 93.8 percent with performance comparable to DeepHiR.

Interconnection networks play an essential role in the architecture of high-performance computing (HPC) systems. In this article, we explore the Galaxyfly family to build flexible-scale interconnection networks. Galaxyfly is guaranteed to retain a small constant diameter while achieving a flexible tradeoff between network scale and bisection bandwidth. Galaxyfly not only supports small-scale interconnection networks with smaller diameter but also lowers the demands for high-radix routers and is able to utilize routers with moderate radix to build exascale interconnection networks. We analyze the constructible configuration of Galaxyfly and evaluate the properties of Galaxyfly. We conduct extensive simulations and analysis to evaluate the performance, cost, and power consumption of Galaxyfly on physical layout against state-of-the-art topologies. The results show that our design achieves better performance than most existing topologies under typical HPC workloads, and is cost-effective to deploy for exascale HPC systems.

In this paper, we propose some efficient data redistribution algorithms for redistributing matrices from 1D or 2D irregular format to block cyclic data distribution (BCDD) format, which can be much faster than the BLACS routinePXGEMR2D. These algorithms can be used to combine direct methods with iterative methods. The proposed algorithms divide the communication into two phases: one for processes in the same column and the other for processes in the same row, and the whole data redistribution task is divided into several independent sub-communications. The communication time can be reduced a lot compared with BLACS. Performance results show that our algorithms can be$2\times$–$5\times$faster than the BLACS routinePXGEMR2Dwhen using 4096 processes and the experiments are performed on Tianhe-2A supercomputer.

Adaptive routing is crucial to the overall performance of network-on-chips (NoCs), and still faces great challenges, especially when emerging applications on many-core architecture exhibit complicated and time-varying traffic patterns. When witnessing most existing heuristic adaptive routing algorithms fail to address multi-objective optimization for complex traffic well, we make the first attempt to propose a novel and comprehensive reinforcement learning framework for adaptive routing on NoCs, called RELAR. RELAR is suitable for diversified traffic patterns and resolve multi-objective optimization simultaneously. We conduct experiments against state-of-the-art routing algorithms to evaluate our design. The results show that RELAR achieves 14.82% reduction in packet latency on average, and reduces packet latency by up to 34.24% under heavy synthetic traffic workload.

Although using Multi-Rail network is a popular choice for many HPC systems to overcome bandwidth limitations, the influence of connection modes between multi-port nodes and switching network is not well understood so far. This work provides a detailed analysis of different node connection modes in Multi-Rail Fat-tree. To gain a deep understanding of relevant issues, we propose a new parametric family of Multi-Rail Fat-tree called MR-tree, which could cover all balanced node connection modes for Multi-Rail Fat-tree. We evaluate different node connection modes by theoretical analysis and flit level simulation. The results leave open the possibility that optimization of node connection mode can yield better results for Multi-Rail Fat-tree.

Congestion control is crucial to the overall performance of datacenter networks and still faces great challenges, especially when network traffic exhibits complicated and time-varying patterns, from long-running flows to burst short-lived flows. Recently congestion control techniques based on in-network-telemetry (INT) are emerging as the promising approach to enable precise flow control. Existing INT-based method mainly relies on receiver ACK packets to transfer INT data and acquire link load information. Their control precision in term of response time, however, is still beyond one round-trip time (RTT), since the receiver ACK feedback needs at least one RTT and the sender cannot obtain INT data of one flow within the first RTT. In this paper, we make the first attempt to explore INT-based techniques and break the one-RTT barrier efficiently. We come up with an ultra-precise and efficient congestion control algorithm, called UECC. UECC utilizes both switch-based and host-based INT simultaneously, and realizes accurate, timely and low-overhead flow control. We tackle the challenging issues of cooperation between the two types of INT and the additional overhead introduced by confirmation packets, and conduct extensive experiments to evaluate the performance of our design. The results show that UECC achieves 42% reduction in average queue length compared to the state-of-the-art INT-based method, and 5% reduction in 95th-percentile flow completion time.

Communication overhead is the key challenge for distributed training. Gradient compression is a widely used approach to reduce communication traffic. When combining with a parallel communication mechanism method like pipeline, gradient compression technique can greatly alleviate the impact of communication overhead. However, there exist two problems of gradient compression technique to be solved. Firstly, gradient compression brings in extra computation cost, which will delay the next training iteration. Secondly, gradient compression usually leads to a decrease in convergence accuracy. In this paper, we combine parallel mechanism with gradient quantization and delayed full-gradient compensation, and propose a new distributed optimization method named CD-SGD, which can hide the overhead of gradient compression, overlap part of the communication and obtain high convergence accuracy. The local update operation in CD-SGD allows the next iteration to be launched quickly without waiting for the completion of gradient compression and the current communication process. Besides, the accuracy loss caused by gradient compression is solved by k-step correction method introduced in CD-SGD. We prove that CD-SGD has convergence guarantee and it achieves at least convergence rate. We conduct extensive experiments on MXNet to verify the convergence properties and scaling performance of CD-SGD. Experimental results on a 16-GPU cluster show that convergence accuracy of CD-SGD is close to or even slightly better than that of S-SGD, and its end-to-end time is 30 less than 2-bit gradient compression under a 56Gbps bandwidth environment.

Although using Multi-Rail network is a popular choice for many HPC systems to overcome bandwidth limitations, the influence of connection modes between multi-port nodes and switching network is not well understood so far. This work provides a detailed analysis of different node connection modes in Multi-Rail Fat-tree. To gain a deep understanding of relevant issues, we propose a new parametric family of Multi-Rail Fat-tree called MR-tree, which could cover all balanced node connection modes for Multi-Rail Fat-tree. We evaluate different node connection modes by theoretical analysis and flit level simulation. We show that there are great differences across various node connection modes. Among the key differences are the following: cost, average shortest path length, fault-tolerance, and path diversity. In addition, our simulation results reveal that the performance gap for Multi-Rail Fat-trees with different node connection modes may vary significantly depending on the oversubscribed conditions. The above results leave open the possibility that optimization of node connection mode can yield better results for Multi-Rail Fat-tree.

General Matrix Multiplication (GEMM) is a key subroutine in high-performance computing. There is a large body of work on evaluating and optimizing large-scale matrix multiplication, but how well the small-scale matrix multiplication (SMM) performs is largely unknown, especially for the ARMv8-based many-core architectures. In this work, we evaluate and characterize the performance of SMM subroutines on Phytium 2000 +, an ARMv8-based 64-core architecture. The evaluation work is extensively performed with the mainstream open-source libraries including OpenBLAS, BLIS, BALSFEO, and Eigen. Given various experimental settings, we observe how well the small-scale GEMM routines perform on Phytium 2000 +, and then discuss the impacting factors behind the performance behaviours of SMM. Built on such a basis, we shed light on the performance bottlenecks and practical optimizations on SMM from various angles: (1) mitigating the data packing overhead, (2) processing the edge cases properly, (3) selecting a suitable micro-kernel, and (4) adopting a right parallelization method. The result of our work facilitates users to develop efficient SMM optimizations on ARMv8-based many-core architectures, and embed them into real-world applications.

Large-scale deep neural networks training have been widely deployed on dense-GPU public cloud clusters. Intensive communication and synchronization cost for gradients and parameters is becoming the bottleneck of distributed deep learning training. Horovod is one of the most popular distributed communication frameworks to address the scale-out issue of deep learning training on GPU clusters. Existing public-cloud GPU datacenters, such as Amazon EC2 and Alibaba GPU cloud, are usually equipped with commodity high-speed Ethernet and TCP networking. In current vanilla Horovod, however, we observe that one GPU device is merely associated with at most one proxy communication process. The proxy process is responsible for dealing with all the communication operations of parameter all-reduce for one or multiple GPUs. Such configuration makes communication interface based on TCP protocols suffer from limited network goodput and incur training performance penalties. In this paper, we make the first attempt to improve the message passing interface of Horovod and address the mismatching between the computation and communication capability when deploying Horovod in TCP-based public-cloud GPU clusters. We propose FastHorovod to exploit more cost-efficient auxiliary communication processes on CPU to expedite parallel message-passing schedule for GPU. We conduct extensive experiments against state-of-the-art Horovod. The experiment results show that our design can significantly accelerate the distributed training communication on TCP-based public-cloud GPU clusters, and FastHorovod improves the training speed of AlexNet and VGG16 models by 64.5% and 72.6% respectively.

General Matrix Multiplication (GEMM) is a key subroutine in highperformance computing. While the mainstream linear algebra libraries can deliver high performance on large and regular-shaped GEMM, they are inadequate for optimizing small and irregular-shaped GEMMs, which are commonly seen in new HPC applications. Some of the recent works in this direction have made promising progress on x86 architectures and GPUs but still leave much room for improvement on emerging HPC hardware built upon the ARMv8 architecture. We present LibShalom, an open-source library for optimizing small and irregular-shaped GEMMs, explicitly targeting the ARMv8 architecture. LibShalom builds upon the classical Goto algorithm but tailors it to minimize the expensive memory accessing overhead for data packing and processing small matrices. It uses analytic methods to determine GEMM kernel optimization parameters, enhancing the computation and parallelization efficiency of the GEMM kernels. We evaluate LibShalom by applying it to three ARMv8 multi-core architectures and comparing it against five mainstream linear algebra libraries. Experimental results show that LibShalom can consistently outperform existing solutions across GEMM workloads and hardware architectures.

Hierarchical organization is widely used in high-radix routers to enable efficient scaling to higher switch port count. A general-purpose hierarchical router must be symmetrically designed with the same input buffer depth, resulting in a large amount of unused input buffers due to the different link lengths. Sharing input buffers between different input ports can improve buffer utilization, but the implementation overhead also increases with the number of shared ports. Previous work allowed input buffers to be shared among all router ports, which maximizes the buffer utilization but also introduces higher implementation complexity. Moreover, such design can impair performance when faced with long packets, due to the head-of-line blocking in intermediate buffers. In this work, we explain that sharing unused buffers between a subset of router ports is a more efficient design. Based on this observation, we propose Centralized Input Buffer Design in Hierarchical High-radix Routers (CIB-HIER), a novel centralized input buffer design for hierarchical high-radix routers. CIB-HIER integrates multiple input ports onto a single tile and organizes all unused input buffers in the tile as a centralized input buffer. CIB-HIER only allows the centralized input buffer to be shared between ports on the same tile, without introducing additional intermediate virtual channels or global scheduling circuits. Going beyond the basic design of CIB-HIER, the centralized input buffer can be used to relieve the head-of-line blocking caused by shallow intermediate buffers, by stashing long packets in the centralized input buffer. Experimental results show that CIB-HIER is highly effective and can significantly increase the throughput of high-radix routers.

Proactive transports nowadays have drawn much attention because of fast convergence, near-zero queueing and low latency. Proactive protocols, however, need an extra RTT to allocate ideal sending rate for new flows. To solve this, some studies, such as pHost, Homa, send unscheduled packets with line rate in the first RTT, which will causes severe network congestion. To avoid the queue buildup, Aeolus directly drops unscheduled packets when congestion occurs. Nevertheless, based on our experiment, a considerable part of small flows (0-100 KB) will be completed in the first RTT under 100 Gbps network, so that dropping unscheduled packets will severely affect performance of the small flows. In this paper we propose SSP, a new scheme aimed to eliminate the extra RTT delay and improve the flow completion time (FCT) of small flows under the proactive mechanism. Like pHost and Homa, SSP sends unscheduled packets at line rate when new flow arrives. Different from Aeolus, SSP selectively drops scheduled packets once queue buildup happens in the switch, thus protecting unscheduled packets which are more likely belong to small flows. Besides, based on the short-job-first (SJF) principle, we give relative higher priorities for small flows at the sender. Our simulation results with realistic workloads show that SSP can improve the FCT of small flows significantly. Specifically, under Web Search workload, SSP facilitates nearly 63% of 0-100 KB flows to complete one RTT faster. Also, SSP reduces the tail FCT by 56.8% at the 99th percentile compared with Expresspass and 29.2% compared with Aeolus while not leads to large queue buildup.

Recent proposals have leveraged emerging credit-based proactive transports to achieve high throughput low latency datacenter network transports. Particularly, those transports that employ hop-by-hop credits have the merits of fast convergence, low buffer occupancy, and strong congestion avoidability. However, they fairly transmit long flows and latency-sensitive short flows, which will cause the transmission latency of short flows and the average flow completion time increased. Although flow scheduling mechanisms have studied extensively to accelerate short flow transmission, they are hard to be directly applied in credit-based transports. The root cause is that most traditional flow scheduling mechanisms mainly work in the long queue containing flows in various sizes, while credit-based proactive transports maintain the extremely short bounded queue, near zero. Based on this observation, this paper makes the first attempt to accelerate short-flow scheduling in credit-based proactive transport, and proposed FastCredit. FastCredit can be used as a general building block to expedite short flows in credit-based proactive transports. In FastCredit, we schedule credit transmission at both receivers and switches to indirectly perform flow scheduling, and develop a mechanism to mitigate credit waste and improve network goodput. Compared to the state-of-the-art credit-based transport protocol, FastCredit reduces average flow completion time to 0.78x and greatly improves the short flow transmission latency to 0.51x in realistic workloads. Especially, FastCredit reduces average flow completion time to 0.76x under incast circumstances and 0.62x in many-to-one traffic mode. Furthermore, FastCredit still maintains the advantages of short queue and high throughput.

High-performance computing (HPC) systems keep increasing in size and bandwidth, thus requiring larger and higher-bandwidth interconnection networks. The race to exascale just exacerbated this trend. The resulting longer average distance and more links between modules makes the use of optical fiber mandatory. However, the system meets the challenge of cable packaging complexity, cable tolerance, and cable maintainability. Splitter cable, like multi-core fiber (MCF), is a new and cost-effective approach that has the potential to replace a bundle of fibers between any pairs of modules with a single cable, thus lowering the packaging complexity and enhancing the maintainability. To the best of our knowledge, we are the first to formally study the problem of building a cost-effective HPC network topology using multicore fiber. In this paper, a new diameter-3 topology is proposed, namely Bundlefly. It achieves a flexible tradeoff between intra-module radixes and inter-module radixes of routers with merely moderate radix to build a diameter-3 exascale interconnection network. It is suitable for the use of multi-core fiber for the requirement of inter-module bandwidth and cable packaging complexity. We analyze the properties of Bundlefly and present effective routing algorithms. We simulate and analyze the performance of Bundlefly against state-of-the-art topologies. The results show that Bundlefly with flexible configurations can achieve better performance than most existing topologies.

The training of modern deep learning neural network calls for large amounts of computation, which is often provided by GPUs or other specific accelerators. To scale out to achieve faster training speed, two update algorithms are mainly applied in the distributed training process, i.e., the Synchronous SGD algorithm (SSGD) and Asynchronous SGD algorithm (ASGD). SSGD obtains good convergence point while the training speed is slowed down by the synchronous barrier. ASGD has faster training speed but the convergence point is lower when compared to SSGD. To sufficiently utilize the advantages of SSGD and ASGD, we propose a novel technology named One-step Delay SGD (OD-SGD) to combine their strengths in the training process. Therefore, we can achieve similar convergence point and training speed as SSGD and ASGD separately. To the best of our knowledge, we make the first attempt to combine the features of SSGD and ASGD to improve distributed training performance. Each iteration of OD-SGD contains a global update in the parameter server node and local updates in the worker nodes, the local update is introduced to update and compensate the delayed local weights. We evaluate our proposed algorithm on MNIST, CIFAR-10, and ImageNet datasets. Experimental results show that OD-SGD can obtain similar or even slightly better accuracy than SSGD, while its training speed is much faster, which even exceeds the training speed of ASGD.

Understanding the I/O characteristics of supercomputers is crucial for grasping accurate I/O workloads and uncovering potential I/O inefficiency. We collect and analyze I/O traces from two production supercomputers, and find that the I/O traffic peaks in the system not only occur in short periods of time but also originate from a minority of adjacent compute nodes, which we call spatially bursty I/O. Since modern supercomputers widely adopt I/O forwarding architecture, in which an I/O node performs I/O on behalf of a subset of compute nodes in the vicinity, spatially bursty I/O will cause significant load imbalance and underutilization on the I/O nodes. To address such problems, we quantitatively analyze the two causes of spatially bursty I/O, including uneven I/O distribution on job's processes and uneven job nodes distribution on the system. Two different solutions are proposed to mobilize more I/O nodes to participate in job's I/O activity. (1) We change the I/O node mapping, making adjacent compute nodes use different I/O nodes instead of a same one. (2) According to the job's I/O characteristics extracted from history I/O traces, we distribute the compute nodes of data-intensive jobs more sparsely to utilize more I/O nodes. Extensive evaluations of both solutions show that they can further exploit the potential of I/O forwarding layer. We have deployed the proposed I/O node mapping on a production supercomputer for 11 months. Our experience finds that it can effectively promote I/O performance, balance loads, and alleviate I/O interference.

Bursty traffic and thousands of concurrent flows incur inevitable congestion in data center networks (DCNs) and then affect the overall performance. Various transport protocols are developed to mitigate the network congestion, including reactive and proactive protocols. Reactive schemes to handling congestion after congestion arises are common to current DCNs. However, with the growth of scale and link speed, reactive schemes such as DCTCP encounter the significant problem of slow responding to congestion. On the contrary, proactive protocols are designed to avoid congestion, and they have the advantages of zero data loss, fast convergence and low buffer occupancy (e.g., credit-reservation protocols). But in actual deployment scenario, it is hard to guarantee one protocol to be deployed in every server at one time. When credit-reservation protocol is deployed to DCNs step-by-step, the network is converted to multi-protocol state and faces the following fundamental challenges: (i) unfairness, (ii) high bu er occupancy, and (iii) heavy tail delay. Therefore, we propose EC4, which is for converging ECN-based and credit-reservation protocols with minimal modification. To the best of our knowledge, EC4is the first to address how to harmonize proactive and reactive congestion control. Targeting the common ECN-based protocol-DCTCP, EC4leverages the Forward Explicit Congestion Notification (FECN) to deliver realtime congestion information and redefines feedback control. After evaluation, the results show that EC4e ectively addresses the unfair link allocation. Furthermore, even workloads at 0.6 does not cause buffer overflow, thus largely eliminating the timeouts problem.

Endpoint congestion is a bottleneck in high-performance computing (HPC) networks and severely impacts system performance, especially for latency-sensitive applications. For long messages (or flows) whose duration is far larger than the round-trip time (RTT), endpoint congestion can be effectively mitigated by proactive or reactive counter-measures such that the injection rate of each source is dynamically controlled to a proper level. However, many HPC applications produce a hybrid traffic, a mix of short and long messages, and are dominated by short messages. Existing proactive congestion avoidance methods face the great challenge of scheduling the rapidly changing traffic pattern caused by these short messages. In this paper, we leverage the advantages of proactive and reactive congestion avoidance techniques and propose the Packet-chaining Reservation Protocol (PCRP) to make a dynamic balance between flows following proactive scheduling and packets subjected to reactive network conditions. We select the chaining packets as a flexible reservation granularity between the whole flow and one packet. We allow small flows to be speculatively transmitted without being discarded and give them higher priority over the entire network. Our PCRP can respond quickly to network conditions and effectively avoid the formation of endpoint congestion and reduce the average flow delay. We conduct extensive experiments to evaluate our PCRP and compare it with the state-of-the-art proactive reservation-based protocols, Speculative Reservation Protocol (SRP) and Bilateral Flow Reservation Protocol (BFRP). The simulation results demonstrate that in our design the flow latency can be reduced by 50.2% for hotspot traffic and 28.38% for uniform traffic.

Hierarchical high-radix router microarchitecture consisting of small SRAM-based intermediate buffers have been used in large-scale supercomputers interconnection networks. While hierarchical organization enables efficient scaling to higher switch port count, it requires intermediate buffers that can cause performance bottleneck. Shallow intermediate buffers can cause head-of-line blocking and result in backpressure towards the input buffers to reduce overall performance. Increasing intermediate buffer size overcomes this problem but is infeasible since the amount of intermediate buffer is proportional to O(p2) where p is the router radix. Adopting new memory technology with higher density can increase intermediate buffer size but is not practical in decentralized, small-size intermediate buffers.

With the fast development of deep learning (DL), the communication is increasingly a bottleneck for distributed workloads, and a series of optimization works have been done to scale out successfully. Nevertheless, the network behavior has not been investigated much yet. We intend to analyze the network behavior and then carry out some research through network simulation. Under this circumstance, an accurate communication measurement is necessary, as it is an effective way to study the network behavior and the basis for accurate simulation. Therefore, we propose to capture the deep learning communication (DLC) trace to achieve the measurement. To the best of our knowledge, we make the first attempt to capture the communication trace for DL training. In this article, we first provide detailed analyses about the communication mechanism of MXNet, which is a representative framework for distributed DL. Secondly, we define the DLC trace format to describe and record the communication behaviors. Third, we present the implementation of method for trace capturing. Finally, we make some statistics and analyses about the distributed DL training, including communication pattern, overlap ratio between computation and communication, computation overhead, synchronization overhead, update overhead, and so forth. Both the statistics and analyses are based on the trace files captured in a cluster with six machines. On the one hand, our trace files provide a sketch on the DLC, which contributes to understanding the communication details. On the other hand, the captured trace files can be used for figuring out various overheads, as they record the communication behaviors of each node.

As the critical pipeline stage in on-chip routers, switch allocation assigns output ports to input ports and allow flits transiting through the switch without conflicts. Previous works strive to design efficient switch allocaiton strategies by maximizing the matching at each cycle, with the information from the current cycle or multiple cycles in time series. However, those works have not taken endpoint congestion into considerations. Tree-saturation, caused by endpoint congestion, can degrade NoC performance due to the congestion fanning out from the original point to upstream routers. In this paper, a novel router design, Eca-Router, is proposed to relieve the impact of endpoint congestion by switch allocation optimization. Eca-Router detects endpoint congestion by recording the destinations of packets in switch allocation. Endpoint congestion is decided in switch allocation once there are multiple input ports competing for the same output port and the packets in these input ports contain the same destination. During switch allocation, requests that contribute to endpoint congestion will be given lower priority to be allocated, and starvation control is also introduced to ensure allocation fairness. Evaluation results show that Eca-Router is efficient in reducing packet latency.

In HPC, endpoint congestion is a bottleneck in the network and seriously affects the performance of the system. The endpoint congestion can be effectively mitigated by quickly responding to the network and reducing the injection rate of the source. However, most of the prior works do not consider the impact of flow completion time on system performance, but only focus on the packet latency and perform scheduling at packet granularity. For HPC applications, the flow completion time and throughput are the metrics that determines the speed of application execution. Although prior works reduce package latency, they do not fundamentally reduce the flow latency in flow level. In this paper, we propose the bilateral flow-based reservation protocol (BFRP). BFRP quickly responds to network conditions through light-weight bilateral reservation mechanism and can effectively avoid the formation of endpoint congestion. BFRP also schedules packets based on flows, and smallest flow is preferentially sent to decrease the average flow latency. BFRP ensures the source and destination send or receive flows according to the allocated time slices without any conflict at both ends. We evaluate our BFRP protocol against state-of-the-art reservation-based protocol, speculative reservation protocol(SRP), and the simulation results show that the flow latency can be reduced by 27.68% under hotspot traffic with fixed flow size and 24.59% under uniform traffic with fixed flow size.

Traditional input-queued routers in network-on-chips (NoCs) only have a small number of virtual channels (VCs) and packets in a VC are organized in a fixed order. Such design is susceptible to head-of-line (HoL) blocking as only the packet at the head of a VC can be allocated by the switch allocator. Since switch allocation is the critical pipeline stage in on-chip routers, HoL blocking significantly degrades the performance of NoCs. In this paper, we propose to schedule packets in input buffers utilizing reorder buffer (RoB) techniques. We design VCs as RoBs to allow packets located not at the head of a VC to be allocated before the head packets. RoBs reduce the conflicts in switch allocation and mitigate the HoL blocking and thus improve the NoC performance. However, it is hard to reorder all the units in a VC due to circuit complexity and power overhead. We propose RoB-Router, which leverages elastic RoBs in VCs to only allow a part of a VC to act as RoB. RoB-Router automatically determines the length of RoB in a VC based on the number of buffered flits. This design minimizes the resource while achieving excellent efficiency. Furthermore, we propose two independent methods to improve the performance of RoB-Router. One is to optimize the packet order in input buffers by redesigning VC allocation strategy. The other combines RoB-Router with current most efficient switch allocator TS-Router. We perform evaluations and the results show that our design can achieve 46 and 15.7 percent performance improvement in packet latency under synthetic traffic and traces from PARSEC than TS-Router, and the cost of energy and area is moderate. Additionally, average packet latency reduction by our two improving methods under uniform traffic is 13 and 17 percent respectively.

As the development of processors/SoCs (Systemon-Chips), NoC (Network-on-Chip) consumes an increasing fraction of the modern processors/SoCs power. Thus, designing energy-efficient NoC architecture is imperative. Multi-NoC (Multiple Network-on-Chip) behaves well in power gating for reducing leakage power, which constitutes a significant fraction of NoC power. In this paper, we propose CRA, a novel Multi-NoC design with distinct routing algorithms for different subnets. Integrated with a congestion-aware power gating and packet scheduling policy, CRA is able to achieve low power without degrading performance at varying network utilization. Our experimental results show that CRA consumes an average of 15.58% less power than Catnap, the state of the art power efficient Multi-NoC design, and the EDP (energy delay product) is 8.59% lower than Catnap on average.

One-to-many group communication is a performance bottleneck for high-performance data center applications, due to sending massive data from one source to hundreds of receivers. The state-of-the-art solutions utilize either electrical packet switch (EPS) or optical circuit switch (OCS) multicast to accelerate massive data disseminations. However, there exist competitions between multicast and unicast flows at core EPSes in the electrical multicast. Moreover, the optical multicast suffers from a non-negligible reconfiguration delay and exclusive optical links. In this paper we present iCAST, a system for reducing multicast flow completion time (MFCT) on a generic hybrid EPS/OCS network, which has multiple EPSes and OCSes supporting multicast. iCAST constructs multicast trees by integrating the electrical and optical multicast to fully utilize network resources, and seamlessly schedules flows between the static electrical and dynamic optical networks to reduce the configuration overhead. We evaluate the performance by implementing a small-scale hybrid EPS/OCS testbed and extending the high-performance framework MPICH to support iCAST. Experiments show that iCAST outperforms one order of magnitude in reducing MFCT compared with the ring algorithm. We also develop an event-based flow level simulator to evaluate the performance of iCAST at the scale of thousands of servers. Simulation results show that iCAST reduces the average MFCT by 32% and 28% compared to OCS and EPS respectively, and significantly outperforms binomial tree and ring algorithm by up to 64% and 46% respectively.

Cost-effective adaptive routing has a significant impact on overall performance for high-radix hierarchical topologies, such as Dragonfly, which achieve a lower network diameter than traditional topologies, Torus and Fat tree, but exhibit a lower degree of adaptiveness for shortest-path rout- ing. Existing adaptive routing methods for those hierarchical topologies improve the adaptiveness by increasing path length, i.e. local or global adaptive routing, and thus suffer from complex and costly deadlock avoidance. This work aims to maximize the routing adaptiveness at the minimum cost of deadlock avoidance. We propose a label routing method for high-radix hierarchical networks. This label routing utilizes a co-design methodology and coordinates the two pipelines, input queue and routing computation, in the router microarchitec- ture. Packets in the input buffer are labeled by our routing algorithm depending on network states. We reorganize the input buffer and develop a label routing algorithm, named Green-Red Routing, GRR. GRR relaxes the requirement of using virtual channels to eliminate routing deadlock, and mitigates buffer resources dedicated to deadlock avoidance. GRR manages the buffer resources and balance its utilization elaborately, and achieve fully adaptive routing efficiently. We conduct extensive experiments to evaluate the performance of GRR on Dragonfly and compare it with state-of-the-art works. The results show that GRR achieves 10%-35% higher performance than existing routing algorithms under most traffic patterns.

High-radix routers with low latency and high bandwidth play an increasingly important role in the design of large-scale interconnection networks such as those used in super-computers and datacenters. The tile-based crossbar approach partitions a single large crossbar into many small tiles and can considerably reduce the complexity of arbitration while providing throughput higher than the conventional switch implementation. However, it is not scalable due to power consumption, placement, and routing problems. In this paper, we propose a truly scalable router microarchitecture called Multiport Binding Tile-based Router (MBTR). By aggregating multiple physical ports into a single tile a high-radix router can be flexibly organized into a different array of tiles, thus the number of tiles and hardware overhead can be considerably reduced. Compared with a hierarchical crossbar, MBTR achieves up to 50%~75% reduction in memory consumption as well as wire area. Simulation results demonstrate MBTR is indistinguishable from the YARC router in terms of throughput and delay, and can even outperform it by reducing potential contention for output ports. We have fabricated an ASIC MBTR chip with 28nm technology. Internally, it runs at 700MHz and 30ns latency without any speedup. We also discuss how the microarchitecture parameters of MBTR can be adjusted based on the power, area, and design complexity constraints of the arbitration logic.

There is a pressing need for high-radix routers in modern HPC (High Performance Computing) interconnects and to build the exascale computers with massive clusters. In this paper, we propose MBL, a high-radix router with a multi-stage bufferless switch Clos network inside. Booksim interconnection network simulator is used to implement our arbitrating designs for the architecture and it runs well under different traffic patterns in a flattened butterfly network, with 136 ports for each router.

Network-on-chip system plays an important role to improve the performance of chip multiprocessor systems. As the complexity of the network increases, congestion problem has become the major performance bottleneck and seriously influence the performance of NoCs. Prior works have focused on designing effective routing algorithm based on collecting contention and congestion information to load balance the traffic. However, most prior works do not consider balancing the traffic load during switch allocation. Due to the lack of congestion information in switch allocation stage, switch allocator performs allocation only based on packet requests and thus aggravates the congestion in the ports of switch. In this paper, we propose CCAS, a new switch allocation strategy to add the contention and congestion information into the switching process to load balance the traffic and achieve efficient switch allocation. We carefully design CCAS to balance the trade-off between traffic load balance and the matching efficiency in switch allocation. We evaluate our design under synthetic traffic and traces of PARSEC benchmarks. Our evaluations show that CCAS can achieve remarkable latency reduction compared to other switch allocation strategies.

Interconnection network plays an essential role in the architecture of large-scale high performance computing (HPC) systems. In the paper, we construct a novel family of low-diameter topologies, Galaxyfly, using techniques of algebraic graphs over finite fields. Galaxyfly is guaranteed to retain a small constant diameter while achieving a flexible tradeoff between network scale and bisection bandwidth. Galaxyfly lowers the demands for high radix of network routers and is able to utilize routers with merely moderate radix to build exascale interconnection networks. We present effective congestion-aware routing algorithms for Galaxyfly by exploring its algebraic property. We conduct extensive simulations and analysis to evaluate the performance, cost and power consumption of Galaxyfly against state-of-the-art topologies. The results show that our design achieves better performance than most existing topologies under various routing algorithms and traffic patterns, and is cost-effective to deploy for exascale HPC systems.

Multi-NoC (multiple network-on-chip) has demonstrated its advantages in power gating for reducing leakage power. This work presents Chameleon, a novel heterogeneous Multi-NoC design. Chameleon employs a fine-grained power gating algorithm which exploits power saving opportunities at different levels of granularity simultaneously. Integrated with a performance-aware traffic allocation policy, Chameleon is able to achieve both high power efficiency and good performance at varying network utilization. Our experimental results show that Chameleon delivers an average of 3.39% higher performance than Catnap, the best in the literature. More importantly, Chameleon consumes an average of 17.16% less power than Catnap.

In this paper, we propose FlyCast, an architecture using the physical layer of free-space optics (FSO) to accelerate multicast communication. FlyCast leverages off-the-shelf devices (e.g. switchable mirror, beam splitter) to physically split the FSO beam to multi receivers on demand, which enables to build dynamical multicast trees in physical layer and accelerates multicast communications. We demonstrate the feasibility of FlyCast through our theoretical analysis and the proof-of-concept prototype.

Greedy geographic routing is widely adopted in practical wireless networks due to its simplicity. However, greedy geographic routing alone cannot guarantee the delivery of packets due to the existence of local minima. A number of solutions have been proposed to address this issue, such as face routing, landmark-based routing, network segmentation or virtual coordinate based methods, etc. However, these solutions either have various limitations, such as depending on exact node locations, requiring costly preprocessing of the global network topology, or have obvious performance shortcomings, such as severe load-imbalance, large path stretch factors, etc. In this work, we attempt to combine the advantages of existing solutions, and present a hierarchical greedy geographic routing scheme. This design, FLYER, neither depends on exact node locations, nor needs to store any global state information in each node, which makes it applicable and scalable in large-scale practical systems. Moreover, our routing scheme is able to produce route paths with lower stretch factors and more load-balancing property than existing solutions. The algorithm works in a completely localized fashion, and the additional storage and computation complexity is extremely low. Extensive simulations are conducted, and the results demonstrate the superior performance of our approach against the state-of-the-art methods.

Wormhole attack is a severe threat to wireless ad hoc and sensor networks. Most of previous countermeasures either require specialized hardware devices or make strong assumptions on the network in order to capture the specific symptom induced by wormholes. Those requirements and assumptions limit the applicability of those approaches. Recently, some approaches based on topological or graph theoretical techniques are proposed to recognize wormholes using only connectivity information, shedding light on the challenging issue of connectivity-based wormhole detections. Unfortunately, those state-of-the-art connectivity-based countermeasures either present the principle of tracing wormholes in continuous domain, which makes it costly to transform them into protocols in discrete networks, or only explore localized (unstable) symptom of wormholes, accordingly incurring high false positive or negative rate. In this work, we make the first attempt towards establishing a graph theoretical method, called Worm Planar, that merely utilizes localized connectivity information and is able to capture the global essential symptoms of wormholes directly in the discrete networks. Worm Planar exploits location free network planarization technique to perform connectivity-based wormhole detection. Our new insights into the symptoms of wormholes make Worm Planar orthogonal to existing connectivity-based methods. We formally prove the correctness and evaluate the effectiveness of our approach through extensive simulations and comparisons with the state-of-the-art approaches. Simulation results demonstrate that Worm Planar is able to accurately identify and isolate wormholes for a large class of network instances.

This poster presents a hierarchical greedy geographic routing scheme in wireless networks, which performs greedy geographic routing with guaranteed delivery under uncertain locations on a landmark graph by leveraging a fine-grained connectivity-based planarization algorithm. This design neither depends on exact node locations, nor needs to store any global state information in each node. The algorithm works in a completely localized fashion, and the additional storage and computation complexity is extremely low. Our simulations demonstrate that the routing scheme is able to produce route paths with lower stretch factors and more load-balancing property than the state-of-the-art methods.

Extracting planar graph from network topologies is of great importance for efficient protocol design in wireless ad hoc and sensor networks. Previous techniques of planar topology extraction are often based on ideal assumptions, such as UDG communication model and accurate node location measurements. To make these protocols work effectively in practice, we need extract a planar topology in a location-free and distributed manner with small stretch factors. The planar topologies constructed by current location-free methods often have large stretch factors. In this paper, we present a fine-grained and location-free network planarization method under ρ-quasi-UDG communication model with ρ≥1/√2. Compared with existing location-free planarization approaches, our method can extract a provably connected planar graph, called topological planar simplification (TPS), from the connectivity graph in a fine-grained manner using local connectivity information. We evaluate our design through extensive simulations and compare with the state-of-the-art approaches. The simulation results show that our method produces high-quality planar graphs with a small stretch factor in practical large-scale networks.

In order to provide reliable data delivery and system management for large-scale wireless sensor networks (WSNs), tracing the route paths of packets in a lightweight manner is crucial and critical. Real-time path tracing technology enables us to observe every data transmission and analyze network dynamics in a fine-grained fashion. Due to resource constraints of WSNs, however, it is difficult, if not impossible, to integrate into each packet with its full path information. We attempt to capture such information with inserting a small and constant overhead into each packet. In this design, PathZip, each sensor node performs lightweight hash-based computations to passively label every packet forwarded. Meanwhile, the sink extracts the label information so as to leverage the pre-knowledge on the network to compute the full packet path. Both topology-aware and geometry-assistant techniques are utilized by PathZip in order to exploit different network knowledge and reduce the computation and storage overhead greatly. We conduct theoretical analysis and extensive simulations to evaluate the performance of our design. The results show that our method is effective to trace the full route path in large-scale WSNs, and outperforms the state-of-the-art methods.

Coverage problem is a fundamental issue in wireless ad hoc and sensor networks. Previous techniques for coverage scheduling often require accurate location information or range measurements, which cannot be easily obtained in resource-limited ad hoc and sensor networks. Recently, a method based on algebraic topology is proposed to achieve coverage verification using only connectivity information. The topological method sheds some light on the issue of location-free coverage. Unfortunately, the needs of centralized computation and rigorous restriction on sensing and communication ranges greatly limit the applicability in practical large-scale distributed sensor networks. In this work, we make the first attempt toward establishing a graph theoretical framework for connectivity-based coverage with configurable coverage granularity. We propose a novel coverage criterion and scheduling method based on cycle partition. Our method is able to construct a sparse coverage set in a distributed manner, using purely connectivity information. Compared with existing methods, our design has a particular advantage, which permits us to configure or adjust the quality of coverage by adequately exploiting diverse sensing ranges and specific requirements of different applications. We formally prove the correctness and evaluate the effectiveness of our approach through extensive simulations and comparisons with the state-of-the-art approaches.

Extracting planar graph from network topologies is of great importance for efficient protocol design in wireless ad hoc and sensor networks. Previous techniques of planar topology extraction are often based on ideal assumptions, such as UDG communication model and accurate node location measurements. To make these protocols work effectively in practice, we need extract a planar topology in a location-free and distributed manner with small stretch factor. Current location-free methods cannot provide any guarantee on the stretch factor of the constructed planar topologies. In this work, we present a fine-grained and location-free network planarization method. Compared with existing location-free planarization approaches, our method can extract a high-quality planar graph, called TPS (Topological Planar Simplification), from the communication graph using local connectivity information. TPS is proved to be a planar graph and has a constant stretch factor for a large class of network instances. We evaluate our design through extensive simulations and compare with the state-of-the-art approaches. The simulation results show that our method produces planar graphs with a small constant stretch factor, often less than 1.5.

Wormhole attack is a severe threat to wireless ad hoc and sensor networks. Most existing countermeasures either require specialized hardware devices or make strong assumptions on the network in order to capture the specific (partial) symptom induced by wormholes. Those requirements and assumptions limit the applicability of previous approaches. In this paper, we present our attempt to understand the impact and inevitable symptom of wormholes and develop distributed detection methods by making as few restrictions and assumptions as possible. We fundamentally analyze the wormhole problem using a topology methodology and propose an effective distributed approach, which relies solely on network connectivity information, without any requirements on special hardware devices or any rigorous assumptions on network properties. We formally prove the correctness of this design in continuous geometric domains and extend it into discrete domains. We evaluate its performance through extensive simulations.

Localization is crucial for wireless ad hoc and sensor networks. As the distance-measurement ranges are often less than the communication ranges for many ranging systems, most communication-dense wireless networks are localization-sparse. Consequently, existing algorithms fail to provide accurate localization supports. In order to address this issue, by introducing the concept of component, we group nodes into components so that nodes are able to better share ranging and anchor knowledge. Operating on the granularity of components, our design, CALL, relaxes two essential restrictions in localization: the node ordering and the anchor distribution. Compared to previous designs, CALL is proven to be able to locate the same number of nodes using the least information. We evaluate the effectiveness of CALL through extensive simulations. The results show that CALL locates 90% nodes in a network with average degree 7.5 and 5% anchors, which outperforms the state-of-the-art design Sweeps by about 40%.

Local monitoring is an effective mechanism for the security of wireless sensor networks (WSNs). Existing schemes assume the existence of sufficient number of active nodes to carry out monitoring operations. Such an assumption, however, is often difficult for a large-scale sensor network. In this work, we focus on designing an efficient scheme integrated with good self-monitoring capability as well as providing an infrastructure for various security protocols using local monitoring. To the best of our knowledge, we are the first to present the formal study on optimizing network topology for edge self-monitoring in WSNs. We show that the problem is NP-complete even under the unit disk graph (UDG) model and give the upper bound on the approximation ratio in various graph models. We provide polynomial-time approximation scheme (PTAS) algorithms for the problem in some specific graphs, for example, the monitoring-set-bounded graph. We further design two distributed polynomial algorithms with provable approximation ratio. Through comprehensive simulations, we evaluate the effectiveness of our design.

Coverage problem is a fundamental issue in wireless ad hoc and sensor networks. Previous techniques for coverage scheduling often require accurate location information or range measurements, which cannot be easily obtained in resource-limited ad hoc and sensor networks. Recently, a method based on algebraic topology has been proposed to achieve coverage verification using only connectivity information. The topological method sheds some light on the issue of location-free coverage. Unfortunately, the needs of centralized computation and rigorous restriction on sensing and communication ranges greatly limit the applicability in practical large-scale distributed sensor networks. In this work, we make the first attempt towards establishing a graph theoretical framework for connectivity-based coverage with configurable coverage granularity. We propose a novel coverage criterion and scheduling method based on cycle partition. Our method is able to construct a sparse coverage set in a distributed manner, using purely connectivity information. Compared with existing methods, our design has a particular advantage, which permits us to configure or adjust the quality of coverage by adequately exploiting diverse sensing ranges and specific requirements of different applications. We formally prove the correctness and evaluate the effectiveness of our approach through extensive simulations and comparisons with the state-of-the-art approaches.

Wormhole attack is a severe threat against wireless ad hoc and sensor networks. It can be launched without compromising any legitimate node or cryptographic mechanisms, and often serves as a stepping stone for many serious attacks. Most existing countermeasures often make critical assumptions or require specialized hardware devices in the network. Those assumptions and requirements limit the applicability of previous approaches. In this work, we explore the impact of wormhole attacks on network connectivity topologies, and develop a simple distributed method to detect wormholes, called WormCircle-. WormCircle relies solely on local connectivity information without any requirements on special hardware devices or making any rigorous assumptions on network properties. We establish the correctness of this design in continuous geometric domains and extend it into discrete networks. We evaluate the effectiveness in randomly deployed sensor networks through extensive simulations.

Location-free boundary recognition is crucial and critical for many fundamental network functionalities in wireless ad hoc and sensor networks. Previous designs, often coarse-grained, fail to accurately locate boundaries, especially when small holes exist. To address this issue, we propose a fine-grained boundary recognition approach using connectivity information only. This algorithm accurately discovers inner and outer boundary cycles without using location information. To the best of our knowledge, this is the first design being able to determinately locate all hole boundaries no matter how small the holes are. Also, this distributed algorithm does not rely on high node density. We formally prove the correctness of our design, and evaluate its effectiveness through extensive simulations.

Localization is crucial for wireless ad hoc and sensor networks. As the distance-measurement ranges are often less than that of the communication range for many ranging systems, most communication-dense wireless networks are often localization-sparse. Consequently, most existing algorithms fail to provide accurate localization supports. In order to address this issue, by introducing a concept ofcomponent, we propose to group nodes into components, so that nodes are able to better share their ranging and anchor knowledge. This design, CALL, relaxes two essential restrictions in localization: node ordering and anchor distribution. We evaluate the effectiveness of CALL through extensive simulations. The results show that CALL locates 80% nodes in a network with average degree 7.5 and 5% percent anchors, which outperforms the state of the art design Sweeps about 20%.

Local monitoring is an effective mechanism for the security of wireless sensor networks (WSNs). Existing schemes assume the existence of sufficient number of active nodes to carry out monitoring operations. Such an assumption, however, is often difficult for a large scale sensor network. In this work, we focus on designing an efficient scheme integrated with good self-monitoring capability as well as providing an infrastructure for various security protocols using local monitoring. To the best of our knowledge, we are the first to present the formal study on finding optimized self-monitoring topology for WSNs. We show the problem is NP-complete even under the unit disk graph (UDG) model, and give the upper bound on the approximation ratio. We further propose two distributed polynomial algorithms with provable approximation ratio to address this issue. Through comprehensive simulations, we evaluate the effectiveness of this design.