Modern data centers increasingly use multi-GPU systems for AI and high-performance computing, where growing data transfer demands lead to high energy consumption and performance bottlenecks in electrical networks. Optical interconnects offer compelling advantages to address these challenges, including high bandwidth, distance-independent latency, and better energy efficiency. This paper presents DOME, a rack-scale optical interconnection network that connects multiple GPUs using high-radix optical switches and extends optical interfaces into GPU packages close to memories and multiprocessors, forming distinct in-GPU and GPU-to-GPU network domains. To efficiently manage the paths across switches and domains, we develop a multi-switch arbitration scheme and a time-slotted path reservation scheme that quickly identifies the earliest time when the path is available in all network domains, reducing unnecessary reservation retries. Evaluations reveal that DOME achieves 14% speedup while maintaining comparable energy consumption compared to the state-of-the-art preemptive chain feedback control scheme.

Photonic computing is poised to revolutionize artificial intelligence (AI) acceleration by offering exceptional speed and energy efficiency for General Matrix Multiplication (GeMM). However, existing works on photonic tensor core architectures face significant challenges in managing real-valued and dynamic operands. Specifically, Mach-Zehnder interferometer (MZI) meshes require computationally intensive singular value decomposition (SVD) for matrix preprocessing, while microring resonator (MRR) weight banks are limited to non-negative operands, complicating operations with dual negative values. Additionally, coherent interference crossbars, although theoretically capable of supporting real-valued multiplication, struggle with fabrication complexities and sensitivity to environmental variations.To address these limitations, we propose FSR-GeMM schema, a scalable photonic accelerator that leverages free-spectral range (FSR) multiplexing. This architecture eliminates the need for SVD preprocessing, supports direct multiplication of two dynamic real-valued operands, and enhances reliability and scalability. Experimental results from a photonic-electronic prototype demonstrate that FSR-GeMM achieves up to 57× improvements in area efficiency and 13.8× gains in energy efficiency compared to existing photonic GeMM accelerators. Furthermore, it reduces energy consumption by 70% relative to MRR-based systems and achieves 21× speedup against leading photonic GeMM accelerator designs, highlighting its potential to advance practical and scalable AI acceleration.

High-Level Synthesis (HLS) tools are widely adopted in FPGA-based domain-specific accelerator design. However, existing tools rely on fixed optimization strategies inherited from software compilations, limiting their effectiveness. Tailoring optimization strategies to specific designs requires deep semantic understanding, accurate hardware metric estimation, and advanced search algorithms - capabilities that current approaches lack.We propose DAPO, a design structure-aware pass ordering framework that extracts program semantics from control and data flow graphs, employs contrastive learning to generate rich embeddings, and leverages an analytical model for accurate hardware metric estimation. These components jointly guide a reinforcement learning agent to discover design-specific optimization strategies. Evaluations on standard HLS benchmarks demonstrate that our end-to-end flow delivers 1.67× speedup on pragma-free designs and a 2.36× speedup on designs with pragmas over Vitis HLS with comparable resource usage.

Hypergraph neural networks (HyperGNNs) have garnered increasing attention for their ability to model high-order relationships in various domains. However, the extremely sparse connections inherent to hypergraphs result in numerous off-chip memory accesses, posing a long-latency inference issue on edge devices. Existing hardware accelerators focus solely on exploiting the limited data reuse opportunities in hypergraphs to mitigate this issue, without addressing the underlying cause: the sparsity of the hypergraph structures themselves.To address the fundamental limitation, this paper proposes Grin, a general HyperGNN training framework. It is designed to restructure hypergraphs for enhancing inference efficiency on edge devices regardless of hardware architectures while improving model performance. Specifically, hyperedge pruning within Grin is utilized to eliminate redundant computation workloads, effectively lowering overall off-chip memory accesses. Moreover, Grin redefines the objective of traditional data augmentation by incorporating hardware efficiency alongside model accuracy. This shift enables significantly increased data reuse in the remaining computation workloads, thereby ensuring model performance and further reducing off-chip memory accesses. Experiments demonstrate that, with increased model accuracy, deploying Grin-optimized hypergraphs on the state-of-the-art (SOTA) accelerator achieves an average inference speedup of 1.41× compared to the original hypergraphs on the same accelerator, while reducing off-chip memory accesses by 27.60%. Furthermore, this deployment achieves a 14.82× speedup over the SOTA GPU-based system.

Rigid Body Dynamics (RBD) computation is a critical component of robotic control, often dominating system runtime due to its algorithmic complexity and high parallelism demands. CPUs suffer from limited parallelism and cache-unfriendly access patterns, while GPUs incur prohibitive memory-access latency and per-task response time, making them unsuitable for real-time control. Both platforms also consume excessive power for edge deployment. FPGAs offer superior latency, energy efficiency, and customizable hardware-level parallelism, emerging as promising targets for RBD acceleration. However, existing FPGA designs still face critical limitations. First, the intensive use of multiply-accumulate operations leads to high Digital Signal Processing (DSP) slices consumptionespecially for high degrees-of-freedom (DOF) robots-resulting in limited scalability. Second, RBD functions include mass matrix inversion function, which is inefficient on FPGA due to reciprocal operations falling on the longest latency path, severely limiting performance. Third, mismatched processing rates across modules introduce idle cycles, resulting in poor DSP utilization. To address these issues, we propose DRACO, a hardwareefficient and high-performance RBD accelerator based on FPGA, introducing three key innovations. First, we propose a precisionaware quantization framework that reduces DSP demand by up to$4 \times$while preserving motion accuracy. This is also the first study to systematically evaluate quantization impact on robot control and motion for hardware acceleration. Second, we leverage a hardware-efficient division deferring optimization in mass matrix inversion algorithm, which decouples reciprocal operations from the longest latency path to improve the performance. Finally, we present an inter-module DSP reuse methodology to improve DSP utilization and save DSP usage. Experiment results show that DRACO achieves up to$8 \times$throughput improvement and$7.4 \times$latency reduction over state-of-the-art (SOTA) RBD accelerators across various robot types, demonstrating its effectiveness and scalability for high-DOF robotic systems.

Coarse-Grained Reconfigurable Architectures (CGRAs) are a promising and versatile accelerator platform, offering a balance between the performance and efficiency of specialized accelerators and software programmability. However, their full potential is severely hindered by control flow in accelerated kernels, as control flow (e.g., loops, branches) is fundamentally incompatible with the parallel, data-driven CGRA fabric. Prior strategies to resolve this mismatch in CGRA kernel acceleration are either inefficient, sacrificing performance for generality, or lack generality due to the difficulty of adapting them across different execution models. Thus, a general and unified solution for efficient CGRA kernel acceleration remains elusive. This paper introduces NEURA, a unified and retargetable compilation framework that systematically resolves the control-dataflow mismatch in CGRAs. NEURA's core innovation is a novel, pure dataflow intermediate representation (IR) built on a predicated type system. In this IR, control contexts are embedded as a predicate within each data, making control an intrinsic property of data. This mechanism enables NEURA to systematically flatten complex control flow into a single unified dataflow graph. This unified representation decouples kernel representation from hardware, empowering NEURA to retarget diverse CGRAs with different execution models and microarchitectural features. When targeted to a high-performance spatio-temporal CGRA, NEURA delivers a 2.20x speedup on kernel benchmarks and up to 2.71x geometric mean speedup on real-world applications over state-of-the-art (SOTA) high-performance baselines. It also provides a competitive solution against the SOTA low-power CGRA when retargeted to a spatial-only CGRA. NEURA is open-source and available at https://github.com/coredac/neura.

Recently, printed circuit board (PCB) placement has emerged as a significant challenge since the scale of PCB designs has rapidly enlarged. Furthermore, the presence of various types of constraints with differing tolerance priorities hampers the automation of PCB layout design, necessitating substantial manual effort. To address this problem, we introduce a novel agent-based framework that automatically generates PCB layouts meeting industrial constraints through user interactions. This framework includes two main agents: a reinforcement learning (RL)-based agent for layout inference and fine-tuning, and a large language model (LLM)-based agent for interactive optimization. Experimental results on 17 industrial tasks show that our framework outperforms other state-of-the-art methods.

Electronic-photonic integrated circuit (EPIC) technologies are revolutionizing computing systems by improving their performance and energy efficiency. However, simulating EPIC is challenging and time consuming. In this paper, we propose the physics-informed neural network (PINN) based SPICE-compatible modeling method for EPICs. Experimental results show our method can speed up EPIC simulation by more than 100 times on average compared to FDTD method.

High-level synthesis (HLS) tools streamline FPGA design by enabling engineers to implement hardware using $\mathrm{C} / \mathrm{C}++$ languages. However, while clock management serves as a critical stage in the FPGA EDA flow that affects system-level performance, area, and especially power consumption, existing commercial HLS tools lack comprehensive solutions for clock management. Specifically, the diversity of clock resources creates a vast design space for finding the optimal configuration, and the insufficient analysis of multiple clock domain scenarios hinders effective clock-oriented optimizations in HLS. This work introduces AutoClock, an open-source integrated clock management framework that complements AMD Vitis HLS. AutoClock allocates resources for clock generation, assigns modules to appropriate clock domains, addresses metastability and time division multiplexing (TDM) malfunctioning introduced by multiclock domain architectures, and hierarchically gates the clock of modules in a design. Experimental results demonstrate that AutoClock can fully utilize clock resources on FPGAs and help reduce dynamic power consumption by up to 74.38%.

In recent years, the advent of photonic integrated circuits (PICs) has demonstrated great prospects and applications to address critical issues such as limited bandwidth, high latency, and high power consumption in data-intensive systems. However, the field of physical design automation for PICs remains in its infancy, with a notable gap in electronic layer layout design tools. Current research on PIC physical design automation primarily focuses on optical layer layouts, often overlooking the equally crucial electronic layer layouts. Although well-established for conventional integrated circuits (ICs), existing EDA tools are inadequately adapted for PICs due to their unique characteristics and constraints. As PICs grow in integration density and size, traditional manual-based design methods become increasingly inefficient and sub-optimal, potentially compromising overall PIC performance. To address this challenge, we propose PICELF, the first framework in the literature for automatic PIC electronic layer layout generation. Our framework comprises a nonlinear binary programming (NBP)-based netlist generator with scalability optimization and a two-stage router featuring initial parallel routing followed by post-routing optimization. We validate our framework's effectiveness and efficiency using a real PIC chip benchmark established by us. Experimental results demonstrate that our method can efficiently generate high-quality PIC electronic layer layouts and satisfy all design rules, within reasonable CPU times, while related existing methods are not applicable.

High-performance computing and AI applications necessitate high-bandwidth communication between GPUs. Traditional electrical interconnects for GPU-to-GPU communication face challenges over longer distances, including high power consumption, crosstalk noise, and signal loss. In contrast, optical interconnects excel in this domain, offering high bandwidth and consistent power dissipation over long distance. This paper proposes BEAM, a Bandwidth-Enhanced optical interconnect Architecture for Multi-GPU systems. BEAM extends electrical-optical interfaces into the GPU package, positioning them close to GPU compute logic and memory. Unlike existing single-channel approaches, each BEAM optical interface incorporates multiple parallel optical channels, further enhancing bandwidth. An arbitration scheme manages channel usage among data transfers. Evaluation on Rodinia benchmarks and LLM training kernels demonstrates that BEAM achieves a speedup of 1.14 – 1.9× and reduces energy consumption by 29 – 44% compared to the electrical-interconnected system and state-of-the-art schemes, while maintaining comparable chip area consumption.

Neural rendering has gained prominence for its high-quality output, which is crucial for AR/VR applications. However, its large voxel grid data size and irregular access patterns challenge real-time processing on edge devices. While previous works have focused on improving data locality, they have not adequately addressed the issue of large voxel grid sizes, which necessitate frequent off-chip memory access and substantial on-chip memory. This paper introduces SpNeRF, a software-hardware co-design solution tailored for sparse volumetric neural rendering. We first identify memory-bound rendering inefficiencies and analyze the inherent sparsity in the voxel grid data of neural rendering. To enhance efficiency, we propose novel preprocessing and online decoding steps, reducing the memory size for voxel grid. The preprocessing step employs hash mapping to support irregular data access while maintaining a minimal memory size. The online decoding step enables efficient on-chip sparse voxel grid processing, incorporating bitmap masking to mitigate PSNR loss caused by hash collisions. To further optimize performance, we design a dedicated hardware architecture supporting our sparse voxel grid processing technique. Experimental results demonstrate that SpNeRF achieves an average 21.07× reduction in memory size while maintaining comparable PSNR levels. When benchmarked against Jetson XNX, Jetson ONX, RT-NeRF. Edge and NeuRex. Edge, our design achieves speedups of 95.1×, 63.5×, 1.5× and 10.3×, and improves energy efficiency by 625.6×, 529.1×, 4×, and 4.4×, respectively.

Fully Homomorphic Encryption (FHE) has emerged as a promising solution for the secure computation on encrypted data without leaking user privacy. Among various FHE schemes, Torus FHE (TFHE) distinguishes itself by its ability to perform exact computations on non-linear functions within the encrypted domain, satisfying the crucial requirement for privacy-preserving AI applications. However, the high computational overhead and strong data dependency in TFHE's bootstrapping process present significant challenges to its practical adoption and efficient hardware implementation. Existing TFHE accelerators on various hardware platforms still face limitations in terms of performance, flexibility, and area efficiency. In this work, we propose UNIT, a novel and highly unified accelerator for Programmable Bootstrapping (PBS) in TFHE, featuring carefully designed computation units. We introduce a unified architecture for negacyclic (inverse) number theoretic transform (I)NTT with fused twisting steps, which reduces computing resources by 33% and the memory utilization of pre-stored factors by nearly 66 %. Another key feature of UNIT is the innovative design of the monomial number theoretic transform unit, called OF-MNTT, which leverages on-the-fly twiddle factor generation to eliminate memory traffic and overhead. This memory-efficient and highly parallelizable approach for MNTT is proposed for the first time in TFHE acceleration. Furthermore, UNIT is highly reconfigurable and scalable, supporting various parameter sets and performance-resource requirements. Our proposed accelerator is evaluated on the Xilinx Alveo U250 FPGA platform. Experimental results demonstrate its superior performance compared to the state-of-the-art GPU and FPGA-based implementations with the improvement of 8.3x and 3.63x, respectively. In comparison with the most advanced FPGA implementation, UNIT achieves 30% enhanced area efficiency and 3.2 x reduced power with much better flexibility.

Modern processors feature complex architectures that necessitate the generation of extensive test programs to ensure functional correctness, making testing the most time-consuming stage of the processor design flow. Existing automated verification frameworks for agile design exhibit significant limitations, such as fixed program structures restricting flexibility, uncontrolled control flows leading to invalid instructions, and low coverage of the vast state space. To address these limitations, we propose CURE-Fuzz, a curiosity-driven reinforcement learning framework designed to enhance agile hardware testing. By integrating a hierarchical test generation model with a curiosity-driven exploration mechanism, CURE-Fuzz enables precise control over test program structure and dependencies while efficiently navigating unexplored processor states. Evaluations on Rocket and Boom core demonstrate that CURE-Fuzz achieves higher coverage and exhibits superior bug detection capabilities compared to state-of-the-art fuzzers.

Legalization is a critical yet time-consuming step in very large-scale integration (VLSI) design, tasked with iteratively relocating standard cells to eliminate overlaps while resolving design rule violations. This process is repeatedly invoked during VLSI physical design. However, increasing spatial constraints and complex design rules impose significant challenges on existing CPU- and GPU-based legalizers, including suboptimal task assignment, inefficient algorithm, and long hardware idle time caused by processing tasks with irregular computational patterns in parallel.

In-network compression has been proposed recently to support efficient communication. However, we find employing compression blindly cannot always pay off since de/compression leads to extra packet transmission delay. We thereby propose selective compression which compresses data adaptively based on network state and predicted compression ratio. Moreover, we observe that simply applying selective compression in a conventional single network is not energy efficient. Therefore, we propose SCNoCs, a heterogeneous Multi-NoC (Main-Net and HelperNet) architecture with the support of selective compression and power gating. SCNoCs can dynamically adjust the policy of selective compression and the utilization degree of the Helper-Net according to the network state at runtime. Experimental results show that our selective compression outperforms conventional compression by 1.5$ \times $. Besides, our proposed SCNoCs achieves comparable performance while reducing energy consumption by 43.4%, compared with the baseline.

GPU-based computing serves as the primary solution driving the performance of HPC systems. However, modern GPU systems encounter performance bottlenecks resulting from heavy memory access traffic and insufficient NoC bandwidth. In this work, we propose a collaborative coalescing mechanism aimed at eliminating redundant memory access and boosting GPU system performance. To achieve this, we design a coalescing unit for each memory partition, effectively merging requests from both inter-cluster and intra-cluster SMs. Additionally, we introduce a hierarchical multicast module to replicate and distribute the coalesced reply messages to multiple destination SMs. Experimental results show that our method achieves 20.6% improvement on performance and 27.1% reduction on NoC traffic over the baseline.

Fully homomorphic encryption (FHE) presents a promising opportunity to remove privacy barriers in various scenarios including cloud computing and secure database search, by enabling computation on encrypted data. However, integrating FHE with real-world applications remains challenging due to its significant computational overhead. In the FHE scheme, Number Theoretic Transform (NTT) consumes the primary computing resources and has great potential for acceleration. For the first time, we present a photonic NTT accelerator, PhotonNTT, with high energy efficiency and parallelism to address the above challenge. Our approach involves formulating the NTT into matrix-vector multiplication (MVM) operations and mapping the data flow into parallel photonic MVM units. A dedicated data mapping scheme is proposed to introduce free spectral range (FSR) and distributed RAM design into the system, which enables a high bit-wise parallelism level. The system's reliability is validated through the Monte-Carlo BER analysis. The experimen-tal evaluation shows that the proposed architecture outperforms SOTA CiM-based NTT accelerators with an improvement of 50x in throughput and 63x improvement in energy efficiency.

Photonic computing, known for its high bandwidth and energy efficiency, harnesses physical phenomena in the optical domain to accelerate a wide range of computational operations such as dot product, matrix multiplication, Fourier transform, 1D convolution, and more. However, the multitude of computational operations mentioned above poses challenges in mapping realistic neural network workloads onto underlying photonic hardware. This complexity requires extensive expertise and laborious programming, impeding the practical adoption and deployment of photonic acceleration. To address this gap, we propose an end-to-end compilation framework comprising a Photonic Compiler Collection (PCC). This framework automates the mapping of high-level deep neural network (DNN) specifications onto target architectures of photonic-electronic accelerators. Additionally, we present a method to streamline neural network workloads by leveraging the multilevel intermediate representation (MLIR) and compiler optimization techniques, targeting photonic-specific patterns. Moreover, we conduct a comprehensive case study illustrating the integration of a typical computational operator, the Mach-Zehnder Interferometer (MZI) mesh, into PCC. Our experimental results demonstrate that PCC achieves up to a 4x speedup on DNN workloads compared to handcrafted implementations. In summary, our proposed framework offers a practical and automated solution for compiling, optimizing, and flexibly sup-porting newer operators of photonic devices. We anticipate that our framework will significantly accelerate the development and deployment of photonic applications in real-world AI scenarios.

In the realm of neural network computation, optical neural network accelerators (ONNs) have emerged as a promising solution, leveraging the inherent speed and parallelism of optical systems. Despite their potential, current ONN designs often fall short due to inefficient data movement and reliance on traditional electronics-based dataflows.

Rechargeable battery has attracted great research popularity in recent years, and numerous data-driven solutions have been proposed for its modeling. While most research works in this domain focus on the statistical correlations among extracted variables, understanding their causal relationships is equally important to provide useful insights, thereby improving the operation in real-world scenarios. In order to reach this goal, causal discovery has been utilized, which learns a directed acyclic graph (DAG) structure, referred to as causal graph, from observation data to represent the causal relationships among variables with specific physical interpretations. However, existing approaches only consider the DAG characteristics between individual variables. It may easily lead to group-level causal conflicts if multiple variables are created from the same source factor, which is a common practice for modeling the degradation of rechargeable batteries. In order to tackle this challenge, a novel group-level DAG constraint is proposed in this paper as a continuous regularization term that discourages cycles among blocks as well as individual elements in the weighted adjacent matrix corresponding to the causal graph. It is efficiently solved using an augmented Lagrangian-based method.

As Moore's law slows down, computing systems must prioritize higher energy efficiency to sustain performance scaling. GPUs have emerged as the primary workhorses of computing resources, making the achievement of high energy efficiency in GPUs a critical concern. However, implementing runtime power management on GPUs poses significant challenges due to the high variations and complexities arising from workloads and hardware configurations, which render offline optimization and reactive-based methods less effective. In this paper, we present a program counter (PC)-oriented prediction-based power management approach for GPGPUs. Our approach leverages the benefits of prediction to address online variations while enhancing prediction capability through knowledge transfer across different levels of resources. Experiments conducted on realistic applications demonstrate that our proposed method achieves the maximum energy savings under a user-defined performance constraint compared to state-of-the-art designs.

Chiplet technology has emerged as a promising solution to address the increasing demand for high-performance computing in light of the slowdown of Moore’s law. While chiplet-based multicore systems offer higher performance through heterogeneous integration, they also pose challenges for power delivery system (PDS) design. The integration of additional vertical and inter-chiplet connections, along with higher power density, impose stringent requirements on power delivery. Moreover, PDS efficiency is affected by workload variations at runtime, necessitating the need to design and manage PDSs and processors as a whole to improve system energy efficiency while balancing performance. In this article, we propose an offline-online co-design optimization methodology that combines offline PDS design optimization with online power management. To address the power consumption and delivery mismatch, we introduce a centralized deep Q-network (DQN)-based online control scheme for power co-management in chiplet-based multicore systems. By carefully designing the state space and reward functions, our approach achieves workload-aware adaptive control to reduce the energy-delay-product (EDP) while maintaining PDS efficiency under a given performance target (PT). We conduct evaluations on realistic applications to validate the effectiveness of our approach. For 64-core systems, our method achieves an average EDP reduction of 67% while meeting a 90% PT, surpassing state-of-the-art modular Q-learning (MQL)-based and heuristic-based approaches by up to 4% and 16%, respectively. Additionally, our approach demonstrates wiser action selection policies, higher control stability, and lower implementation overhead compared to the MQL-based approach.

As Moore's law slows down, computing systems must pivot towards higher energy efficiency to continue scaling performance. Reinforcement learning (RL) performs more adaptively than conventional methods in runtime power management under varied hardware configurations and varying software workloads. However, prior works on either model-free or model-based RL approaches face a non-negligible challenge: relearning the policies to adapt to the new environment is unacceptably time-consuming, especially when encountering significant variances in workloads or hardware configurations. Moreover, existing research on accelerating learning has focused on the speedup while largely ignoring the efficiency degradation of the results. In this paper, we present a smart transfer-enabled Q-learning (STQL) approach to boost the learning process and guarantee the learning efficiency through a contradiction checking mechanism, which wisely evicts inappropriate transferred knowledge. Experiments on realistic applications show that the proposed method can speed up the learning process to up to 2.3x and achieve a 6.2% energy-delay product (EDP) reduction compared to the state-of-the-art design.

Modern GPU systems integrate hundreds of SMs on a single die, and future scaling envisions even more SMs being incorporated. However, the limited number of transistors per die constrains this growth. While current chiplet technology shows promise, its performance is limited by the bandwidth and energy efficiency of existing chiplet interconnect technologies. In contrast, optical interconnects offer ultra-high bandwidth and energy efficiency, making them ideal for high-performance chiplet-based GPUs. This work proposes a novel region-based optical network, called RONet, that divides a chiplet-based GPU system with a 2D Mesh layout into multiple row and column regions, where each region is connected by a separate optical link. Additionally, RONet employs a tuning-free transmission mechanism to further enhance inter-chiplet bandwidth. Experimental results show that RONet achieves 43% improvement on performance and 25.4% reduction on system energy consumption over the baseline.

Recent advances in the architecture design for photonic accelerators have demonstrated great promise to accelerate deep neural network (DNN) applications, and also allude to the essential collaboration of the electronic subsystems for efficient logic arithmetic and memory access. However, available tools to design and evaluate photonic accelerators usually neglect the cross-stack effects or low-level details in real-world scenarios, ranging from programming-stack inefficiency to electronic peripheral implementation complexity. This frustrating fact makes it difficult to holistically estimate the performance metrics of a practical photonic-electronic collaborative computing system. In addition, until now, no toolchain can provide programmable, hardware-reconfigurable, and end-to-end rapid verification for photonic accelerators. Here we present FIONA, a Full-stack Infrastructure for Optical Neural Accelerator, which comprises a photonic-electronic co-simulation framework for multilevel design space exploration (DSE), and a transferable hardware prototyping template for physical verification. Specifically, the co-simulation framework consists of a functional simulator at the instruction set architecture (ISA) level to agilely verify the programming software stack and a register-transfer level (RTL) cycle-accurate simulator to precisely profile the overall system. We also demonstrate LightRocket as a case study of the FIONA toolchain to show the full workflow of designing a Turing-complete photonic accelerator system that supports arbitrary DNN workloads and on-chip training. The toolchain is open-sourced and available at https://github.com/hkust-fiona/.

Resistive random access memory (ReRAM) has demonstrated great promises of in-situ matrix-vector multiplications to accelerate deep neural networks. However, subject to the intrinsic properties of analog processing, most of the proposed ReRAM-based accelerators require excessive costly ADC/DAC to avoid distortion of electronic analog signals during inter-tile transmission. Moreover, due to bit-shifting before addition, prior works require longer cycles to serially calculate partial sum compared to multiplications, which dramatically restricts the throughput and is more likely to stall the pipeline between layers of deep neural networks.

Achieving high performance with low energy consumption has become a primary design objective in multi-core systems. Recently, power management based on reinforcement learning has shown great potential in adapting to dynamic environments without much prior knowledge. However, conventional Q-learning (QL) algorithms adopted in most existing works encounter serious problems about scalability, instability, and overestimation. In this paper, we present a deep reinforcement learning-based approach to improve the stability and robustness of power management while reducing the energy-delay product (EDP) under user-specified performance requirements. The comprehensive status of the system is monitored periodically, making our controller sensitive to environmental change. To further improve the learning effectiveness, knowledge sharing among multiple devices is implemented in our approach. Experimental results on multiple realistic applications show that the proposed method can reduce the instability up to 68% compared with QL. Through knowledge sharing among multiple devices, our federated approach achieves around 4.8% EDP improvement over QL on average.

Emerging integrated photonic neural networks have experimentally proved to achieve an ultra-high speedup of deep neural network training and inference in the optical domain. However, photonic devices suffer from the inherent crosstalk noise and loss, inevitably leading to reliability concerns. This paper systematically analyzes the impacts of crosstalk and loss on photonic computing systems. We propose a crosstalk-aware model for reliability estimation and find out the worst-case bounds as we increase the footprints and scales of the photonic chips. Our evaluations show that −30dB crosstalk noise can cause maximal photonic chip integration to a sharp drop by 109x. To facilitate very-large-scale photonic integration for future computing, we further propose multiple heterogeneous bijou photonic-cores to address the crosstalk-aware reliability concern.

Cache coherence overhead in manycore systems is becoming prominent with the increase of system scale. However, traditional electrical networks restrict the efficiency of cache coherence transactions in the system due to the limited bandwidth and long latency. Optical network promises high bandwidth and low latency, and supports both efficient unicast and multicast transmission, which can potentially accelerate cache coherence in manycore systems. This work proposes a novel photonic cache coherence network with a physically centralized logically distributed directory called PCCN for chiplet-based manycore systems. PCCN adopts a channel sharing method with a contention solving mechanism for efficient long-distance coherence-related packet transmission. Experiment results show that compared to state-of-the-art proposals, PCCN can speed up application execution time by 1.32x, reduce memory access latency by 26%, and improve energy efficiency by 1.26x, on average, in a 128-core system.

To establish flatten networks and accomplish rapid and efficient communications in the future hyper-scale data centers, HERO, a high-radix optical switch based on integrated silicon photonics, is proposed in this work. The architecture of HERO, including the switch fabric, switch interface, and switch controller, is described in detail. Two new switch control approaches: 1) split-transaction predictive control and 2) wavelength-group switching, are developed. The efficient control together with the optimized high-radix integrated optical switch fabrics help HERO achieve over 1 Pbps switching capacity. Even for small packets, such as 64–256 B Ethernet packets, the maximal utilization of the switch can reach up to 83%, and the throughput can approximate 1 Pbps. Further design explorations on the packet length and some key configurations, including the number of wavelengths and wavelength groups, are also conducted in this work, paving the way to the design of high-performance flatten data-center networks in the future.

Detailed cycle-accurate simulation of multicore systems is naturally slow. Statistical simulation is one alternative that permits trading off simulation speed for accuracy. However, there is a lack of effective memory locality models for multicore applications. Hence, existing statistical simulators neglect data-sharing between threads. Additionally, the standard method to speed up statistical simulations is to blindly reduce the trace length to be synthesized. While this gives good control over the speedup, it leaves the simulation error unbounded. In this work, we introduce a novel statistical simulation methodology for exploration of shared-memory multicore systems. It includes a newsharing-localitymodel (Shalom) that captures and reproduces data-sharing in multithread applications. Furthermore, we propose a method to bound the simulation error for a particular metric while maximizing speedup. The technique works by monitoring the convergence of the statistical synthesis. It is referred to asconvergence-deterministicsimulation (Condens). The combination ofShalomandCondensis around 130x faster than cycle-accurate simulations with reasonable accuracy loss. Our approach is also 5x faster than state-of-the-art sampling simulation under the same accuracy level. Compared to previous statistical simulators ignoring sharing, our approach is 2x more accurate for performance metrics and 8x more accurate for cache miss estimations.

Achieving high energy efficiency is a primary design objective for multi-core systems. Dynamic voltage and frequency scaling (DVFS) is one of the most widely-adopted low-power techniques. In this paper, we present a reinforcement learning-based DVFS control approach to reduce energy consumption under user-specified performance requirements. The learning agent periodically selects the voltage and frequency level for all cores based on observations of their computation intensiveness, memory behaviors as well as synchronization among cores. Experimental results on multiple real applications show that the proposed method can achieve significant energy reduction.

Silicon photonics is the leading candidate technology for high-speed and low-energy-consumption networks. Thermal and process variations are the two main challenges of achieving high-reliability photonic networks. Thermal variation is due to the heat issues created by application, floorplan, and environment, while process variation is caused by fabrication variability in the deposition, masking, exposition, etching, and doping. Tuning techniques are then required to overcome the impact of the variations and efficiently stabilize the performance of silicon photonic networks. We extend our previous optical switch integration model, BOSIM, to support the variation and thermal analyses. Based on device properties, we propose indirect feedback tuning (IFT) to simultaneously alleviate thermal and process variations. IFT can improve the BER of silicon photonic networks to 10-9under different variation situations. Compared to state-of-the-art techniques, IFT can achieve an up to 1.52 ×108times bit-error-rate improvement and 4.11X better heater energy efficiency. Indirect feedback does not require high-speed optical signal detection, and thus, the circuit design of IFT saves up to 61.4% of the power and 51.2% of the area compared to state-of-the-art designs.

With the increasing popularity of data-intensive applications in data centers, the switching fabric in the internode network becomes significant. Silicon-photonic switching fabrics have a bright future in data centers, which offer high bandwidth, high energy efficiency, and low latency. However, integrating a high radix multistage switching fabric in a single chip faces challenges. A large number of waveguide crossings on the silicon photonic die causes massive power loss and introduces a tremendous amount of crosstalk noise. In this article, we propose a chip partition optimization platform (POP), which can decrease the number of waveguide crossings and shorten the on-chip traversal distance of optical signals. Our algorithms can effectively reduce the power loss and crosstalk noise in silicon-photonic multistage switching fabrics, and help to improve the signal integrity. For example, compared with the common design, POP can achieve 33-dB improvement on average power loss, 42-dB improvement on the worst-case power loss, and 39-dB improvement on the worst-case signal to noise ratio, in a$1024\times1024$butterfly based silicon-photonic switching fabric.

A lot of efforts have been devoted to optically enabled high-performance communication infrastructures for future manycore processors. Silicon photonic network promises high bandwidth, high energy efficiency and low latency. However, the ever-increasing network complexity results in high optical power demands, which stress the optical power delivery and affect delivery efficiency. Facing these challenges, we propose Ring-based Optical Active Delivery (ROAD) system, to effectively manage and efficiently deliver high optical power throughout photonic-electronic hybrid systems. Experimental results demonstrate up to 5.49X energy efficiency improvement compared to traditional design without affecting processor performance.

Silicon photonic networks are revolutionizing computing systems by improving the energy efficiency, bandwidth, and latency of data movements. Optical modulators, such as microresonators (MRs) and Mach–Zehnder interferometers (MZIs), are the basic building blocks of silicon photonic networks. This paper proposes a SPICE-compatible electro-optical co-simulation model, basic optical switch integration model (BOSIM), to systematically study optical modulators using PN, PIN, and metal–insulator–silicon (MIS) capacitor device technologies. BOSIM holistically models both transient and steady state properties, such as switching speed, power, transmission spectrum, area, and carrier distribution. BOSIM is validated by the measured data from eight research groups and companies. Compared to MRs, BOSIM shows MZIs are fast, with a high extinction ratio and large bandwidth but in the sacrifice of loss, energy, and area. Using a PIN diode over a PN diode can save area, but retain the loss and energy, while an MIS capacitor has the shock response of carrier distribution in a narrow range and is marginalized gradually. For instance, an MZI can achieve a$2.5 {\times }$bit rate,$6.06{\times }$extinction ratio,$71.04 {\times }\,\,3$-dB bandwidth, but costs at least$1.93 {\times }$passing loss,$1.46 {\times }$energy consumption, and$16.67 {\times }$area, compared with MR.

While many new applications prefer manycore processor with a large number of cores, the exploding communications among multiple cores, caches, and off-chip memories is posing a fundamental challenge on manycore designs. Silicon photonics-based interconnection network promises high bandwidth, low latency, and high energy efficiency, and can potentially meet the communication requirements of manycore processors. In this paper, we propose CAMON, a small low-cost silicon photonic chiplet integrated into the manycore processor package. CAMON chiplet can effectively alleviate the communication bottlenecks of manycore processors and improve the energy efficiency of data movement, especially for large-scale systems. We develop a distributed arbitration system, a low-power low-latency optical interface, and an off-chip laser preactivation mechanism for CAMON. The experimental results show that compared with the electrical network, CAMON can improve the full-system performance per energy by 4.6×, speedup the manycore processor by 2.7×, and save the area of the processor die by 3%, in a 512-core system.

The advancement of silicon photonics promises integrated optical switches to provide high-bandwidth, low-latency, and low-power communications in data centers. An optical switch’s loss limits its scale and affects the energy efficiency of the switch system. In this paper, we present cross-layer optical switch optimization (CLOSO), a cross-layer optimization (CLO) framework, based on not only photonic device models at the physical layer but also optical switch models at the fabric layer. With the proposed framework, optimal losses of optical switches can be evaluated efficiently, and the corresponding losses and design parameters of photonic devices can be obtained. Using CLOSO, we optimize four categories of integrated optical switches, Crossbar, PILOSS, DRAGON, and FODON, and compare them regarding their optimal worst-case loss with variation of the switch scale and data rate of signals. Furthermore, system-level evaluations of the optimized optical switches are performed, demonstrating a significant improvement of energy efficiency from the CLO. For instance, CLOSO helps to reduce the energy consumption of a 64-port DRAGON and FODON to as low as 6 pJ/bit and that of a 128-port DRAGON and FODON to as low as 10 pJ/bit. The investigation of 128-port switches also shows the necessity of adaptive power control on lasers for high-radix integrated optical switches. Through quantitative analyses and comparisons, CLOSO shows the capability of facilitating initial design exploration of optical switches and paves the way to fair evaluations and comparisons of switch systems in data centers.

Rack-scale computing systems are promising to undertake the emerging large-scale applications by distributing massive tasks to processing cores. The communication and coordination efficiency of these tasks and resources directly affect the system performance and energy consumption. Silicon photonic interconnects are expected to address the communication and system power consumption challenges imposed on rack-scale systems. However, the control for optical interconnects can cause server performance degradation if not properly designed, especially for the complicated and time-consuming multidomain networks. In this paper, we study the optical interconnects for rack-scale computing systems and propose a new communication flow and control scheme for the efficient coordination of distributed resources. Particularly, we first propose a forward propagation strategy that parallels the path reservation process with the distributed tasks connection setup. Second, we develop a pre-emptive chain feedback (PCF) scheme to optimize multidomain path reservation. The PCF scheme pre-emptively allocates network resources with the help of multicell reservation window and quickly releases resources with a feedback mechanism. This solution increases the network resources utilization and task coordination efficiency while minimizing path reservation overheads. Comparing to the baseline InfiniBand network fabric and handshake scheme, PCF can improve network throughput greatly under uniform and hotspot traffic patterns. Realistic benchmark results show that the PCF scheme on average reduces 52% and 60% energy consumption per unit system performance than InfiniBand and the handshake scheme for a 256-node rack system.

Energy efficiency has become a critical design metric for high-performance systems. Various power management techniques have been proposed for the processor cores such as dynamic voltage and frequency scaling (DVFS), whereas few solutions consider the power losses suffered on the power delivery system (PDS), despite the fact that they have a significant impact on the overall energy efficiency of the system. With the explosive growth of system complexity and highly dynamic workloads variations, it is also challenging to find the optimal power management policies which can effectively match the power delivery with the power consumption. In addition, process variations (PVs) add heterogeneity to systems and make traditional power management methods less effective. To tackle the above problems, we propose a reinforcement-learning-based Chip-Specific Power co-Management (CSPM) scheme for PV-aware manycore systems. Both PDS and processor cores are jointly adjusted by distributed agents with modular Q-learning to improve the overall energy efficiency of the system. System characteristics are naturally included in the learning process to obtain chip-specific policies. Experimental results show that when applied to PV-aware manycore systems with a hybrid PDS constructed by both on- and off-chip voltage regulators, the proposed method achieves a 60.1% reduction of the overall energy delay product (EDP) of the system, on average, compared to a traditional DVFS approach.

Minimizing energy consumption while satisfying the user-specified performance requirement is a primary design objective for mobile devices. To achieve this, off-the-shelf mobile devices are usually equipped with dynamic voltage and frequency scaling (DVFS)-enabled processors to tradeoff performance for energy reduction through online adjustment of operating points. Recently, reinforcement learning algorithms have been widely studied and show great potential for runtime DVFS control because of their adaptability to the changing environment. However, the rapid evolution of hardware and the ever-growing diversity of mobile applications increase the system complexity dramatically and make it hard for the learning agent to quickly obtain an efficient power management policy. To address this challenge, we propose a decentralized collaborative Q-learning (DCQL)-based approach in this article to solve the DVFS control problem of multicore mobile processors. By exchanging learning experiences and knowledge among multiple devices, DCQL increases the convergence rate of the learning algorithm and improves the quality of the derived power management policy. On each device, the 4-phase action selection strategy is applied for efficient exploration of the action space defined by various power settings. Experimental results on realistic applications show that DCQL can achieve 3.4%-8.0% energy reduction over various existing approaches while providing an average of 3.4× speedup over the state-of-the-art individual learning algorithm. We also show that the proposed approach can scale well with the number of cores through clustered DVFS control.

High-radix integrated silicon photonic switches promise ultrahigh bandwidth communications required by next generation data centers. To holistically explore the characteristics of high-radix integrated optical switches, this work systematically studies the latency, throughput and energy consumption, with detailed models and various system configurations. Three categories of space switches, blocking, rearrangeable non-blocking and strictly non-blocking switches, are investigated, together with one of the widely used wavelength switches, arrayed waveguide grating router (AWGR). The work paves the ways to automatically optimize high-radix integrated silicon photonic switches.

In recent years, optical interconnection networks have been proposed in order to achieve the ultrahigh bandwidth and low latency requirements for inter/intra-chip communication. In these optical interconection networks, series of basic optical elements are employed. Via these series of optical elements, the intrinsic crosstalk noise is generated. With a large scale of these optical elements, the signal-to-noise ratio (SNR) of an optical interconnect can be reduced by this crosstalk noise. In this paper, we utilize the adaptive power control (APC) to enhance the SNR under the crosstalk noise constraints. APC has been known to save energy and reduce power consumption. We apply this technique in one of the inter/intra-chip optical interconnect called I2CON. A new cluster design, namely the Beam cluster, is also introduced. Results have demonstrated that the APC can help to reduce crosstalk noise, hence, the overall SNR is improved. Comparison results have also indicated the further improvement of SNR in I2CON using Beam cluster when APC is applied.

Rapidly evolving embedded applications continuously demand more functionalities and better performance under tight energy and thermal budgets, and maintaining high energy efficiency has become a significant design challenge for mobile devices. Learning-based methods are adaptive to dynamic conditions and show great potential for runtime power management. However, with the ever-increasing complexity of both hardware and software, it is a challenging issue for a learning agent to explore the state-action space sufficiently and quickly find an efficient management policy. In this paper, we propose a reinforcement learning-based multi-device collaborative power management approach to address this issue. Multiple devices with different runtime conditions can acquire related knowledge during the learning process. Efficient knowledge sharing among these devices can potentially accelerate the learning process and improve the quality of the learned policies. We integrate the proposed method with dynamic voltage and frequency scaling on the multicore processors in mobile devices. Experimental results on realistic applications show that the collaborative power management can achieve up to a $7 \times$ speedup and 10% energy reduction compared with state-of-the-art learning-based approaches.

I. VLSI Systems: A Glance Into The Last Decades Since their inception in 1970s, VLSI systems have enabled several new technological capabilities and made them accessible to an unceasingly wider range of users, reaching a scale that has been exponentially increasing over the decades[1](seeFig. 1). Relentless integration of more complex systems has driven such remarkable evolution, as made possible by the inexorable miniaturization. As shown inFig. 1, more functionality has been crammed in a consistently smaller form factor, as exemplified by the physical volume shrinking of computers by 100 X/decade[2],[3]. At the same time, the energy per task has been decreasing at 10–100 X/decade, as shown inFig. 2, for several systems and system-on-chip subsystems[4]. This allowed packing more capabilities into the same power envelope, as generally observed in the electronic systems, even before the advent of the integrated circuit[5].

Rapidly evolving embedded applications continuously demand more functionality and better performance under tight energy and thermal budgets, and maintaining high energy efficiency has become a significant design challenge for mobile devices. Although learning-based runtime power management can adapt to dynamic conditions, it is a challenging issue to quickly find an efficient management policy under ever-increasing hardware and software complexity. In this work, we propose a multi-device collaborative power management approach to address this issue. The collaborative power management periodically shares knowledge among multiple devices to accelerate the learning process and improve the quality of learned policies. We integrate the proposed method with dynamic voltage and frequency scaling (DVFS) on the multicore processors in mobile devices. Experimental results on realistic applications show that the collaborative power management can achieve on average 8x speedup and 10% energy saving compared with state-of-the-art learning-based approaches.

The increasing demand for more computational power from scientific computing, big data processing, and machine learning is pushing the development of HPC (high-performance computing) systems. As the basic HPC building blocks, modularized server racks with a large number of multicore nodes are facing performance and energy efficiency challenges. This paper proposes RSON, an optical network for rack-scale computing systems. RSON connects processor cores, caches, local memories, and remote memories through a novel inter/intra-chip silicon photonic network architecture. We develop a low-latency scalable channel partition and low-power dynamic path priority control scheme for RSON. Experimental results show that RSON can help rack-scale computing systems achieve up to 6.8X higher performance under the same energy consumption than state-of-the-art systems under the latest APEX (application performance at extreme scale) benchmarks.

Maintaining high energy efficiency has become a critical design issue for high-performance systems. Many power management techniques have been proposed for the processor cores such as dynamic voltage and frequency scaling (DVFS). However, very few solutions consider the power losses suffered on the power delivery system (PDS), despite the fact that they have a significant impact on the system overall energy efficiency. With the explosive growth of system complexity and highly dynamic workloads variations, it is challenging to find the optimal power management policies which can effectively match the power delivery with the power consumption. To tackle the above problems, we propose a reinforcement learning-based power management scheme for manycore systems to jointly monitor and adjust both the PDS and the processor cores aiming to improve system overall energy efficiency. The learning agents distributed across power domains not only manage the power states of processor cores but also control the on/off states of on-chip VRs to proactively adapt to the workload variations. Experimental results with realistic applications show that when the proposed approach is applied to a large-scale system with a hybrid PDS, it lowers the system overall energy-delay-product (EDP) by 41% than a traditional monolithic DVFS approach with a bulky off-chip VR.

Battery-powered mobile devices have limited energy capacity, urging the development of efficient power management approaches. Reinforcement learning (RL) algorithms are adaptive to the changing environment and have been widely used for runtime power management. Recently, collaborative RL-based approaches have been explored to accelerate the learning process. Nonetheless, existing works usually require a cloud service provider to achieve centralized multi-device knowledge sharing, which suffers from the single point of failure and does not guarantee users' privacy. To address this issue, we propose a decentralized multi-device collaborative power management approach in this work, where devices directly share their knowledge with their trusted neighbors. Experimental results show that the proposed method can achieve an up to 21% energy reduction with a 4× speedup over the individual learning-based approach.

No abstract available.

The power delivery system (PDS), which plays a crucial role in guaranteeing the proper functionality of computing systems, has been a serious constraint on performance due to its significant power loss, especially for high-performance many-core processors. As the PDS design is usually optimized to provide power to the target chip at its best performance level, the energy efficiency can be notably degraded when the workload of the processors is highly dynamic. Therefore, dynamically adjusting the PDS to adapt to the run-time chip workloads are expected to be beneficial. In addition, the introduction of on-chip voltage regulators (VRs) has also significantly broadened the design space of PDS. In this paper, we present a workload-aware quantized power management scheme to dynamically manage the PDS in order to improve system energy efficiency. VRs at different stages are scheduled as online or offline based on chip workload estimation and prediction. The simulation results show that with the proposed scheme, a hybrid PDS with on/off-chip VRs can achieve 74.6% overall efficiency on average, 12.7% higher than a conventional PDS with one off-chip VR. The proposed scheme also shows its potential advantage in improving system energy efficiency with large-scale many-core processors and with more advanced processor technology nodes.

Energy-efficiency is becoming increasingly important to modern computing systems with multi-/many-core architectures. Dynamic Voltage and Frequency Scaling (DVFS), as an effective low-power technique, has been widely applied to improve energy-efficiency in commercial multi-core systems. However, due to the large number of cores and growing complexity of emerging applications, it is difficult to efficiently find a globally optimized voltage/frequency assignment at runtime. In order to improve the energy-efficiency for the overall multicore system, we propose an online DVFS control strategy based on core-level Modular Reinforcement Learning (MRL) to adaptively select appropriate operating frequencies for each individual core. Instead of focusing solely on the local core conditions, MRL is able to make comprehensive decisions by considering the running-states of multiple cores without incurring exponential memory cost which is necessary in traditional Monolithic Reinforcement Learning (RL). Experimental results on various realistic applications and different system scales show that the proposed approach improves up to 28% energy-efficiency compared to the recent individual-RL approach.

The memory wall problem is due to the imbalanced developments and separation of processors and memories. It is becoming acute as more and more processor cores are integrated into a single chip and demand higher memory bandwidth through limited chip pins. Optical memory interconnection network (OMIN) promises high bandwidth, bandwidth density, and energy efficiency, and can potentially alleviate the memory wall problem. In this paper, we propose an optical inter/intra-chip processor-memory communication architecture, called MOCA. Experimental results and analysis show that MOCA can significantly improve system performance and energy efficiency. For example, comparing to Hybrid Memory Cube (HMC), MOCA can speedup application execution time by 2.6x, reduce communication latency by 75%, and improve energy efficiency by 3.4x for 256-core processors in 7 nm technology.

The power delivery system (PDS) plays a crucial role of guaranteeing the proper functionality of many-core processors. However, as PDS is usually optimized to provide power to the target chip at its best performance level, its energy efficiency can be seriously degraded under highly dynamic workloads, making it a major source of system power losses. On-chip voltage regulators (VR), which are able to achieve fast and fine-grained power control, have been popular choices for PDS implementation and provided design opportunities for improving system energy efficiency. In this paper, we propose the adaptive Quantized Power Management (QPM) scheme to dynamically adjust the PDS with both on-chip and off-chip VRs based on run-time workloads. Experimental results on different applications show that QPM applied on a hybrid PDS with both on/off-chip voltage regulators (VR) achieves 74.1% average overall energy efficiency, 12.3% higher than the conventional PDS with single off-chip VR.

Exploring the design space of the memory hierarchy requires the use of effective methodologies, tools, and models to evaluate different parameter values. Reuse distance is of one of the locality models used in the design exploration and permits analytical cache miss estimation, program characterization, and synthetic trace generation. Unfortunately, the reuse distance is limited to a single locality granularity. Hence, it is not a suitable model for caches with hybrid line sizes, such as sectored caches, an increasingly popular choice for large caches. In this work, we introduce a generalization to the reuse distance, which is able to capture locality seen at multiple granularities. We refer to it as Hierarchical Reuse Distance (HRD). The proposed model has same profiling and synthesis complexity as the traditional reuse distance, and our results show that HRD reduces the average miss rate error on sectored caches by more than three times. In addition, it has superior characteristics in exploring multi-level caches with conventional single line size. For instance, our method increases the accuracy on L2 and L3 by a factor of 4 and converges three orders of magnitude faster.

In order to overcome limitations of traditional electronic interconnects in terms of power efficiency and bandwidth density, optical networks-on-chip (NoCs) based on 3D integrated silicon photonics have been proposed as an emerging on-chip communication architecture for multiprocessor systems-on-chip (MPSoCs) with large core counts. However, due to thermo-optic effects, wavelength-selective silicon photonic devices such as microresonators, which are widely used in optical NoCs, suffer from temperature-dependent wavelength shifts. As a result, on-chip temperature variations cause significant thermal-induced optical power loss which may counteract the power advantages of optical NoCs. To tackle this problem, in this work, we present a thermal-sensitive design and power optimization approach for a 3D torus-based optical NoC architecture. Based on an optical thermal modeling platform which models the thermal effect in optical NoCs from a system-level perspective, a thermal-sensitive routing algorithm is proposed for the 3D torus-based optical NoC to optimize its power consumption in the presence of on-chip temperature variations. Simulation results show that in an 8×8×2 3D torus-based optical NoC under a set of real applications, as compared with a matched 3D mesh-based optical NoC with traditional dimension order routing, the power consumption is reduced by 25% if thermal tuning for microresonators is not utilized, by 19% if thermal tuning is utilized for microresonators, and by 17% if athermal microresonators are used.

The design of power delivery system plays a crucial role in guaranteeing the proper functionality of many-core processor systems. The power loss suffered on power delivery has become a salient part of total power consumption, and the energy efficiency of a highly dynamic system has been significantly challenged. Being able to achieve a fast response time and multiple voltage domain control, on-chip voltage regulators (VRs) have become popular choices to enable fine-grain power management, which also enlarge the design space of power delivery systems. This paper analytically studies different power delivery system paradigms and power management schemes in terms of energy efficiency, area overhead, and power pin occupation. The analysis shows that compared to the conventional paradigm with off-chip VRs, hybrid paradigms with both on-chip and off-chip VRs are able to maintain high efficiency in a larger range of workloads, though they suffer from low efficiency at light workload. Employed with the quantized power management scheme, the hybrid paradigm can improve the system energy efficiency at light workload by a maximum of 136% compared to the traditional load balanced scheme. Besides this, the in-package (iP) hybrid paradigm further shows its advantage in reducing the physical overheads. The results reveal that at 120 W workload, it occupies only a 10.94% total footprint area or 39.07% power pins of that of the off-chip paradigm. We conclude that the iP hybrid paradigm achieves the best tradeoffs between efficiency, physical overhead, and realization of fine-grain power management.

As I start my second two-year term (2017–2018) as the Editor-in-Chief (EIC) of the IEEE Transactions on Very Large Scale Integration Systems (TVLSI), I wish the TVLSI readership a very happy new year and continued professional success. It gives me great pleasure to report on the state of the journal and our performance metrics. Over the past two years, TVLSI has seen a healthy increase in the number of submissions—from 687 in 2014 to 770 in 2015, and at the time of writing of this editorial, we are at 760 submissions for 2016. We expect the number of submissions for 2016 to cross 800 before the end of the year. TVLSI, therefore, continues to be the premier archival journal for university researchers and industry practitioners in the broad area of VLSI system design.

Recent advances in photonics technologies have made optical interconnection network an attractive option for computing systems from high-performance computers and data centers to automobiles and cellphones. Optical interconnection network promises ultra-high bandwidth, low latency, and great energy efficiency to alleviate the inter-rack, intra-rack, intraboard, and intra-chip communication bottlenecks in multiprocessor systems. Silicon-based photonics technologies piggyback onto developed silicon fabrication processes to provide viable and cost-effective solutions. Both industry and academia have invested significant efforts to develop and commercialize optical interconnection network technologies. This paper reviews the latest progresses and provides insights into the challenges and future developments.

The number of chip pins is limited due to the cost and reliability issues of sophisticated packages, and it is predicted that the chip pin count will be overstretched to satisfy the requirements of both power delivery and memory access. The gap between the achievable pin count and the demand will increase as the technology scales, due to the increasing computation resources and supply current. Pin reduction techniques are thus required for continued computing performance growth. In this article, we propose a chip pin constraint alleviation strategy, through on/off-chip power delivery system co-design, to effectively reduce the demand for power pins. An analytical model of a power delivery system, consisting of on/off-chip regulators and a power delivery network, is proposed to evaluate the influence of regulator design and package conduction loss. By combining this model with a multi-core processor model of performance and memory bandwidth requirements, we characterize the entire multi-core processor system to investigate the relationship between the chip pin constraint and performance in multi-core processor scaling and the effectiveness of our strategy. Experiments show that with the conventional power delivery system design, the chip pin constraint severely limits the performance growth as the technology scales. Using the on/off-chip power delivery system co-design, our strategy achieves a significant pin count reduction, for example, 31.3% at the 8nm technology node, compared to the conventional design with the same chip performance, while, provided with the same chip pin count, it is able to improve, by 35.0%, the chip performance at 8nm compared to the conventional design. For real applications of different parallelism, our strategy outperforms its counterpart, with a 23.7% performance improvement on average at the 8nm technology node.

Recently, interchip/intrachip optical interconnection networks have been proposed for ultrahigh-bandwidth and low-latency communications. These networks employ the microresonators (MRs) to modulate, direct, or detect the optical signal. However, utilized MRs suffer from intrinsic crosstalk noise and signal power loss, degrading the network efficiency via the signal-to-noise ratio (SNR). The amount of crosstalk noise and signal power loss may differ from network to network. Hence, there exists a need to systematically analyze the effect of the crosstalk noise and the power loss issues. In this paper, we have developed the analytical models considering both coherent and incoherent crosstalk for both the interchip and intrachip optical networks. The interchip/intrachip optical interconnection networks—the$\text{I}^{2}$CON—are analyzed as a case study. The quantitative results on the individual networks have demonstrated that the architectural design determines the impact of crosstalk on the SNR. We have also demonstrated that the optical interconnection networks with interchip/intrachip interconnects result in better bit error rate (BER) compared with that of only intrachip interconnect. Our analyses of the worst case can be utilized as a platform to compare the realistic performance among different optical interconnection networks via the degradation of SNR/BER and data bandwidth.

As transistor density continues to increase with the advent of nanotechnology, reliability issues raised by more frequently appeared soft errors are becoming even more critical to the next-generation multiprocessor systems. In this paper, we present a systematic approach to address the soft-error problem in multiprocessor system-on-chip with the consideration of system performance optimization. To guarantee the system correctness, a hardware-software collaborated approach is proposed to protect the processors from soft errors. Tiny hardware sensors are embedded in the processor cores to detect the soft errors, and the software-based rollback scheduling mechanisms are applied for error recovery. The protection costs on hardware duplication and software redundancy are effectively reduced. To optimize the system performance, a distributed control system is built on top of the on-chip communication network and collaboratively manages the entire chip for application execution. With the cluster-based task migration techniques, an efficient runtime task remapping and rescheduling algorithm is proposed to further mitigate the overheads induced by soft-error protection and to minimize the total performance degradation. The distributed control strategy makes the system more adaptable and flexible to the development of the next-generation hardware and software with larger scales. Extensive performance evaluations using SystemC-based cycle-accurate simulations on a set of real-world applications show that our approach has on average 49% performance improvement and 79.6% energy consumption reduction compared with the related state-of-the-art techniques, and hardware synthesis results show that our approach only introduces 2.9% chip area overheads.

Power gating (PG) is one of the most effective techniques to reduce the leakage power in multiprocessor system-on-chips (MPSoCs). However, the power-mode transition during the PG period of an individual processing unit (PU) will introduce serious power/ground (P/G) noise to the neighboring PUs. As technology scales, the P/G noise problem becomes a severe reliability threat to MPSoCs. At the same time, the increasing manufacturing process variations (PVs) also bring uncertainties to the P/G noise problem and make it difficult to predict and mitigate. To tackle this problem, in this paper, we analyze the PG-induced P/G noise in the presence of PVs and propose a hardware–software collaborated runtime technique to adaptively protect PUs from P/G noise. Sensor network-on-chip is used to gather noise information and coordinate different system components. An online PV-aware algorithm is developed to effectively decide the noise impact range and arrange protections for affected PUs based on the collected noise information. We evaluate the proposed technique through cycle-level Monte Carlo simulations of NoC-based MPSoCs in different scales. The experimental results on various realistic applications show that our technique could achieve comparable reliability to the most reliable static technique while improve on average 3.78%–29.5% the system energy efficiency and reduce 15.7%–70.4% the performance penalty on different MPSoC scales.

With the fast development of inter/intra-chip optical interconnects, the gap between the data rates of electrical interconnects and optical interconnects is continuously increasing. Electrical–optical (E-O) interfaces and optical–electrical (O-E) interfaces are a pair of components that convert data between parallel electrical interconnects and serial optical interconnects. This paper holistically models and analyzes E-O and O-E interfaces in terms of energy consumption, area, and latency. Traditional interfaces, where data are converted between parallel and serial ports by serializers and deserializers (SerDes), are studied. A new type of E-O and O-E interface, which serializes and deserializes data by optical weaving technologies, are proposed alongside. Traditional interfaces will become a bottleneck for the further development of optical interconnects in the near future because of the high energy consumption and large area of SerDes necessitating new technologies. Our analysis shows that optical weaving interfaces have a better overall performance than traditional interfaces. For example, if there are 64 parallel electrical interconnects and four optical wavelengths, optical weaving interfaces can achieve a 81.6% improvement in energy consumption and a 40.8% improvement in area, compared with traditional interfaces.

With the fast development of processor chips, power-efficient, high-bandwidth, and low-latency interchip interconnects become more and more important. Studies show that the bandwidth of traditional parallel interconnects with low I/O clock frequencies will become bottlenecks in the near future. To solve this problem, two types of high-bandwidth interchip interconnects are developed. Low-swing differential electrical interconnects have widely been used in high-speed I/O designs. On the other hand, optical interconnects promise high bandwidth, low latency, and could improve the chip pin performance for manycore processors. They are becoming potential alternatives for electrical interconnects. This paper systematically models these two types of interconnects in terms of crosstalk noises, attenuation, and receiver sensitivities. Based on the proposed models, we developed optical and electrical interfaces and links (OEIL) and an analysis tool for OEIL. The OEIL can be used to analyze the energy consumption, bandwidth density, and latency of interconnects. Analytical models are verified by the results of published experiments. It shows that the optical interconnects have much higher bandwidth densities than the electrical interconnects. With this feature, the optical interconnects can significantly reduce I/O pin count compared with the electrical interconnects. For example, they can save at least 92% signal pins when connecting chips more than 25 cm (10 in) apart. The energy consumption of optical interconnects is comparable with that of electrical interconnects, and the latency of polymer waveguide-based optical interconnects is 18% less than that of electrical interconnect.

Chip I/O pins are an increasingly limited resource and significantly affect the performance, power and cost of multicore processors. Optical interconnects promise low power and high bandwidth, and are potential alternatives to electrical interconnects. This work systematically developed a set of analytical models for electrical and optical interconnects to study their structures, receiver sensitivities, crosstalk noises, and attenuations. We verified the models by published implementation results. The analytical models quantitatively identified the advantages of optical interconnects in terms of bandwidth, energy consumption, and transmission distance. We showed that optical interconnects can significantly reduce chip pin counts. For example, compared to electrical interconnects, optical interconnects can save at least 92% signal pins when connecting chips more than 25 cm (10 inches) apart.

Design of power delivery system has great influence on the power management in many-core processor systems. Moving voltage regulators from off-chip to on-chip gains more and more interest in the power delivery system design, because it is able to provide fine-grained dynamic voltage scaling. Previous works are proposed to implement power efficient on-chip voltage regulators. It is important to analyze the characteristics of the entire power delivery system to explore the tradeoff between the promising properties and costs of employing on-chip voltage regulators, especially the on-chip buck converters. In this paper, we present a novel analysis and design optimization platform of power delivery system called power supply on-chip (PowerSoC). It employs an analytical model to provide an accurate and fast evaluation of important characteristics, e.g., power efficiency, output stability, and dynamic voltage scaling, for the entire power delivery system consisting of on-chip/off-chip buck converters and power delivery network. Based on our model, geometric programming is utilized to find the optimal design for different power delivery systems and explore the tradeoff of using on-chip converters. Compared with SPICE simulations, our model achieves a simulation time reduction of six to seven orders of magnitude within 5% model error for the characteristic evaluation of different power delivery systems. By using PowerSoC, various architectures of power delivery systems are optimized for power efficiency under constraints of output stability, area, etc. Simulation results show that the hybrid architecture, consisting of both on-chip and off-chip converters, achieves 1.0% power efficiency improvement and 66.4% area reduction of converters, compared to the conventional design. We conclude the hybrid architecture has potential for efficient dynamic voltage scaling, small area, and the adaptability of the change of power delivery network parasitic, but careful account for the overhead of on-chip converters is needed.

Optical networks-on-chip (ONoCs) have shown the potential to be substituted for electronic networks-on-chip (NoCs) to bring substantially higher bandwidth and more efficient power consumption in both on- and off-chip communication. However, basic optical devices, which are the key components in constructing ONoCs, experience inevitable crosstalk noise and power loss; the crosstalk noise from the basic devices accumulates in large-scale ONoCs and considerably hurts the signal-to-noise ratio (SNR) as well as restricts the network scalability. For the first time, this paper presents a formal system-level analytical approach to analyze the worst-case crosstalk noise and SNR in arbitrary fat-tree-based ONoCs. The analyses are performed hierarchically at the basic optical device level, then at the optical router level, and finally at the network level. A general 4$\,\times\,$4 optical router model is considered to enable the proposed method to be adaptable to fat-tree-based ONoCs using an arbitrary 4$\,\times\,$4 optical router. Utilizing the proposed general router model, the worst-case SNR link candidates in the network are determined. Moreover, we apply the proposed analyses to a case study of fat-tree-based ONoCs using an optical turnaround router (OTAR). Quantitative simulation results indicate low values of SNR and scalability constraints in large scale fat-tree-based ONoCs, which is due to the high power of crosstalk noise and power loss. For instance, in fat-tree-based ONoCs using the OTAR, when the injection laser power equals 0 dBm, the crosstalk noise power is higher than the signal power when the number of processor cores exceeds 128; when it is equal to 256, the signal power, crosstalk noise power, and SNR are${-}{17.3}$,${-}{11.9}$, and${-}{\rm 5.5}~{\rm dB}$, respectively.

Optical networks-on-chip (ONoCs) using wavelength-division multiplexing (WDM) technology have progressively attracted more and more attention for their use in tackling the high-power consumption and low bandwidth issues in growing metallic interconnection networks in multiprocessor systems-on-chip. However, the basic optical devices employed to construct WDM-based ONoCs are imperfect and suffer from inevitable power loss and crosstalk noise. Furthermore, when employing WDM, optical signals of various wavelengths can interfere with each other through different optical switching elements within the network, creating crosstalk noise. As a result, the crosstalk noise in large-scale WDM-based ONoCs accumulates and causes severe performance degradation, restricts the network scalability, and considerably attenuates the signal-to-noise ratio (SNR). In this paper, we systematically study and compare the worst case as well as the average crosstalk noise and SNR in three well-known optical interconnect architectures, mesh-based, folded-torus-based, and fat-tree-based ONoCs using WDM. The analytical models for the worst case and the average crosstalk noise and SNR in the different architectures are presented. Furthermore, the proposed analytical models are integrated into a newly developed crosstalk noise and loss analysis platform (CLAP) to analyze the crosstalk noise and SNR in WDM-based ONoCs of any network size using an arbitrary optical router. Utilizing CLAP, we compare the worst case as well as the average crosstalk noise and SNR in different WDM-based ONoC architectures. Furthermore, we indicate how the SNR changes in respect to variations in the number of optical wavelengths in use, the free-spectral range, and the microresonators$\boldsymbol {Q}$factor. The analyses’ results demonstrate that the crosstalk noise is of critical concern to WDM-based ONoCs: in the worst case, the crosstalk noise power exceeds the signal power in all three WDM-based ONoC architectures, even when the number of processor cores is small, e.g., 64.

By integrating multiple processing units (PUs) and memories on a single chip, multiprocessor system-on-chip (MPSoC) can provide higher performance per energy and lower cost per function to applications with growing complexity. On the other hand, shrinking feature sizes and reducing power supply voltages also make MPSoCs more susceptible to various reliability threats, such as power/ground (P/G) noises. Power gating is an effective technique to minimize leakage power. However, it also introduces significant P/G noises in MPSoCs. With significant area, power and performance overheads, traditional methods rely on reinforced circuits or fixed protection strategies to reduce P/G noises caused by power gating. In this paper, we propose a systematic approach to actively alleviating P/G noises using the parasitic capacitance of on-chip memories through sensor network on-chip (SENoC). We use the parasitic capacitance of on-chip memories as dynamic decoupling capacitance to suppress P/G noises and develop a detailed HSPICE model for related study. SENoC is developed to not only monitor and report P/G noises, but also coordinate PUs and memories to alleviate such transient threats at run time. Extensive evaluations show that compared with traditional method, our approach saves 12.6%–62.8% energy consumption and achieves 14.3%–69.8% performance improvement for different applications and MPSoCs with different scales. We implement the circuit details of our approach and show its low area and energy consumption overheads.

Manycore processor system is becoming an attractive platform for applications seeking both high performance and high energy efficiency. However, huge communication demands among cores, large power density, and low process yield will be three significant limitations for the scalability of future manycore processors. Breaking a large chip into multiple smaller ones can alleviate the problems of power density and yield, but would worsen the problem of communication efficiency due to the limited off-chip bandwidth. In response, we propose an inter/intra-chip optical network, which will not only fulfill the intra-chip communication requirements but also address the inter-chip communication, by exploiting the advantages of optical links with high bandwidth and energy efficiency. The network is composed of an inter-chip subnetwork and multiple intra-chip subnetworks, and the subnetworks closely coordinate with each other to balance the traffic. The proposed network effectively explores the distinctive properties of optical signals and photonic devices, and dynamically partitions each data channel into multiple sections. Each section can be utilized independently to boost performance as well as reduce energy consumption. Simulation results show that our network can achieve higher throughput with lower power consumption than alternative designs under most of synthetic traffics and real applications.

Design of power delivery system has great influence on the power management in many-core processor systems. Moving voltage regulators from off-chip to on-chip gains more and more interest in the power delivery system design, because it is able to provide fast voltage scaling and multiple power domains. Previous works are proposed to implement power efficient on-chip regulators. It is also important to analyze the characteristics of the entire power delivery system to explore the tradeoff between the promising properties and costs of employing on-chip regulators. In this work, we develop an analytical model to evaluate important characteristics of the power delivery system, including on-chip/off-chip voltage regulators and the passive on-chip/on-board parasitic. Compared with SPICE simulations, our model achieves a fast system-level evaluation with comparable accuracy. Based on the model, geometric programming is utilized to find the optimal power efficiency of different architectures of power delivery systems under constraints of output voltage stability and area. Experiments show that compared with the conventional architecture using off-chip regulators, the hybrid one using both on-chip and off-chip voltage regulators achieves 1.0% power efficiency improvement and 68% area reduction of voltage regulators on average. We conclude that the hybrid architecture has potential for high power efficiency and small area at heavy workload, but careful account for the overhead of on-chip regulators is needed.

Basic photonic devices in inter- and intra-chip optical networks suffer from inevitable power loss and crosstalk noise. Incoherent crosstalk introduces quick power fluctuations, while coherent crosstalk varies the optical power of the optical signal in optical interconnection networks (OINs). As a result, the accumulative crosstalk in large scale OINs considerably hurts the signal-to-noise ratio (SNR) and imposes high power penalties. In this work, we aim at studying the worst-case incoherent and coherent crosstalk in OINs at the system level. The proposed analytical models are integrated into a newly developed crosstalk and loss analysis platform, called CLAP, to facilitate the SNR analyses in arbitrary OINs.

The performance of a multiprocessor system-on-chip (MPSoC) is determined not only by the performance of its processing cores and memories, but also by how efficiently they collaborate with one another. It is the MPSoCs communication architecture which determines the collaboration efficiency. The migration towards MPSoCs is propelled by the shrinking feature sizes in each generation of process technology. On the one hand, smaller transistors allow for more processor cores and memories on a single chip and result in more on-chip computations as well as communications. On the other hand, reducing feature sizes makes on-chip communication more difficult. The International Roadmap for Semiconductors (ITRS) shows that the latency of metallic interconnects increases exponentially as feature sizes decrease. On-chip communication using metallic interconnects will need more than one clock cycle to send information from sources to destinations. Moreover, metallic interconnects consume a significant amount of power. Studies shows that global metallic interconnects could consume kilowatts of power to achieve required communication bandwidth by 2020 [1].

As transistor density continues to increase with the advent of nanotechnology, reliability issues raised by the more frequent appearance of soft errors are becoming critical for future embedded multiprocessor systems design. State-of-the-art techniques for soft error protections targeting multiprocessor systems result either high chip cost and area overhead or high performance degradation and energy consumption, and do not fulfill the increasing requirements for high performance and dependability. In this article we present a systematic approach, that is, the Sensor Networks-on-Chip (SENoC), to collaboratively and efficiently manage on-chip applications and overcome reliability threats to Multiprocessor Systems-on-Chip (MPSoC). A hardware-software collaborative approach is proposed to solve soft error problems: a hardware-based on-chip sensor network is built for soft error detection, and a software-based recovery mechanism is applied for soft error correction. A two-step scheduling scheme is presented for reliable application and chip management, combining an off-line static optimization stage for application performance maximization and an online lightweight dynamic adjustment stage to handle runtime variations and exceptions. This strategy introduces only trivial overhead on hardware design and much lower overhead on software control and execution, and hence performance degradation and energy consumption is greatly reduced. We build a cycle-accurate simulator using SystemC, and verify the effectiveness of our technique by comparing performance with related techniques on several real-world applications.

Chip multiprocessor (CMP) is becoming an attractive platform for applications seeking both high performance and high energy efficiency. In large-scale CMPs, the communication efficiency among cores is crucial for the overall system performance and energy consumption. In this article, we propose a ring-based optical network-on-chip, called SUOR, to fulfill the communication requirement of CMPs. SUOR effectively explores the distinctive properties of optical signals and photonic devices, and dynamically partitions each data channel into multiple sections. Each section can be utilized independently to boost performance as well as reduce energy consumption. We develop a set of distributed control protocols and algorithms for SUOR, but physically allocate the corresponding cluster agents close to each other to benefit from the strengths of optical interconnects at long distances as well as electrical interconnects at short distances. Simulation results show that SUOR outperforms the alternative optical networks under a wide range of traffic patterns. For example, compared with MWSR design, SUOR achieves 2.58× throughput as well as saves 64% energy consumption on average in a 256-core CMP. Compared with MWMR design, SUOR achieves 1.52× throughput and reduces 73% energy consumption on average.

Chip multiprocessor (CMP) is becoming increasingly popular in the processor industry. Efficient network-on-chip (NoC) that has similar performance to the processor cores is important in CMP design. Fat-tree-based on-chip network has many advantages over traditional mesh or torus-based networks in terms of throughput, power efficiency, and latency. It has a bright future in the development of CMP. However, the floorplan design of the fat-tree-based NoC is very challenging because of the complexity of topology. There are a large number of crossings and long interconnects, which cause severe performance degradation in the network. In electronic NoCs, the parasitic capacitance and inductance will be significant. In optical ones, large crosstalk noise and power loss will be introduced. The novel contribution of this paper is to propose a method to optimize the fat-tree floorplan, which can effectively reduce the number of crossings and minimize the interconnect length. Two types of floorplans are proposed, which could be applied to fat-tree-based networks of arbitrary size. Compared with the traditional one, our floorplans could reduce more than 87% of the crossings. Since the traversal distance for signals is related to the aspect ratio of the processor cores, we also present a method to calculate the optimum aspect ratio of the processor cores to minimize the traversal distance.

Photonic devices are widely used in optical networks-on-chip (ONoCs) and suffer from crosstalk noise. The accumulative crosstalk noise in large scale ONoCs diminishes the signal-to-noise ratio (SNR), causes severe performance degradation, and constrains the network scalability. For the first time, this paper systematically analyzes and models the worst-case crosstalk noise and SNR in folded-torus-based ONoCs. Formal analytical models for the worst-case crosstalk noise and SNR are presented. The crosstalk noise analysis is hierarchically performed at the basic photonic device level, then at the optical router level, and finally at the network level. We consider a general 5$\,\times\,$5 optical router model to enable crosstalk noise and SNR analyses in folded-torus-based ONoCs using an arbitrary 5$\,\times\,$5 optical router. Using the general optical router model, the worst-case SNR link candidates, which restrict the network scalability, are found. Also, we present a novel crosstalk noise and loss analysis platform, called CLAP, which can analyze the crosstalk noise and SNR of arbitrary ONoCs. Case studies of optimized crossbar and Crux optical routers using recent photonic device parameters are presented. Moreover, we compare the worst-case crosstalk noise and SNR in folded-torus-based and mesh-based ONoCs using optimized crossbar and Crux optical routers. The quantitative simulation results show the critical behavior of crosstalk noise in large scale ONoCs. For example, in folded-torus-based ONoCs using the Crux optical router, the noise power exceeds the signal power for network sizes larger than 12$\,\times\,$12; when the network size is 20$\,\times\,$20 and the injection signal power equals 0 dBm, the signal power and noise power are${-}{\rm 9.4}~{\rm dBm}$and${-}{\rm 6.1}~{\rm dBm}$, respectively.

Multiprocessor systems-on-chip show a trend toward integration of tens and hundreds of processor cores on a single chip. With the development of silicon photonics for short-haul optical communication, wavelength division multiplexing (WDM)-based optical networks-on-chip (ONoCs) are emerging on-chip communication architectures that can potentially offer high bandwidth and power efficiency. Thermal sensitivity of photonic devices is one of the main concerns about the on-chip optical interconnects. We systematically modeled thermal effects in optical links in WDM-based ONoCs. Based on the proposed thermal models, we developed OTemp, an optical thermal effect modeling platform for optical links in both WDM-based ONoCs and single-wavelength ONoCs. OTemp can be used to simulate the power consumption as well as optical power loss for optical links under temperature variations. We use case studies to quantitatively analyze the worst-case power consumption for one wavelength in an eight-wavelength WDM-based optical link under different configurations of low-temperature-dependence techniques. Results show that the worst-case power consumption increases dramatically with on-chip temperature variations. Thermal-based adjustment and optimal device settings can help reduce power consumption under temperature variations. Assume that off-chip vertical-cavity surface-emitting lasers are used as the laser source with WDM channel spacing of 1 nm, if we use thermal-based adjustment with guard rings for channel remapping, the worst-case total power consumption is 6.7 pJ/bit under the maximum temperature variation of 60 °C; larger channel spacing would result in a larger worst-case power consumption in this case. If we use thermal-based adjustment without channel remapping, the worst-case total power consumption is around 9.8 pJ/bit under the maximum temperature variation of 60 °C; in this case, the worst-case power consumption would benefit from a larger channel spacing.

As modern computing systems become increasingly complex, communication efficiency among and inside chips has become as important as the computation speeds of individual processing cores. Traditionally, to maximize design flexibility, interchip and intrachip communication architectures are separately designed under different constraints. Jointly designing communication architectures for both interchip and intrachip communication could, however, potentially yield better solutions. In this paper, we present a unified inter/intrachip optical network, called UNION, for chip multiprocessors (CMPs). UNION is based on recent progresses in nanophotonic technologies. It connects not only cores on a single CMP, but also multiple CMPs in a system. UNION employs a hierarchical optical network to separate interchip communication traffic from intrachip communication traffic. It fully utilizes a single optical network to transmit both payload and control packets. The network controller on each CMP not only manages intrachip communications, but also collaborates with each other to facilitate interchip communications. We compared UNION with a matched electrical counterpart in 45-nm process. Simulation results for eight real CMP applications show that on average UNION improves CMP performance by 3× while reducing 88% of network energy consumption.

By integrating multiple processing units and memories on a single chip, multiprocessor system-on-chip (MPSoC) can provide higher performance per energy and lower cost per function to applications with growing complexity. In order to maintain the power budget, power gating technique is widely used to reduce the leakage power. However, it will introduce significant power/ground (P/G) noises, and threat the reliability of MPSoCs. With significant area, power and performance overheads, traditional methods rely on reinforced circuits or fixed protection strategies to reduce P/G noises caused by power gating. In this paper, we propose a systematic approach to actively alleviating P/G noises using the parasitic capacitance of on-chip memories through sensor network on-chip (SENoC). We utilize the parasitic capacitance of on-chip memories as dynamic decoupling capacitance to suppress P/G noises and develop a detailed Hspice model for related study. SENoC is developed to not only monitor and report P/G noises but also coordinate processing units and memories to alleviate such transient threats at run time. Extensive evaluations show that compared with traditional methods, our approach saves 11.7% to 62.2% energy consumption and achieves 13.3% to 69.3% performance improvement for different applications and MPSoCs with different scales. We implement the circuit details of our approach and show its low area and energy consumption overheads.

Network-on-chip (NoC) can improve the performance, power efficiency, and scalability of multiprocessor system-on-chip (MPSoC). Optical NoCs, which are based on CMOS-compatible optical waveguides and microresonators, have significant bandwidth and power advantages over metallic interconnects. We propose a low-cost mesh-based hybrid optical-electronic NoC, HOME, with non-blocking 5×5, 4×4 and 3×3 optical switching fabrics. We systematically analyzed the key characteristics of HOME for a 64-core MPSoC in 45nm under different traffic conditions. Besides, we quantitatively analyzed the thermal effects in the 64-core HOME under temperature variations.

Optical networks-on-chip (ONoCs) are emerging communication architectures that can potentially offer ultrahigh communication bandwidth and low latency to multiprocessor systems-on-chip (MPSoCs). In addition to ONoC architectures, 3-D integrated technologies offer an opportunity to continue performance improvements with higher integration densities. In this paper, we present a 3-D mesh-based ONoC for MPSoCs, and new low-cost nonblocking 4$\,\times\,$4, 5$\,\times\,$5, 6$\,\times\,$6, and 7$\,\times\,$7 optical routers for dimension-order routing in the 3-D mesh-based ONoC. Besides, we propose an optimized floorplan for the 3-D mesh-based ONoC. The floorplan follows the regular 3-D mesh topology but implements all optical routers in a single optical layer. The floorplan is optimized to minimize the number of extra waveguide crossings caused when merging the 3-D ONoC to one optical layer. Based on a set of real applications and uniform traffic pattern, we develop a SystemC-based cycle-accurate NoC simulator and compare the 3-D mesh-based ONoC with the matched 2-D mesh-based ONoC and 2-D electronic NoC for performance and energy efficiency. Additionally, we quantitatively analyze thermal effects on the 3-D 8$\,\times\,$8$\,\times\,$2 mesh-based ONoC.

Reducing feature sizes and power supply voltage allows integrating more processing units (PUs) on multiprocessor system on chip (MPSoC) to satisfy the increasing demands of applications. However, it also makes MPSoC more susceptible to various reliability threats, such as high temperature and power/ground (P/G) noise. As the scale and complexity of MPSoC continuously increase, monitoring and mitigating reliability threats at runtime could offer better performance, scalability, and flexibility for MPSoC designs. In this paper, we propose a systematic approach, on-chip sensor network (SENoC), to collaboratively predict, detect, report, and alleviate runtime threats in MPSoC. SENoC not only detects reliability threats and shares related information among PUs, but also plans and coordinates the reactions of related PUs in MPSoC. SENoC is used to alleviate the impacts of simultaneous switching noise in MPSoC's P/G network during power gating. Based on the detailed noise behaviors under different scenarios derived by our circuit-level MPSoC P/G noise simulation and analysis platform, simulation results show that SENoC helps to achieve on average 26.2 percent performance improvement compared with the traditional stop-go method with 1.4 percent area overhead in an 8*8-core MPSoC in 45 nm. An architecture-level cycle-accurate simulator based on SystemC is implemented to study the performance of the proposed SENoC. By applying sophisticated scheduling techniques to optimize the total system performance, a higher performance improvement of 43.5 percent is achieved for a set of real-life applications.

Crosstalk noise is an intrinsic characteristic as well as a potential issue of photonic devices. In large scale optical networks-on-chips (ONoCs), crosstalk noise could cause severe performance degradation and prevent ONoC from communicating properly. The novel contribution of this paper is the systematical modeling and analysis of the crosstalk noise and the signal-to-noise ratio (SNR) of optical routers and mesh-based ONoCs using a formal method. Formal analytical models for the worst-case crosstalk noise and minimum SNR in mesh-based ONoCs are presented. The crosstalk analysis is performed at device, router, and network levels. A general 5$\,\times\,$5 optical router model is proposed for router level analysis. The minimum SNR optical link candidates, which constrain the scalability of mesh-based ONoCs, are identified. It is also shown that symmetric mesh-based ONoCs have the best SNR performance. The presented formal analyses can be easily applied to other optical routers and mesh-based ONoCs. Finally, we present case studies of mesh-based ONoCs using the optimized crossbar and Crux optical routers to evaluate the proposed formal method. We find that crosstalk noise can significantly limit the scalability of mesh-based ONoCs. For example, when the mesh-based ONoC size, using optimized crossbar, is larger than 8$\,\times\,$8, the optical signal power is smaller than the crosstalk noise power; when the network size is 16$\,\times\,$16 and the input power is 0 dBm, in the worst-case, the signal power is${-}{\rm 24.9}~{\rm dBm}$and the crosstalk noise power is${-}{\rm 11}~{\rm dBm}$.

The performance of multiprocessor systems, such as chip multiprocessors (CMPs), is determined not only by individual processor performance, but also by how efficiently the processors collaborate with one another. It is the communication architecture that determines the collaboration efficiency on the hardware side. Optical networks-on-chip (ONoCs) are emerging communication architectures that can potentially offer ultra-high communication bandwidth and low latency to multiprocessor systems. Thermal sensitivity is an intrinsic characteristic of photonic devices used by ONoCs as well as a potential issue. This paper systematically modeled and quantitatively analyzed the thermal effects in ONoCs. We used an 8$\times$8 mesh-based ONoC as a case study and evaluated the impacts of thermal effects in the average power efficiency for real MPSoC applications. We revealed three important factors regarding ONoC power efficiency under temperature variations, and proposed several techniques to reduce the temperature sensitivity of ONoCs. These techniques include the optimal initial setting of microresonator resonant wavelength, increasing the 3-dB bandwidth of optical switching elements by parallel coupling multiple microresonators, and the use of passive-routing optical router Crux to minimize the number of switching stages in mesh-based ONoCs. We gave a mathematical analysis of periodically parallel coupling of multiple microresonators and show that the 3-dB bandwidth of optical switching elements can be widened nearly linearly with the ring number. Evaluation results for different real MPSoC applications show that, on the basis of thermal tuning, the optimal device setting improves the average power efficiency by 54% to 1.2 pJ/bit when chip temperature reaches 85$^{\circ}$C. The findings in this paper can help support the further development of this emerging technology.

Networks-on-chip (NoCs) are emerging as a key on-chip communication architecture for multiprocessor systems-on-chip (MPSoCs). Optical communication technologies are introduced to NoCs in order to empower ultra-high bandwidth with low power consumption. However, in existing optical NoCs, communication locality is poorly supported, and the importance of floorplanning is overlooked. These significantly limit the power efficiency and performance of optical NoCs. In this work, we address these issues and propose a torus-based hierarchical hybrid optical-electronic NoC, called THOE. THOE takes advantage of both electrical and optical routers and interconnects in a hierarchical manner. It employs several new techniques including floorplan optimization, an adaptive power control mechanism, low-latency control protocols, and hybrid optical-electrical routers with a low-power optical switching fabric. Both of the unfolded and folded torus topologies are explored for THOE. Based on a set of real MPSoC applications, we compared THOE with a typical torus-based optical NoC as well as a torus-based electronic NoC in 45nm on a 256-core MPSoC, using a SystemC-based cycle-accurate NoC simulator. Compared with the matched electronic torus-based NoC, THOE achieves 2.46X performance and 1.51X network switching capacity utilization, with 84% less energy consumption. Compared with the optical torus-based NoC, THOE achieves 4.71X performance and 3.05X network switching capacity utilization, while reducing 99% of energy consumption. Besides real MPSoC applications, a uniform traffic pattern is also used to show the average packet delay and network throughput of THOE. Regarding hardware cost, THOE reduces 75% of laser sources and half of optical receivers compared with the optical torus-based NoC.

Subpixel-based down-sampling is a method that can potentially improve the apparent resolution of a down-scaled image by controlling individual subpixels rather than pixels. However, the increased luminance resolution often comes at the price of chrominance distortion. A major challenge is to suppress color fringing artifacts while maintaining sharpness. In [1], we proposed a novel human visual quality based (HVS) subpixel downsampling method. In this paper, we propose a hardware-friendly subpixel based downsampling scheme based on our previous work which can achieve similar performance as [1] but eliminate all floating point operations with limited bandwidth and much better performance than Direct Pixel based Downsampling (DPD) and Pixel-based downsampling with Anti-aliasing Filter (PDAF). We further propose a hardware architecture for our subpixel downsampling method which is highly data re-useable. We are the first few, if not the first, to implement subpixel downsampling method to video by hardware. The design is implemented with TSMC 0.18um CMOS technology and costs 244k gates. At a clock frequency of 63 MHz, the architecture achieves real-time 1920×1080 subpixel downsampling at 30fps.

To achieve superior performance, rate-distortion optimized motion estimation (ME) for variable block size (RDO VBSME) is often used in state-of-the-art video coding systems such as the H.264 JM software. However, the complexity of RDO-VBSME is very high both for software and hardware implementations. In this paper, we propose a hardware-friendly ME algorithm called RDOMFS with a novel hardware-friendly rate-distortion (RD)-like cost function, and a hardware-friendly modified motion vector predictor. Simulation results suggest that the proposed RDOMFS can achieve essentially the same RD performance as RDO-VBSME in JM. We also propose a matching hardware architecture with a novel Smart Snake Scanning order which can achieve very high data re-use ratio and data throughout. It is also reconfigurable because it can achieve variable data re-use ratio and can process variable frame size. The design is implemented with TSMC 0.18 μm CMOS technology and costs 103 k gates. At a clock frequency of 63 MHz, the architecture achieves real-time 1920 × 1080 RDO-VBSME at 30 frames/s. At a maximum clock frequency of 250 MHz, it can process 4096 × 2160 at 30 frames/s.

Task graph scheduling on multiprocessor systems is a representative multiprocessor scheduling problem. A solution to this problem consists of the mapping of tasks to processors and the scheduling of tasks on each processor. Optimal solution can be obtained by exploring the entire design space of all possible mapping and scheduling choices. Since the problem is NP-hard, scalability becomes the main concern in solving the problem optimally. In this paper, a SAT-based optimization framework is proposed to address this problem, in which SAT solver is enhanced by integrating with a scheduling analysis tool in a branch and bound manner to prune the solution space efficiently. Performance evaluation results show that our technique has average performance improvement in more than an order of magnitude compared to state-of-the-art techniques. We further build a cycle-accurate network-on-chip simulator based on SystemC to verify the effectiveness of the proposed technique on realistic multiprocessor systems.

Shrinking the feature size allows more and better functions on a single chip. However, it makes multiprocessor system-on-chip (MPSoC) more susceptible to various reliability threats. Power supply noise is a major reliability problem faced by low power MPSoCs using power gating techniques. Powering on and off a processing unit in MPSoCs will induce large power/ground (P/G) noise and can cause timing divergence and even functional errors in surrounding processing units. Previous work on resilient architectures mainly focused on power/thermal management and neglected the important side-effect: P/G noise induced by power gating. In this paper, for the first time, we formulate a task scheduling problem with the consideration of P/G noise based on our detailed P/G noise analysis platform for MPSoC. Two efficient algorithms are proposed to reduce noise protection penalty and improve MPSoC performance. Our experiments show that both simulated annealing and heuristic algorithms can achieve on average 25% performance improvement together with up to 80% noise protection penalty saving compared with the conservative stop-go method for short tasks (shorter than 20 K clock cycles). For longer tasks up to 200 K clock cycles, the performance improvement of our methods will become relatively low. However, we can still achieve at least 35.2% noise protection penalty saving. Furthermore, a lightweight online adjustment strategy accompanying the offline scheduling method is proposed to adapt to runtime variations and improve reliability.

Crosstalk noise is an intrinsic characteristic of photonic devices used by optical networks-on-chip (ONoCs) as well as a potential issue. For the first time, this paper analyzed and modeled the crosstalk noise, signal-to-noise ratio (SNR), and bit error rate (BER) of optical routers and ONoCs. The analytical models for crosstalk noise, minimum SNR, and maximum BER in meshbased ONoCs are presented. An automated crosstalk analyzer for optical routers is developed. We find that crosstalk noise significantly limits the scalability of ONoCs. For example, due to crosstalk noise, the maximum BER is 10-3 on the 8x8 mesh-based ONoC using an optimized crossbar-based optical router. To achieve the BER of 10-9 for reliable transmissions, the maximum ONoC size is 6x6. A novel compact high-SNR optical router is proposed to improve the maximum ONoC size to 8x8.

Motion Estimation (ME) is the most computationally intensive part in the whole video compression process. The ME algorithms can be divided into full search ME (FS) and fast ME (FME). The FS is not suitable for high definition (HD) frame size videos because its relevant high computation load and hard to deal with complex motions in limited search range. A lot of FME algorithms have been proposed which can significantly reduce the computation load compared to FS. Though many kinds of hardware implementations of ME have been proposed, almost all of them fail to consider about the motion vector field (MVF) coherence and rate-distortion (RD) cost which have significant impact to the coding efficiency. In this paper, we propose a hardware friendly ME algorithm and corresponding highly data reusable hardware architecture. Simulation results show that the proposed ME algorithm performs better RD performance than conventional FME algorithm. The proposed reconfigurable ME hardware is implemented in VHDL and mapped to a low cost Xilinx XC3S1500 FPGA. It works at 100MHz and is capable to process 1920 × 1080 of 30fps video format in real time and have very high data reuse ratio.

Reducing feature sizes and power supply voltage allows integrating more processing units (PUs) on multiprocessor system-on-chip (MPSoC) to satisfy the increasing demands of applications. However, it also makes MPSoC more susceptible to various reliability threats, such as high temperature and power/ground (P/G) noise. As the scale and complexity of MPSoC continuously increase, monitoring and mitigating reliability threats at run time could offer better performance, scalability, and flexibility for MPSoC designs. In this paper, we propose a systematic approach, on-chip sensor network (SENoC), to collaboratively detect, report, and alleviate run-time threats in MPSoC. SENoC not only detects reliability threats and shares related information among PUs, but also plans and coordinates the reactions of related PUs in MPSoC. SENoC is used and explained in our case study to alleviate the impacts of simultaneous switching noise in MPSoC's P/G network during power gating. Based on the detailed noise behaviors under different scenarios derived by our circuit-level MPSoC P/G noise simulation and analysis platform, simulation results show that SENoC helps to achieve on average 26.12% performance improvement compared with the traditional stop-go method with 1.4% area overhead in an 8*8-core MPSoC in 45nm.

Multiprocessor system-on-chip (MPSoC) is an attractive platform for high-performance applications. Networks-on-chip (NoCs) can improve the on-chip communication bandwidth of MPSoCs. However, traditional metallic interconnects consume significant amount of power to deliver even higher communication bandwidth required in the near future. Optical NoCs are based on CMOS-compatible optical waveguides and microresonators, and promise significant bandwidth and power advantages. This paper proposes a fat tree-based optical NoC (FONoC) including its topology, floorplan, protocols, and a low-power and low-cost optical router, optical turnaround router (OTAR). Different from other optical NoCs, FONoC does not require building a separate electronic NoC for network control. It carries both payload data and network control data on the same optical network, while using circuit switching for the former and packet switching for the latter. The FONoC protocols are designed to minimize network control data and the related power consumption. An optimized turnaround routing algorithm is designed to utilize the low-power feature of OTAR, which can passively route packets without powering on any microresonator in 40% of all cases. Comparing with other optical routers, OTAR has the lowest optical power loss and uses the lowest number of microresonators. An analytical model is developed to characterize the power consumption of FONoC. We compare the power consumption of FONoC with a matched electronic NoC in 45 nm, and show that FONoC can save 87% power comparing with the electronic NoC on a 64-core MPSoC. We simulate the FONoC for the 64-core MPSoC and show the end-to-end delay and network throughput under different offered loads and packet sizes.

Multiple description coding (MDC) is a source coding technique that exploits path diversity to solve packet losses over error-prone channels. In this paper, we propose an improved drift-free multi-state MDC method based on H.264/AVC coding scheme. At the encoder side, we compress original video into multiple independent H.264 streams with different coding parameters, which can help us to control correlations between the descriptions. At the decoder side, each description is considered as a noisy observation of the original video, and a linear minimum mean square error (LMMSE) based merge algorithm is proposed to combine the descriptions. Experimental results show that the proposed algorithm can achieve better coding efficiency and visual quality than temporary MDC present in [1]. The error resilience ability is also improved by the fact that each frame is coded twice with different parameters.

With the help of HW/SW codesign, system-on-chip (SoC) can effectively reduce cost, improve reliability, and produce versatile products. The growing complexity of SoC designs makes on-chip communication subsystem design as important as computation subsystem design. While a number of codesign methodologies have been proposed for on-chip computation subsystems, many works are needed for on-chip communication subsystems. This paper proposes application-specific networks-on-chip (ASNoC) and its design methodology. ASNoC is used for two high-performance SoC applications. The methodology (1) can automatically generate optimized ASNoC for different applications, (2) can generate a corresponding distributed shared memory along with an ASNoC, (3) can use both recorded and statistical communication traces for cycle-accurate performance analysis, (4) is based on standardized network component library and floorplan to estimate power and area, (5) adapts an industrial-grade network modeling and simulation environment, OPNET, which makes the methodology ready to use, and (6) can be easily integrated into current HW/SW codesign flow. Using the methodology, ASNoC is generated for a H.264 HDTV decoder SoC and Smart Camera SoC. ASNoC and 2D mesh networks-on-chip are compared in performance, power, and area in detail. The comparison results show that ASNoC provide substantial improvements in power, performance, and cost compared to 2D mesh networks-on-chip. In the H.264 HDTV decoder SoC, ASNoC uses 39% less power, 59% less silicon area, 74% less metal area, 63% less switch capacity, and 69% less interconnection capacity to achieve 2X performance compared to 2D mesh networks-on-chip.

This paper studied an H. 264 HDTV decoder on two multiprocessor system-on-chip architectures. Two types of networks-on-chip, the RAW network and the application specific networks-on-chip, were used. Regular-topology networks-on-chip (mesh, torus, and fat tree) have been proposed. However, we showed in this paper that the application-specific networks-on-chip provided substantial improvements in power, performance, and cost compared to regular-topology networks-on-chip. We measured the power, performance, area, total switch and link capacity, and switch and link utilization based on floorplans and circuit designs. Measurement results showed th at the application-specific networks-on-chip was both faster in absolute terms and more efficient. The application-specific networks-on-chip used 39% less power, 59% less silicon area, 74% less metal area, 63% less switch capacity, and 69% less link capacity to achieve 2X performance compared to the RAW network.

In this paper we study bus-based and switch-based on-chip networks for an embedded video application, the smart camera SoC (system on chip). We analyze network performance and overall system performance in detail. We explore system performance using crossbars with different sizes, fixed size but different numbers of ports, and different numbers of shared memories. We find that network is a performance bottleneck in our design, and the system using an optimized NoC can outperform one using a bus by 132%. Our simulations are based upon recorded real communication traces, which give more accurate system performance. Our study finds that for the Smart Camera system, a 16-bit/port 3/spl times/3 crossbar with two shared memories shows 85.7% performance improvement over the bus-based model and also has less maximum network throughput than the bus-based model. This design example illustrates a methodology to quickly and accurately estimate the performance of NoC's at architecture level.

This paper presents methods for optimizing application-specific networks-on-chips (NoCs). We show that wave pipelining provides more energy efficient data transport than non-wave pipelined communication. We observe 52% energy saving, 60% transistor area saving, and 1.7 times speedup by using wave pipelining in simulation. Wave pipelining is particularly well suited to networks-on-chips because the networkes structured interconnection provides better delay control. Our analysis shows how designers can tune their network to the requirements of the application by choosing a design point along area/performance or area/energy curves.