The demand for real-time streaming graph analysis has grown significantly, as hundreds of thousands of updates come every second. Monotonic graph algorithms such as Shortest Path are widely used in real-time analytics, but there are two bottlenecks that limit their performance, specially on planar graphs. One is massive redundant data accesses due to irregular state propagations and the other is high memory latency caused by irregular data accesses. We observe that existing systems mainly focus on general-purpose graph algorithms. If the properties of specific graph algorithms are exploited, the analysis performance can be further improved. Moreover, these systems typically tackle these two bottlenecks separately through either software or hardware mechanisms, but not both. However, both bottlenecks need to be addressed simultaneously in real scenarios such as road navigation. This article proposes WSGraph, a software-hardware co-design framework for high-performance streaming graph processing. WSGraph tackles these two challenges by enforcing regularized processing orders and enabling precise data prefetching. Specifically, at the software level, WSGraph integrates a priority-based work scheduler with sliding-window bucket mapping scheme to regulate state propagations, thereby drastically reducing redundant data accesses. At the hardware level, WSGraph incorporates a lightweight in-core Proactive Data Engine (PDE). By exploiting intra-vertex access regularity, the PDE accurately prefetches relevant graph data to effectively hide the high latency of irregular memory accesses. Experimental results demonstrate that WSGraph achieves significant performance improvements over existing systems. Compared with the state-of-the-art software system KickStarter, WSGraph gains a 2.13× speedup primarily by reducing graph data accesses by an average of 78.6%.

With the growing prevalence of cloud services, GPUs are increasingly shared across multiple tenants, making address translation a performance-critical path. Our study makes three key observations: 1) Shared L2 TLB becomes the bottleneck in multi-tenant systems. Intense competition for entries can lead to high miss rates and thrashing under heavy pressure. 2) Benchmarks vary significantly in TLB sensitivity due to differing access patterns and memory reuse potential. 3) The prevailing management strategies, including free-for-all sharing and static partitioning, fail to account for tenant access behavior. And the state-of-the-art token-based strategy relies solely on miss rate, an inadequate metric that fails to capture per-tenant access patterns. Based on these observations, we propose LAMP, a localityaware TLB sharing framework. LAMP comprehensively profiles the access pattern considering three key aspects: spatial locality, temporal locality, and access density. These aspects allow LAMP to assess each tenant's sensitivity to TLB capacity and their reuse potential. Based on these insights, LAMP optimizes the sharing by dynamically partitioning L2 TLB, granting more entries to reuse-efficient tenants while limiting wasteful occupancy. This reduces thrashing and cross-tenant interference. Experimental results show that LAMP improves system performance by$\mathbf{1 2. 3 \%}$on average across a range of multi-tenant workloads, with negligible hardware overhead.

Heterogeneous multi-core accelerators have exhibited remarkable computational performance, establishing their competitiveness within the realm of high-performance computing. The on-chip memory to support reliable and correct data access is a crucial element of the accelerator. With the continuous scaling down of semiconductor process nodes, memory units become more susceptible to soft errors caused by Single Event Upset (SEU), thus significantly increasing the risk of memory access errors. Additionally, data interaction between cores supports a variety of granularities, making it challenging to achieve error tolerant designs for multi-granular data with low hardware overhead. This paper proposes the Light-Weight ECC (LWECC) integrated in the 64-bit accelerator’s Scalar Memory (SM) to improve the efficiency of data protection for multi-granularity memory access. LWECC adopts a software-hardware co-design approach, effectively balancing area overhead and providing comprehensive and flexible data protection. Compared to the finest-grained ECC solution as a baseline, using LWECC can reduce redundant memory area by 80.90% and ECC circuit area by 16.87%, markedly reducing the whole hardware overhead of the SM. In addition, LWECC efficiently caters to diverse application domains, supporting not only high-performance computing applications but also accommodating fine-grained memory access applications with low operational latency.

Network-on-Chip (NoC), known for its high bandwidth and scalability, is extensively utilized in chip multiprocessors. However, as technology advances to the nanometer scale, NoC is becoming increasingly vulnerable to errors caused by crosstalk, radiation, electromagnetic interference, and other factors. In addition to ensuring network reliability, designers must consider the overhead instead of blindly pursuing fault-tolerance capabilities in NoC design. In this paper, we analyze the characteristics of conventional End-to-End (E2E) and Switch-to-Switch (S2S) designs and propose AFTP, an adaptive high cost-effectiveness fault-tolerant NoC design based on prediction. Our design prioritizes stronger protection for head flits and relaxed protection for body and tail flits, and it is capable of dynamically adjusting the decoding times for flits in response to changes in error rate, aiming to achieve a balance between the overhead and the reliability of NoC. Our design demonstrates a significant improvement in cost-effectiveness compared to conventional E2E and S2S designs, achieving a 5.9x and 8.9x improvement, respectively, under common synthetic traffic and PARSEC benchmarks.

Although modern GPUs are equipped with expanding memory, accommodating the entire working set of large-scale workloads can still be a challenge. With the support of unified virtual memory and demand paging, programmers can transparently oversubscribe the main memory. However, this transparent management still comes at a severe performance cost, especially for applications with inter-kernel data sharing. While there have been many efforts to reduce additional data migrations caused by the memory oversubscription, few consider the reuse of shared data during the boundary of adjacent kernels. Due to limited memory capacity, we observe that adjacent kernel often demands shared pages that were evicted by the previous kernel, resulting in a significant number of costly data migrations. In this paper, we propose a CTA-Page collaborative framework, called CPC, that transparently reduces the impact of memory oversubscription using CTA dispatch switching and page replacement switching coordinately to reuse inter-kernel shared data. We evaluate CPC with a variety of GPGPU benchmark suites. Experimental results show that the system performance is improved by 65 % compared with the state-of-the-art technique for applications with inter-kernel data sharing.

The demand for multitasking GPUs increases whenever the GPU may be shared by multiple applications, either spatially or temporally. This requires that GPUs can be preempted and switch context to a new application while already executing one. Unlike CPUs, context switching in GPUs is prohibitively expensive due to the large context states to swap out. There have been a number of efforts on reducing the overhead of preemption, through reducing the context sizes or overlapping context switching with execution. All those techniques are reactive approaches, meaning that context switching occurs when the preemption request arrives. In this paper, we propose a dynamic and proactive mechanism to reduce the latency of preemption. We observe that kernel execution is almost always preceded by known commands in both CUDA and OpenCL implementations. Hence, a preemption can be anticipated before the actual request arrives. We study such lead time and develop a prediction scheme to perform an early state saving. When the actual preemption is invoked, an incremental update relative to the previous saved state is performed, much like the conventional checkpointing mechanism. Our design can also choose to drain or checkpointing dynamically and accurately according to the feature of kernels in the runtime. This design effectively reduces the stall time of the preempting kernel due to context switching by 58.6%. Moreover, through careful handling of the saved state, we can also reduce the overall size of saved state by an average of 23.3%, compared with a full context switching.

Modern discrete GPUs support unified memory and demand paging. Automatic management of data movement between CPU memory and GPU memory dramatically reduces developer effort. However, when application working sets exceed physical memory capacity, the resulting data movement can cause great performance loss.

The demand for multitasking GPUs increases whenever the GPU may be shared by multiple applications, either spatially or temporally. This requires that GPUs can be preempted and switch context to a new application while already executing one. Unlike CPUs, context switching in GPUs is prohibitively expensive due to the large context states to swap out. There have been a number of efforts on reducing the overhead of preemption, through reducing the context sizes or overlapping context switching with execution. All those techniques are reactive approaches, meaning that context switching occurs when the preemption request arrives.

Non-uniform cache architecture (NUCA) is often employed to organize the last level cache (LLC) by Networks-on-Chip (NoC). However, along with the scaling up for network size of Systems-on-Chip (SoC), two trends gradually begin to emerge. First, the network latency is becoming the major source of the cache access latency. Second, the communication distance and latency gap between different cores is increasing. Such gap can seriously cause the network latency imbalance problem, aggravate the degree of non-uniform for cache access latencies, and then worsen the system performance.

As the 3D stacking technology still faces several challenges, the 2.5D stacking technology gains better application prospects nowadays. With the silicon interposer, the 2.5D stacking can improve the bandwidth and capacity of the memory system. To satisfy the communication requirements of the integrated memory system, the free routing resources in the interposer should be explored to implement an additional network. Yet, the performance is strongly limited by the unbalanced loads between the CPU-layer network and the interposer-layer network. In this paper, to address this issue, we propose a dynamic latency-aware load-balancing (DLL) strategy. Our key innovations are detecting congestion of the network layer via the average latency of recent packets and making the network layer selection at each source node. We leverage the free routing resources in the interposer to implement a latency propagation ring. With the ring, the latency information tracked at destination nodes is propagated back to source nodes. We achieve load-balance by using these information. Experimental results show that compared with the baseline design, a destination-detection strategy and a buffer-aware strategy, our DLL strategy achieves 45%, 14.9% and 6.5% of average throughput improvements with minor overheads.

To achieve high throughput, core count in compute accelerators such as General-Purpose Graphics Processing Units (GPGPUs) increases continuously. The communication demand of these cores boosts the demand for a low-latency packet switched network. As packet latency is mainly composed of per-hop latency, contention latency and serialization latency, a favorable Network-on-Chip (NoC) design should efficiently decrease these three latency contributors to meet the communication demand while keeping hardware cost low. In this paper, we first make two observations about the NoC differences between CMPs and GPGPUs, and then design a Heterogeneous Ring-Chain network (HRCnet) for the GPGPU reply network. HRCnet eliminates conflicts in the network by proposing a ring-similar topology, using a novel node placement and introducing unidirectional channels. Eliminating conflicts reduces the per-hop latency and removes the contention latency, and exploiting the ring-similar topology reduces the serialization latency. Experimental results show the benefits of the low-cost low-latency design. With the same bisection bandwidth compared to the baseline mesh, our work yields a 45% performance improvement while reducing the area by 42% and reducing energy consumption by 60%. Compared to two state-of-the-art GPGPU NoCs, BENoC and DA2mesh, HRCnet achieves more than 42% performance gain at reduced hardware cost. Our work also achieves the highest power and area efficiency among the designs.

The interaction between packets affects performance and global fairness of Network-on-Chip. Preferentially transferring packets with small remaining hop counts (PPSR) can reduce the flying packet amount to improve the performance. Yet, the global fairness is negatively affected. In contrast, preferentially transferring packets with large remaining hop counts (PPLR) can achieve better global fairness with a poorer performance. In this paper, we propose adaptive remaining hop count flow control, which dynamically switches between PPSR and PPLR. In this way, we can achieve higher performance and better global fairness.