Accelerating neural network training is critical in exploring design space of neural networks. Data parallelism is commonly used to accelerate training for Convolutional Neural Networks (CNN) where input batch is distributed across the multiple workers; however, the increase in communication of weight gradients across the workers limits scalability. In this work, we propose multi-dimensional parallel (MDP) training of convolution layer by exploiting both data parallelism and intratile parallelism available in Winograd transformed convolution. Workers are organized across two dimensions - one dimension exploiting intra-tile parallelism while the other dimension exploits data parallelism. MDP reduces the amount of communication necessary for weight gradients since weight gradients are only communicated across the data parallelism dimension. However, Winograd transform fundamentally requires more data accesses and the proposed MDP architecture also introduces a new type of communication which we refer to as tile transfer - gather/scatter of Winograd domain feature maps (tiles). We propose a scalable near-data processing (NDP) architecture to minimize the cost of data accesses through 3D stacked memory while leveraging a memory-centric network organization to provide high-connectivity between the workers with intra-tile parallelism to accelerate tile transfer. To minimize tile gathering communication overhead, we exploit prediction of activation of spatial domain neurons in order to remove the communication of tiles that are transformed to non-activated neurons. In order to balance the communication required for weight gradients and tile transfer, we also propose a reconfigurable memory-centric network architecture that reconfigures network channel connectivity between the workers for each convolution layer. Our evaluations show that the proposed MDP with NDP architecture accelerates training by 2.7×, 9.5-21× compared to the data parallel training with the NDP architecture and a multi-GPU system, respectively.

Recent technology advances in memory system design, along with 3D stacking, have made near-data processing (NDP) more feasible to accelerate different workloads. In this work, we explore the near-data processing opportunity of a fundamental operation - linked-list traversal (LLT). We propose a new NDP architecture which does not change the existing sequential programming model and does not require any modification to the core microarchitecture. Instead, we exploit the packetized interface between the core and the memory modules to off-load LLT for NDP. We assume a system with multiple memory modules (e.g., hybrid memory cube (HMC) modules) interconnected with a memory network and our initial evaluation shows that simply off-loading LLT computation to near-memory can actually reduce performance because of the additional off-chip memory network channel traversal. Thus, we first propose NDP-aware data localization to exploit packaging locality - including locality within a single memory module and memory vault - to minimize latency and improve energy efficiency. In order to improve overall throughput and maximize parallelism, we propose batching multiple LLT operations together to amortize the cost of NDP by utilizing the highly parallel execution of NDP processing units and the high bandwidth of 3D stacked DRAM. Our evaluation shows that the combination of NDP-aware data localization and batching can provide significant improvement in performance and energy efficiency.

Key-value stores such as Memcached have become widely used by cloud and web-service providers. While there has been a significant amount of research done on improving the absolute performance of key-value stores, this work proposes an adaptive and a flexible approach to key-value stores. We first propose soft data partitioning that divides memory into multiple groups within a single node, or a single server process, to enable scale-up of key-value stores, while providing NUMA locality and an adaptive approach that can reduce overall request miss rate. The soft-partitioning enables a flexible Memcached server implementation in a NUMA system through NUMA-aware allocation as well as power-efficient NUMA server operation by migrating frequently accessed key-value pairs among the groups. We also propose an adaptive replacement policy within Memcached server that compares miss rates across the different memory groups to determine a more optimal replacement policy. To overcome the limitation of partitioning, we propose Group Auto-Balancing (GAB) where memory allocation from the different groups can be borrowed to minimize miss rate. Our results improve Memcached throughput by 12.9%, on average, over previously proposed MemC3 algorithm (up to 3.1× for write intensive workloads) while the adaptive replacement policy shows the lowest miss rate on adversarial access patterns.

When designing an SoC (system-on-chip), one should ensure that the chip's architecture delivers optimal data throughput for high-speed connectivity blocks on practical use, taking of several layers of software into account. Previous works such as simulation based estimation method or analytical method that focused on estimation of low level throughput in link layer are not adequate for accurately estimating throughput of the real silicon SoC because practical use cases of such complex connectivity blocks typically require extensive software operation on top of platform operating systems like Linux. FPGA-based emulation, therefore, is commonly used to verify functional correctness of complex connectivity blocks in a more realistic usage scenario with real connectivity devices like USB 2.0 mass storage devices. Speed of FPGA emulation, though, is scaled down significantly because of inherent limitation of FPGA emulation while speed of interface to real connectivity devices is required to be at-speed for compliance with connectivity specification standards. For this reason, scaling factors of speed are not uniform and, thus, measurement of performance is not accurate. This paper proposes a method to compensate errors due to the non-uniform scaling factors. In the proposed method, measurements of various parameters in FPGA emulation are applied to a high-accuracy estimation model that we created. Our method was applied to USB2.0, and the experimental result shows 4.1% of estimation error.