Deep neural networks (DNNs) have great potential to solve many real-world problems, but they usually require an extensive amount of computation and memory. It is of great difficulty to deploy a large DNN model to a single resource-limited device with small memory capacity. Distributed computing is a common approach to reduce single-node memory consumption and to accelerate the inference of DNN models. In this paper, we explore the "within-layer model parallelism", which distributes the inference of each layer into multiple nodes. In this way, the memory requirement can be distributed to many nodes, making it possible to use several edge devices to infer a large DNN model. Due to the dependency within each layer, data communications between nodes during this parallel inference can be a bottleneck when the communication bandwidth is limited. We propose a framework to train DNN models for Distributed Inference with Sparse Communications (DISCO). We convert the problem of selecting which subset of data to transmit between nodes into a model optimization problem, and derive models with both computation and communication reduction when each layer is inferred on multiple nodes. We show the benefit of the DISCO framework on a variety of CV tasks such as image classification, object detection, semantic segmentation, and image super resolution. The corresponding models include important DNN building blocks such as convolutions and transformers. For example, each layer of a ResNet-50 model can be distributively inferred across two nodes with 5x less data communications, almost half overall computations and less than half memory requirement for a single node, and achieve comparable accuracy to the original ResNet-50 model.

In modern solid-state drives (SSDs), the indexing of flash pages is a critical component in their storage controllers. It not only affects the data access performance, but also determines the efficiency of the precious in-device DRAM resource. A variety of address mapping schemes and optimizations have been proposed. However, most of them were developed with human-driven heuristics.

Garbage collection (GC) is ubiquitously used to reclaim useful space during the data update process in NAND flash memories. Conventional GC algorithms keep track of a table storing the number of valid pages in each block and select ones with the smallest number of valid pages to erase. We notice that NAND flash not only stores user data but also keeps updating meta data frequently and one of the most important requirements of GC for meta data updates is latency predictability, which means the latency of GC should not only be low on average and on tail distributions, but also independent of workload, i.e., the sequence of meta data updates. It is also desirable to have all NAND pages/blocks written the same number of times to avoid any latency incurred by wear leveling. In this paper, we propose GC algorithms that do not require any look-up tables, which avoids the latency for reading the table and sorting the number of valid pages, and intrinsically enables all pages to be worn equally. Several GC algorithms are proposed to make trade-offs between over-provisioning and write amplification. We also provide sufficient and necessary conditions that the proposed GC algorithms will not encounter a deadlock, i.e., the algorithms can run continuously on all meta data update sequences without data loss.

Traditionally, computer storage has been separated into a hierarchy based on response time, volatility, and cost of media. This tiering is undergoing a significant upheaval as a new breed of memory technologies, termed Storage Class Memories (SCM), now make it feasible to replace several tiers of the hierarchy with a single, cost-effective, uniform type of memory/storage. To make large-scale SCM deployments practical, however, memory system designers will first need to solve the problem of how to guard against unavoidable storage wear-out and failures-problems traditionally absent from "main memory" and handled by software at leisurely timescales in the domain of storage. In this paper, we propose a novel approach to providing fault tolerance in SCM-based main memory. Our key insight is to treat memory as a distributed storage system and rely on data replication and a consensus protocol to keep the replicas consistent. Separate memory instances store replicated copies of the data, and we use a programmable network interconnect to provide fast consensus between the memory instances. Our initial experiments using software memory controller emulation demonstrate reasonable overhead over local memory reads and show great promise as scalable main memory.

Rabin fingerprints are short tags for large objects that can be used in a wide range of applications, such as data deduplication, web querying, packet routing, and caching. We present a pipelined hardware architecture for computing Rabin fingerprints on data being transferred on a high throughput bus. The design conducts real-time fingerprinting with short latencies, and can be tuned for optimized clock rate with "split fresh" technique. A pipelined sampling logic selects fingerprints based on the Minwise theory and adds only a few clock cycles of latency before returning the final results. The design can be replicated to work in parallel for higher throughput data traffic. This architecture is implemented on a Xilinx Virtex-6 FPGA, and is tested on a storage prototyping platform. The implementation shows that the design can achieve clock rates above 300 MHz with an order of magnitude improvement in latency over prior software implementations, while consuming little hardware resource. The scheme is extensible to other types of fingerprints and CRC computations, and is readily applicable to primary storages and caches in hybrid storage systems.

Data deduplication has proven important in backup storage systems as large amount of identical or similar data chunks exist. Recent studies have shown the great potential of data deduplication in primary storage and storage caches. Deduplications in these environments require high speed processing not to drag down production performance. This paper presents a hardware accelerator for similarity based data deduplication. It implements three compute-intensive kernel modules to improve throughput and latency in dedupe systems: sketch computation for data blocks, index searching for reference block, and delta encoding over similar blocks. Adopting pipelined computation and parallel data lookup across multiple hardware modules, our HW design is capable of processing high throughput data traffic by working on multiple data units concurrently, thus enabling wire speed dedupe for data stream where similar blocks present. Using a PC host system connected to the FPGA-based accelerator through a PCIe Gen 2×4 interface, our experiments show that the similarity based data dedupe performs 30% better in data reduction ratio than conventional dedupe techniques that look at identical blocks only. By comparing the hardware implementation with its software counterpart, the experimental results show that our preliminary FPGA implementation with maximum clock speed of 250MHz achieves at least 6 times improvement in latency over the software implementation running on state-of-art servers.