Applications powered by Deep Neural Networks (DNNs) are shifting from cloud to edge computing due to the promising performance efficiency of Resistive Random Access Memory (RRAM) devices. However, modern RRAM devices have various possible faults, including hard and soft faults. Existing RRAM-based deep learning systems lack a comprehensive fault tolerance approach, leading to safety risks in deploying edge-end DNN inference. In this paper, we propose an effective fault recovery framework called TOSA to safeguard edge-based DNN accelerators from Stuck-At-Faults (SAFs). Two novel ideas are presented to enhance reliability in edge-based RRAM inference accelerators: (1) SAFs-aware priority-based bit-memristor mapping algorithm to match integrated SAF variations and DNN’s defect tolerance, (2) a new retraining method called joint retraining to recover model accuracy from multiple faulty edge devices. We thoroughly evaluate TOSA on multiple DNN models, demonstrating its effectiveness in reducing costs and improving efficiencies. Experimental results show that TOSA achieves a 45.28× speedup when compared to the per-device retraining methods while preserving inference accuracy better than the latest competitor, making it a reliable architecture for accelerating DNN inference on edge devices.

Approximate computing applications lead to large energy consumption and performance demand for the memory system. However, traditional SRAM based cache cannot satisfy these demands due to high leakage power and limited density. Spin Transfer Torque Magnetic RAM (STT-MRAM) is a promising candidate of cache due to low leakage power and high density. However, STT-MRAM suffers from high write energy. To leverage the ability of tolerating acceptable quality loss via approximations to data, we propose an STT-MRAM based APProximate cache architecture (APPcache) to write/read approximate data thus largely reducing energy. We find many similar elements (e.g. pixels in images) existing in cache lines while running approximate computing applications. Therefore, APPcache uses several lightweight similarity-based encoding schemes to eliminate the similar elements to reduce the data size thus reducing the write energy of STT-MRAM based cache. Besides, we design a software interface to manually control the output quality. APPcache can significantly eliminate similar elements, thus improving energy efficiency. Experimental results show that our scheme can reduce write energy and improve the image raw data compression ratio by 21.9% and 38.0% compared with the state-of-the-art scheme with 1 % error rate, respectively. As for the output quality, the losses of all benchmarks are within 5% with 1 % error rate.

Although GPUs have emerged as the mainstream for the acceleration of convolutional neural network (CNN) training processes, they usually have limited physical memory, meaning that it is hard to train large-scale CNN models. Many methods for memory optimization have been proposed to decrease the memory consumption of CNNs and to mitigate the increasing scale of these networks; however, this optimization comes at the cost of an obvious drop in time performance. We propose a new memory optimization strategy named Layup that realizes both better memory efficiency and better time performance. First, a fast layer-type-specific method for memory optimization is presented, based on the new finding that a single memory optimization often shows dramatic differences in time performance for different types of layers. Second, a new memory reuse method is presented in which greater attention is paid to multi-type intermediate data such as convolutional workspaces and cuDNN handle data. Experiments show that Layup can significantly increase the scale of extra-deep network models on a single GPU with lower performance loss. It even can train ResNet with 2,504 layers using 12GB memory, outperforming the state-of-the-art work of SuperNeurons with 1,920 layers (batch size = 16).

Byte-addressable persistent memory provides fast access to persistent data, but problems such as system crashes and power failures may cause persistent data corruption. Though using traditional logging or copy-on-write(COW) can guarantee crash consistency, it results in extra storage and performance overheads, and also shortens the lifetime of persistent memory. To reduce the overhead caused by logging and COW, existing mechanisms ensure consistency by leveraging the different versions of data in a non-volatile CPU cache(NV cache) and persistent memory. However, the performance loss is still significant compared with the system that does not enforce crash consistency. To improve the performance, we analyze the overheads of memory transactions quantitatively, including the access pattern of cache and the asymmetry of read and write of persistent memory media. Based on our analysis, we propose DV-NVLLC which uses the NV cache as the last-level cache(NV-LLC). In NVLLC the dynamic versioning method is used to manage different versions of data based on their update frequency. Hot data is updated-of-place in NV-LLC. Cold data is updated by version control between NV-LLC and persistent memory. Therefore, DV-NVLLC can reduce write traffic to persistent memory while guaranteeing crash consistency. Our evaluation shows that DV-NVLLC improves performance by 18% to 57%, and reduces write traffic to persistent memory by 35% to 89% compared with the state-of-the-art design that uses NV cache as the last-level cache.

File update operations generate many invalid flash pages in Solid State Drives (SSDs) because of the-of-place update feature. If these invalid flash pages are not securely deleted, they will be left in the “missing” state, resulting in leakage of sensitive information. However, deleting these invalid pages in real time greatly reduces the performance of SSD. In this paper, we propose a Per-File Secure Deletion (PSD) scheme for SSD to achieve non-real-time secure deletion. PSD assigns a globally unique identifier (GUID) to each file to quickly locate the invalid data blocks and uses Security-TRIM command to securely delete these invalid data blocks. Moreover, we propose a PSD-MLC scheme for Multi-Level Cell (MLC) flash memory. PSD-MLC distributes the data blocks of a file in pairs of pages to avoid the influence of programming crosstalk between paired pages. We evaluate our schemes on different hardware platforms of flash media, and the results prove that PSD and PSD-MLC only have little impact on the performance of SSD. When the cache is disabled and enabled, compared with the system without the secure deletion, PSD decreases SSD throughput by 1.3% and 1.8%, respectively. PSD-MLC decreases SSD throughput by 9.5% and 10.0%, respectively.

Growing accuracy and robustness of Deep Neural Networks (DNN) models are accompanied by growing model capacity (going deeper or wider). However, high memory requirements of those models make it difficult to execute the training process in one GPU. To address it, we first identify the memory usage characteristics for deep and wide convolutional networks, and demonstrate the opportunities of memory reuse on both intra-layer and inter-layer levels. We then present Layrub, a runtime data placement strategy that orchestrates the execution of training process. It achieves layer-centric reuse to reduce memory consumption for extreme-scale deep learning that cannot be run on one single GPU.

Due to the popularity of Deep Neural Network (DNN) models, we have witnessed extreme-scale DNN models with the continued increase of the scale in terms of depth and width. However, the extremely high memory requirements for them make it difficult to run the training processes on single many-core architectures such as a Graphic Processing Unit (GPU), which compels researchers to use model parallelism over multiple GPUs to make it work. However, model parallelism always brings very heavy additional overhead. Therefore, running an extreme-scale model in a single GPU is urgently required. There still exist several challenges to reduce the memory footprint for extreme-scale deep learning. To address this tough problem, we first identify the memory usage characteristics for deep and wide convolutional networks, and demonstrate the opportunities for memory reuse at both the intra-layer and inter-layer levels. We then present Layrub, a runtime data placement strategy that orchestrates the execution of the training process. It achieves layer-centric reuse to reduce memory consumption for extreme-scale deep learning that could not previously be run on a single GPU. Experiments show that, compared to the original Caffe, Layrub can cut down the memory usage rate by an average of 58.2% and by up to 98.9%, at the moderate cost of 24.1% higher training execution time on average. Results also show that Layrub outperforms some popular deep learning systems such as GeePS, vDNN, MXNet, and Tensorflow. More importantly, Layrub can tackle extreme-scale deep learning tasks. For example, it makes an extra-deep ResNet with 1,517 layers that can be trained successfully in one GPU with 12GB memory, while other existing deep learning systems cannot.