Reservoir computing networks (RCNs) have been recognized as a popular machine learning tool for modeling the temporal evolution of dynamic data due to their close relationship with dynamical systems. A defining characteristic of RCNs is their fixed (training parameter-free) hidden layers, which offers significant computational benefits. However, this feature also introduces adverse impacts, such as increased warm-up time, limited long-term memory, and sensitivity to hyperparameters. To balance these advantages and disadvantages and expand the application domains of RCNs, we develop a novel deep reservoir computing network (DRCN) architecture that integrates control-theoretic concepts and techniques into RCNs. This architecture is designed as a cascade of shallow RCNs and is represented as a piecewise time-invariant control system. We further propose a layer-by-layer training strategy for the DRCN, resulting in an iterative deep learning algorithm for modeling dynamical systems. This enables us to exploit the DRCN as a generative model to generate output-of-sample data using the learned dynamical systems. The performance and efficiency of the DRCN-based dynamic generative model are demonstrated through various learning problems arising from time-series analysis and control systems, using both synthetic and real-world datasets.

Time series data with missing entries are ubiquitous in a broad spectrum of practical and clinical applications, from climatology and cell biology to personalized medicine. This undesired structure arising either due to undesired artifacts (e.g., noise) or by design (e.g., asynchronous or aperiodic sampling in distributed sensors) results in irregularity in the temporal dimension and forms a bottleneck in data mining. Although extensive data science approaches have been proposed to address learning problems involving irregular data, the emphasis was largely placed on filling in the missing entries via interpolation and binning, or the methods were tailored to specific data analytic tasks. In this article, we develop a reservoir computing (RC)-based iterative learning method for recovering missing data in irregular time series generated by dynamical systems and networks. In particular, we formulate this learning task as a fixed-point iterative learning problem and develop a training procedure using an RC network (RCN). We find that when the irregular time series has "sufficient" samples to train an RCN within a tolerant training error then the missing samples in the time series can be recovered systematically. We also derive sufficient conditions with respect to the choices of the reservoir parameters that guarantee the convergence of the iterative procedure. We present several numerical experiments to demonstrate the efficacy of the developed iterative RCN approach. Specifically, we illustrate the capability of our approach to recover missing data in irregular time series generated by chaotic Rössler and Kuramoto-Sivashinsky (KS) systems. Finally, we also report the results of incorporating our approach in an irregular medical data classification task.

The advancement of nonvolatile memory (NVM) technology reduces the cost-per-unit of solid-state drives (SSDs). Flash memory-based SSDs have become ubiquitous because they provide better performance and energy efficiency than hard disk drives. However, it suffers from wear-out problems caused by the out-of-place updates that limit its lifetime. Log-structured merge tree (LSM-tree) is a level-based data structure that is widely used in many database systems because it eliminates the random write operations to the storage devices. By transferring the random write operations into sequential write operations, the write performance of hard disk drives can be improved. However, LSM-tree is not efficient for SSDs because it is not aware of the access characteristics of flash memory. Moreover, the level-based indexing strategy of the LSM-tree significantly shortens the lifetime of SSDs because the data must be frequently updated due to the compaction operations between different levels. In contrast to many previous works that focus on alleviating the write amplification on SSDs for the database systems implemented by LSM-tree, we propose LLSM, a lifetime-aware wear-leveling for LSM-tree on NAND flash memory with open-channel SSD. By considering the data access frequency of the LSM-tree between different levels, LLSM rethinks the block allocation strategy during the compaction to evenly erase all the blocks of SSD storage devices, prolonging the SSD lifetime. Moreover, a proactive swapping strategy is designed to reorganize the data blocks for resolving the potential wear-leveling issues caused by the behaviors of the LSM-tree. The extensive experiments show that the results of lifetime improvement are encouraging.

Owing to high cell density caused by the advanced manufacturing process, the reliability of flash drives turns out to be rather challenging in flash system designs. To enhance the reliability of flash drives, error-correcting code (ECC) has been widely utilized in flash drives to correct error bits during programming/reading data to/from flash drives. Although ECC can effectively enhance the reliability of flash drives by correcting error bits, the capability of ECC would degrade while the program/erase (P/E) cycles of flash blocks is increased. Finally, ECC could not correct a flash page, because a flash page contains too many error bits. As a result, reducing error bits is an effective solution to further improve the reliability of flash drives when a specific ECC is adopted in the flash drive. This work focuses on how to reduce the probability of producing error bits in a flash page. Thus, we propose a pattern-aware write strategy for flash reliability enhancement. The proposed write strategy considers both the P/E cycle of blocks and the pattern of written data while a flash block is allocated to store the written data. Since the proposed write strategy allocates young blocks (respectively, old blocks) for hot data (respectively, cold data) and flips the bit pattern of the written data to the appropriate bit pattern, the proposed strategy can effectively improve the reliability of flash drives. The experimental results show that the proposed strategy can reduce the number of error pages by up to 50%, compared with the well-known DFTL solution. Moreover, the proposed strategy is orthogonal with all ECC mechanisms so that the reliability of the flash drives with ECC mechanisms can be further improved by the proposed strategy.

Since 3D NAND flash memory could provide more reliable storage than a 2D planar flash memory by relaxing the design rule of a memory cell, a kind of brand new programming technique, namely erase-free scheme, has been proposed to further enhance the endurance of a 3D SLC NAND flash memory. The erase-free scheme brings tons of benefits to flash memory performance and endurance. For example, the erase-free scheme could reclaim invalid (page) space without physically erasing a flash block. However, current flash management designs could not fully exploit the benefits of the erase-free scheme. With the considerations of the features of the erase-free scheme, this paper is the first work to propose a novel flash management design, namely VirtualGC strategy, to deal with the erase-free garbage collection process. By taking the advantages of the erase-free scheme, the proposed strategy reduces the overhead of copying live pages so as to increase flash memory performance. The results show that the proposed strategy significantly improves the performance of rewritable 3D flash memory drives.

Reliability has become a critical design issue in flash storage systems, because of the adoption of the low-cost, high-error-rate flash chips to fulfill the needs of the fast-growing storage capacity. In this paper, a multi-version checkpointing strategy is proposed to resolve the reliability issue of flash storage systems from the perspective of flash file systems. The proposed strategy can efficiently and effectively utilize checkpoints of file systems to guarantee the integrity and consistency of flash file systems after files or flash pages are corrupted. By utilizing the coexistence fact of multiple versions of the same data in flash memory, a control/recovery mechanism is presented to maintain checkpoints and to recover file systems with minimized management and recovery time overheads. A series of experiments was conducted based on realistic traces that were collected from benchmarks running over flash file systems in Linux operating systems. The results illustrate that the proposed strategy can significantly improve the reliability of flash file systems, as compared with other existing designs.

The high cell density, multilevel-cell programming, and manufacturing process variance force the new coming flash memory to have large bit-error-rate variance among blocks and pages, where a flash chip consists of multiple blocks and each block consists of a fixed number of pages. In order to avoid storing the crucial user data in more fragile pages, conventional flash management software tends to aggressively discard the high bit-error-rate area in the unit of a block. However, together with the aggressive discarding strategies and the enlarging sizes of pages/blocks of next generation flash memory, the available space of flash devices might encounter a very sharp degradation and therefore result in rapidly-shortened device lifespan. Thus, we advocate the concept of “graceful space degradation” to mitigate this problem by discarding the high bit-error-rate (or worn-out) area in the unit of pages (instead of blocks). To furthermore realize this concept, we are the pioneer to put forward an “uneven space management” to manage flash blocks containing different number of bad pages. Our design especially focuses on placing data with different access behaviors to make the best uses of blocks with different available space so as to ultimately prolong the device lifespan with good access performance. The experiments were conducted based on representative realistic workloads, and the results reveal that the proposed design can extend the device lifetime by at least 2.38 times of that of existent approaches, with very limited performance overheads.

DRAM is widely adopted as a cache for secondary storage due to its small access latency. Compared with DRAM, PRAM draws a lot of attention recently, since it provides higher density and has no need to refresh the capacitor charge periodically. The non-volatile nature of PRAM can even reduce compulsory miss, which cannot be avoided by DRAM cache. However, PRAM cache cannot replace DRAM cache due to its endurance issue. Thus DRAM/PRAM hybrid cache becomes a good alternative for traditional DRAM cache. Least recently used (LRU) replacement algorithm and CLOCK-Pro algorithm work well for traditional DRAM cache. But these algorithms shall not be directly applied to DRAM/PRAM hybrid cache since the characteristics of PRAM are not considered. This paper proposed a double circular caching scheme (DCCS) to manage DRAM/PRAM hybrid cache. In our scheme, cached data migrate between DRAM cache and PRAM cache adaptively to achieve good hit ratio while frequent writes to PRAM cache are avoided for endurance concern. The experimental results showed that our scheme can reduce up to 87.10 percent PRAM write accesses for readintensive access pattern and up to 44.90 percent energy consumption for write-intensive access pattern, compared with other caching schemes.

Embedded database systems are widely adopted in various control and motoring systems, e.g., cyber-physical systems (CPSes). To support the functionality to access the historical data, a multiversion index is adopted to maintain multiple versions of data items and their index information. However, CPSes are usually battery-powered embedded systems that have limited energy, computing power, and storage space. In this work, we consider the systems with phase-change memory (PCM) as their storage due to its non-volatility and low energy consumption. In order to resolve the problem of the limited storage space and the fact that existing multiversion index designs are lack of space efficiency, we propose a space-efficient multiversion index scheme to enhance the space utilization and access performance of embedded multiversion database systems on PCM by utilizing the byte-addressability and write asymmetry of PCM. A series of experiments was conducted to evaluate the efficacy of the proposed scheme. The results show that the proposed scheme achieves very high space utilization and has good performance on serving update transactions and range queries.

DRAM is widely adopted as a cache for secondary storage due to its small access latency. Compared with DRAM, PRAM draws a lot of attention recently, since it provides higher density and has no need to refresh the capacitor charge periodically. The non-volatile nature of PRAM can even reduce compulsory miss, which cannot be avoided by DRAM cache. However, PRAM cache cannot replace DRAM cache due to its endurance issue. Thus DRAM/PRAM hybrid cache becomes a good alternative for traditional DRAM cache. Least recently used (LRU) replacement algorithm and CLOCK-Pro algorithm work well for traditional DRAM cache. But these algorithms shall not be directly applied to DRAM/PRAM hybrid cache since the characteristics of PRAM are not considered. This paper proposed a double circular caching scheme to manage DRAM/PRAM hybrid cache. In our scheme, cached data migrate between DRAM cache and PRAM cache adaptively to achieve good hit ratio while frequent writes to PRAM cache are avoided for endurance concern.