Log-Structured-Merge-tree (LSM-tree) has been extensively adopted because of its exceptional write efficiency and high space utilization. Compaction is invoked periodically in LSM-tree based key-value(LSM KV) systems to maintain good system performance. As the size of LSM-KV grows, large-scale compaction is now frequently seen. Compaction throughput significantly degrades with larger inputs, leading to frequent write stalls and decrement in overall write throughput. This paper proposes STEM, a stream-based compaction framework with FPGA to address this issue. A clean-cut algorithm is introduced to enable streaming-based compaction for large-scale data. With a multi-unit pipeline and dynamic pipeline schedule, STEM can handle large-scale compaction tasks efficiently. Based on the experiment result, the compaction throughput of STEM can achieve$27\times$on average and up to$35\times$improvement compared with the current RocksDB compaction,$2.09\times$to$2.27\times$improvement compared with the state-of-the-art FPGA accelerator.

Mobile devices have been increasingly deployed in large-scale cyber-physical systems (CPS) to traverse the field and retrieve various data measurements from designated physical entities with stringent performance requirements. This work studies the Availability-constrained real-time Fresh Data Retrieval problem in CPS with a Speed Adjustable mobile device (AFDR-SA). The goal is to maintain the temporal validity of the real-time data with different priorities to be retrieved in the system while meeting the data availability constraints imposed by the communication range between the mobile device and the physical entities. The general case of the AFDR-SA problem is proved to be NP-hard. A dynamic programming (DP)-based optimal algorithm is proposed for a special scenario where the retrieval times of individual data items with the same priority are of the same length. For the general case where data items can have arbitrary retrieval times and different priorities, another different DP-based scheme is proposed, which is proved to be optimal given the retrieval order. A fast heuristic with low complexity is also proposed for the general problem to improve the computational efficiency. The experimental results show that the proposed schemes for the general case outperform the state-of-the-art methods and have close performance compared to the optimal solution while incurring much less computational overhead.

Deep-learning-based compressor has received interests recently due to much improved compression ratio. However, modern approaches suffer from long execution time. To ease this problem, this paper targets on cutting down the execution time of deep-learning-based compressors. Building history-dependencies sequentially (e.g., recurrent neural networks) is responsible for long inference latency. Instead, we introduce transformer into deep learning compressors to build history-dependencies in parallel. However, existing transformer is too heavy in computation and incompatible to compression tasks.

Recent works on learned index open a new direction for the indexing field. The key insight of the learned index is to approximate the mapping between keys and positions with piece-wise linear functions. Such methods require partitioning key space for a better approximation. Although lots of heuristics are proposed to improve the approximation quality, the bottleneck is that the segmentation overheads could hinder the overall performance. This paper tackles the approximation problem by applying a distribution transformation to the keys before constructing the learned index. A two-stage Normalizing-Flow-based Learned index framework (NFL) is proposed, which first transforms the original complex key distribution into a near-uniform distribution, then builds a learned index leveraging the transformed keys. For effective distribution transformation, we propose a Numerical Normalizing Flow (Numerical NF). Based on the characteristics of the transformed keys, we propose a robust After-Flow Learned Index (AFLI). To validate the performance, comprehensive evaluations are conducted on both synthetic and real-world workloads, which shows that the proposed NFL produces the highest throughput and the lowest tail latency compared to the state-of-the-art learned indexes.

With the rapid growth of big data, LSM-tree based key-value stores are widely applied due to its high efficiency in write performance. Compaction plays a critical role in LSM-tree, which merges old data and could significantly reduce the overall throughput of the whole system especially for write-intensive workloads. Hardware acceleration for database is a popular trend in recent years. In this paper, we design and implement an FPGA-based compaction engine to accelerate compaction in LSM-tree based key-value stores. To take full advantage of the pipeline mechanism on FPGA, the key-value separation and index-data block separation strategies are proposed. In order to improve the compaction performance, the bandwidth of FPGA-chip is fully utilized. In addition, the proposed acceleration engine is integrated with a classic LSM-tree based store without modifications on the original storage format. The experimental results demonstrate that the proposed FPGA-based compaction engine can achieve up to 92.0x acceleration ratio compared with CPU baseline, and achieve up to 6.4x improvement on the throughput of random writes.

Hardware acceleration through field programmable gate arrays (FPGAs) has recently become a technique of growing interest for many data-intensive applications. Join query is one of the most fundamental database query types useful in relational database management systems. However, the available solutions so far have been beset by higher costs in comparison with other query types. In this paper, we develop a novel solution to accelerate the processing of sort-merge join queries with low match rates. Specifically, our solution makes use of hierarchical indexes to identify result-yielding regions in the solution space in order to take advantage of result sparseness. Further, in addition to one-dimensional equi-join query processing, our solution supports processing of multidimensional similarity join queries. Experimental results show that our solution is superior to the best existing method in a low match rate setting; the method achieves a speedup factor of 4.8 for join queries with a match rate of 5%.

Maintaining the temporal validity of real-time data in cyber-physical systems is of critical importance to ensure the correct decision making and appropriate system operation. Most existing work on real-time data retrieval assume that the real-time data under study are always available for retrieval, and the developed scheduling algorithms mainly focus on making real-time decisions while meeting the temporal validity constraints. This assumption, however does not hold in many real-time applications with intermittent data availability. In this paper, we study the Availability-constrained Fresh Data Retrieval (AFDR) problem, which aims to retrieve all required real-time data for a given set of decision tasks on time while taking both the temporal validity and data availability constraints into consideration. We formulate the AFDR problem as an ILP problem and study its complexity under different settings. Given the general case of the AFDR problem is proved to be NP-hard, we focus on the cases that data items have unit-size retrieval time. For the single decision task scenario, we propose a polynomial-time optimal data retrieval algorithm, which consists of a task finish time selection phase and an optimal retrieval schedule construction phase, to solve the AFDR problem. For the multiple decision task scenario, we propose an efficient heuristic algorithm by transforming the temporal validity constraint of a real-time data item to the availability constraint. The effectiveness of the proposed algorithms has been validated through extensive experiments. Our results show that the heuristic algorithm outputs around $1.5\times$1.5× feasible cases compared to that of the state-of-the-art scheme.

Hardware acceleration through field programmable gate arrays (FPGAs) has recently become a technique of growing interest for many data-intensive applications. Join query is one of the most fundamental database query types useful in relational database management systems. However, the available solutions so far have been beset by higher costs in comparison to other query types. In this paper, we develop a novel solution to accelerate the processing of sort-merge join queries with low match rates. Specifically, our solution makes use of hierarchical indexes to identify result-yielding regions in the solution space in order to take advantage of result sparseness. Further, in addition to one-dimensional equi-join query processing, our solution supports processing of multidimensional similarity join queries. Experimental results show that our solution is superior to the best existing method in a low match rate setting; the method achieves a speedup factor of 4.8 for join queries with a match rate of 5 percent.