Recently, deep learning-based video compressive sensing reconstruction (VCSR) technologies have significantly improved reconstructed video quality by taking advantage of spatial and temporal correlations. However, existing VCSR work mainly focuses on improving deep learning-based motion compensation without optimizing local and global information, leaving much room for further improvement. This paper proposes a novel VCSR method, JVCSR+, which focuses on optimizing feature information, removing reconstruction artifacts, and increasing the resolution simultaneously. Specifically, the measurement matrix in the proposed compressive sensing (CS) module can learn adaptively, so that sampled measurements retain more image structure information for better reconstruction. An average search module is also proposed to detect more suitable areas for references, thereby attaining superior motion compensation performance. Within the loop, the enhanced frame is utilized as a reference to improve recovery of the current frame. Furthermore, we propose an out-loop super-resolution module for VCSR to obtain high-quality images at low bitrates. The results of extensive experiments demonstrate that our proposed JVCSR+ obtains promising performance compared to state-of-the-art CS methods within the same bitrate range.

In many digital systems, the transmission bandwidth, as well as storage capacity, are usually very limited. This introduces challenges for both video transmission and video storage. To seek lower bit rates and further obtain high-quality up-sampled videos, this paper proposes a temporal down-sampling based video coding system and a frame-recurrent enhancement based video upsampling strategy. The structure of our proposed method is shown in Fig. 1. Unlike the existing work [1], instead of downsampling all video frames, only the intermediate frames are downsampled and two frames remain with high quality on the video coding system. Then, these two high-quality frames are used to iteratively enhance the quality of the low-bitrate low-quality frames through a deep-learned enhancement network. Compared to the latest video coding standard Versatile Video Coding (VVC), our work can obtain a BD-rate reduction from $39.261 {\%} \sim 85. 455$ % in All-Intra and Low-Delay-P configurations on the downsampled frames. A temporal down-sampling based video coding framework (TDS) is proposed. It can be combined with all the existing coding standards including HEVC/H.265 and VVC/H.266. A method of super-resolution with frame recurrent image enhancement (SRFR) is applied to up-sampling the frames by the neighboring high resolution frame. The temporal information from high resolution frames can be fully used to improve the video quality through frame recurrent.

The reconstruction of Compressed Sensing is iteration-based and contains numerous divisions, thereby costing tremendous processing time. In order to eliminate divisions, we adopt a sparse sensing matrix consisting mainly of zero-vectors. After deleting these zero-vectors, an invertible full-rank matrix is obtained. Then the iteration-based reconstruction procedure can be replaced by a matrix multiplication operated in one iteration. Moreover, because the inverse matrix is sparse and deterministic, the multiplication can be simply processed by the shift and add operators. The proposed architecture is verified on the Xilinx Artix-7 FPGA. The result shows that our work accelerates the state-of-the-art method by 65 × and achieves 81.92Gpixels/s reconstruction.

Object detection in a dynamic environment is crucial for accurate tracking and mapping in Simultaneous Localization and Mapping (SLAM). Dynamic feature points from objects cause unreliable performance in SLAM system. Previous researchers used varied techniques to solve this problem, such as, semantic segmentation, optical flow, and moving consistency check algorithm. In our proposal, Object Detection and Tracking SLAM (ODTS), a weighted grid-based attention model is defined for the tracking module, to track landmarks and objects feature points. Our system tracks landmarks, such as buildings in the background, and objects, such as vehicles, in the foreground. For evaluation, the trajectory of SLAM is tracked, and the accuracy is evaluated by obtaining the root mean square error (RMSE). Additionally, the number of background and foreground feature points were observed. ODTS significantly minimizes the tracking lost problem and produces more accurate maps and tracking of feature points.

Artificial intelligence of things has brought artificial intelligence to the cutting-edge Internet of Things. In recent years, compressive sensing (CS), which relies on sparsity, is widely embedded and expected to bring more energy efficiency and a longer battery lifetime to IoT devices. Different from the other image compression standards, CS can get various reconstructed images by applying different reconstruction algorithms on coded data. Using this property, it is the first time to propose a deep learning based compressive sensing image enhancement framework using multiple reconstructed signals (CSIE-M). In this article, first, images are reconstructed by different CS reconstruction algorithms. Second, reconstructed images are assessed and sorted by a no-reference quality assessment module before being input to the quality enhancement module by order of quality scores. Finally, a multiple-input recurrent dense residual network is designed for exploiting and enriching the useful information from the reconstructed images. Experimental results show that CSIE-M obtains 1.88–8.07 dB peek-signal-to-noise (PSNR) improvement while the state-of-the-art works achieve a 1.69–6.69 dB PSNR improvement under sampling rates from 0.125 to 0.75. On the other hand, using multiple reconstructed versions of the signal can improve 0.19–0.23 dB PSNR, and only 4% reconstructing time is increasing compared to using a reconstructed signal.

This paper proposes a speedup convolutional neural network (CNN)-based deblurring framework (SpeedDeblur) for reconstructed blurry videos. First, we extract the coding information and the reconstructed video from the compressed data. Second, a CNN-based algorithm is used for deblurring the first reconstructed frame. Pixels of the deblurred frames are transferred to the subsequent frames guided by HEVC decoded data. The transferring process is simple and faster than applying a deblurring algorithm on all frames. However, passing pixels throughout a long video propagates accumulated errors and reduces the deblurring performance. To bridge this gap, we design an adaptive reset strategy for deciding which frame needs CNN-based deblur during the transfering process. Besides, a data generation strategy simulating blurry real-world factors such as camera shake and fast movement is proposed. Compared to frame-by-frame deblurring approaches, our framework can retain the same comparable results and boost the deblurring processing by up to 4.0 × and 99.4 × on GPU and CPU, respectively.

This work proposes a bi-directional intra prediction-based measurement coding algorithm for compressive sensing images. Compressive sensing is capable of reducing the size of the sparse signals, in which the high-dimensional signals are represented by the under-determined linear measurements. In order to explore the spatial redundancy in measurements, the corresponding pixel domain information extracted using the structure of measurement matrix. Firstly, the mono-directional prediction modes (i.e. horizontal mode and vertical mode), which refer to the nearest information of neighboring pixel blocks, are obtained by the structure of the measurement matrix. Secondly, we design bi-directional intra prediction modes (i.e. Diagonal + Horizontal, Diagonal + Vertical) base on the already obtained mono-directional prediction modes. Experimental results show that this work improves 0.01 - 0.02 dB PSNR improvement and the birate reductions of on average 19%, up to 36% compared to the state-of-the-art.