Existing image compressed sensing (CS) coding frameworks usually solve an inverse problem based on measurement coding and optimization-based image reconstruction, which still exist the following two challenges: (1) the widely used random sampling matrix, such as the Gaussian Random Matrix (GRM), usually leads to low measurement coding efficiency, and (2) the optimization-based reconstruction methods generally maintain a much higher computational complexity. In this article, we propose a new convolutional neural network based image CS coding framework using local structural sampling (dubbed CSCNet) that includes three functional modules: local structural sampling, measurement coding, and Laplacian pyramid reconstruction. In the proposed framework, instead of GRM, a new local structural sampling matrix is first developed, which is able to enhance the correlation between the measurements through a local perceptual sampling strategy. Besides, the designed local structural sampling matrix can be jointly optimized with the other functional modules during the training process. After sampling, the measurements with high correlations are produced, which are then coded into final bitstreams by the third-party image codec. Last, a Laplacian pyramid reconstruction network is proposed to efficiently recover the target image from the measurement domain to the image domain. Extensive experimental results demonstrate that the proposed scheme outperforms the existing state-of-the-art CS coding methods while maintaining fast computational speed.

Most multi-exposure image fusion (MEF) methods perform unidirectional alignment within limited and local regions, which ignore the effects of augmented locations and preserve deficient global features. In this work, we propose a multi-scale bidirectional alignment network via deformable self-attention to perform adaptive image fusion. The proposed network exploits differently exposed images and aligns them to the normal exposure in varying degrees. Specifically, we design a novel deformable self-attention module that considers variant long-distance attention and interaction and implements the bidirectional alignment for image fusion. To realize adaptive feature alignment, we employ a learnable weighted summation of different inputs and predict the offsets in the deformable self-attention module, which facilitates that the model generalizes well in various scenes. In addition, the multi-scale feature extraction strategy makes the features across different scales complementary and provides fine details and contextual features. Extensive experiments demonstrate that our proposed algorithm performs favorably against state-of-the-art MEF methods.

The development of healthcare industry, especially Internet of Medical Things (IoMT), has generated considerable unlabeled electrocardiogram (ECG) signals. This article proposes a new unsupervised feature learning method for these unlabeled 12-lead ECGs, a type of 12-channel 1-D time series. Based on contrastive predictive coding (CPC), it considers the characteristics of 12-lead ECGs and develops novel lead-separation CPC (LSCPC) and lead-combination CPC (LCCPC). Specifically, LSCPC captures intralead features for each lead, while LCCPC combines all the leads and explores interlead relationships. Furthermore, a fusion model of LSCPC and LCCPC generates final representations. The Physikalisch-Technische Bundesanstalt (PTB)-XL database that contains 21837 12-lead records is used for unsupervised feature learning. Using learned features, linear classifiers are trained to accomplish the downstream tasks. 448 ECG records from 148 myocardial infarction (MI) and 52 healthy control subjects of the PTB database are used for MI detection. 6877 records from the CPSC-2018 database are used for atrial fibrillation (AF) detection, including 918 normal records, 1098 AF records, and 4861 other records. Using fivefold cross-validation, our model achieves 90.38% and 73.27% accuracy in MI and AF detection, respectively. Compared with existing models, it improves the performances by at least 2.32% for MI detection and 3.99% for AF detection. The model has been deployed on a lightweight embedded system (800-MHz ARM processor, 1-GB RAM). The maximum latency is only 465.34 ms, which can satisfy the real-time constraints. Overall, all the results have demonstrated the potential of our method for real-world healthcare, and lightweight IoMT applications.

The compressed sensing (CS) has been successfully applied to image compression in the past few years as most image signals are sparse in a certain domain. Several CS reconstruction models have been proposed and obtained superior performance. However, these methods suffer from blocking artifacts or ringing effects at low sampling ratios in most cases. To address this problem, we propose a deep convolutional Laplacian Pyramid Compressed Sensing Network (LapC-SNet) for CS, which consists of a sampling sub-network and a reconstruction sub-network. In the sampling sub-network, we utilize a convolutional layer to mimic the sampling operator. In contrast to the fixed sampling matrices used in traditional CS methods, the filters used in our convolutional layer are jointly optimized with the reconstruction sub-network. In the reconstruction sub-network, two branches are designed to reconstruct multi -scale residual images and muti -scale target images progressively using a Laplacian pyramid architecture. The proposed LapCSNet not only integrates multi-scale information to achieve better performance but also reduces computational cost dramatically. Experimental results on benchmark datasets demonstrate that the proposed method is capable of reconstructing more details and sharper edges against the state-of-the-arts methods.

Gaussian random matrix (GRM) has been widely used to generate linear measurements in compressed sensing (CS) of natural images. However, there actually exist two disadvantages with GRM in practice. One is that GRM has large memory requirement and high computational complexity, which restrict the applications of CS. Another is that the CS measurements randomly obtained by GRM cannot provide sufficient reconstruction performances. In this paper, a Deep neural network based Sparse Measurement Matrix (DSMM) is learned by the proposed convolutional network to reduce the sampling computational complexity and improve the CS reconstruction performance. Two sub-networks are included in the proposed network, which are the sampling sub-network and the reconstruction sub-network. In the sampling sub-network, the sparsity and the normalization are both considered by the limitation of the storage and the computational complexity. In order to improve the CS reconstruction performance, a reconstruction sub-network are introduced to help enhance the sampling sub-network. So by the offline iterative training of the proposed end-to-end network, the DSMM is generated for accurate measurement and excellent reconstruction. Experimental results demonstrate that the proposed DSMM outperforms GRM greatly on representative CS reconstruction methods.

Compressed Sensing (CS) has been successfully applied to image compression in the past few years. However, there are still several challenges that restrict its applications in practice including large memory requirement and unsatisfactory reconstruction performance. To address these challenges, in this paper, we propose a classification guided deep convolutional network for image compressed sensing (CCSNet), which includes a sampling sub-network and a reconstruction sub-network. In the sampling sub-network, multiple convolutional layers are used to sample the original image, which significantly reduces the parameters of the sampling matrix while causes performance degradation moderately compared against existing convolution based sampling methods. In the reconstruction sub-network, a novel two-branch architecture is proposed to improve the adaptability of the model to various textures in natural images. The first branch, named the classification branch, is to classify the sampled measurements of the original image to one of the predefined textural classes. The second branch, named the reconstruction branch, consists of multiple sub-branches, which are responsible for reconstructing the original images belonging to the corresponding textural classes. By jointly utilizing two sub-networks, the entire network can be trained in the form of end-to-end metric with a joint loss function. Experimental results demonstrate that the proposed method provides a significant quality improvement in terms of PSNR compared against state-of-the-art methods.

Traditional image compressed sensing (CS) coding frameworks solve an inverse problem that is based on the measurement coding tools (prediction, quantization, entropy coding, etc.) and the optimization based image reconstruction method. These CS coding frameworks face the challenges of improving the coding efficiency at the encoder, while also suffering from high computational complexity at the decoder. In this paper, we move forward a step and propose a novel deep network based CS coding framework of natural images, which consists of three sub-networks: sampling sub-network, offset sub-network and reconstruction sub-network that responsible for sampling, quantization and reconstruction, respectively. By cooperatively utilizing these sub-networks, it can be trained in the form of an end-to-end metric with a proposed rate-distortion optimization loss function. The proposed framework not only improves the coding performance, but also reduces the computational cost of the image reconstruction dramatically. Experimental results on benchmark datasets demonstrate that the proposed method is capable of achieving superior rate-distortion performance against state-of-the-art methods.

Ring Learning With Errors (RLWE)-based key exchange is one of the most efficient and secure primitive for post-quantum cryptography. One common approach to achieve key exchange over RLWE is error reconciliation. Recently, an efficient attack against reconciliation-based RLWE key exchange protocols with reused keys was proposed. This attack can recover a long-term private key if a key pair is reused. We also know that in the real world, key reuse is commonly adopted in applications like the Transport Layer Security (TLS) protocol to improve performance. Directly motivated by this attack, we construct a new randomized RLWE-based key exchange protocol against this attack. Our lightweight approach incorporates an additional ephemeral public error term into key exchange materials, so that this attack no longer works. With the same attack, we practically show that the signal value of our protocol is indistinguishable from uniform random, therefore, this attack no longer works. We explain how the attack fails, present 200-bit classic and 80-bit quantum secure parameter choice, efficient implementations, comparisons and discussion. Benchmark shows our protocol is truly efficient and even faster than related vulnerable protocols.

Video quality in real-time mobile communication is often influenced by the ambient light. In the low-lighting condition, videos in real-time mobile communication are usually dark and lack details. To solve the above problem, we propose a fast low-lighting video enhancement algorithm by exploiting the constrained spatial-temporal model, in which three terms including luminance accuracy, contrast accuracy and temporal consistency are all considered. For the first term, the average luminance level of each frame is taken into account of a set of the off-line trained luminance enhancement functions. For the second term, the adjustment of the luminance range in the frame based statistical histogram is obtained for the adaptive contrast enhancement function. For the third term, the temporal consistency is utilized by considering the current enhancement function and these of the previous frames to avoid the flicker between these adjacent frames. Furthermore, the proposed method has been already implemented on Wechat, a social application that connects 800 million people with chat, calls and more. Extensive experiments demonstrate that the proposed method achieves better enhancement results than many current state-of-the-art methods.

Gaussian random matrix (GRM) has been widely used to generate linear measurements in compressive sensing (CS) of natural images. However, in practice, there actually exist two problems with GRM. One is that GRM is non-sparse and complicated, leading to high computational complexity and high difficulty in hardware implementation. The other is that regardless of the characteristics of signal the measurements generated by GRM are also random, which results in low efficiency of compression coding. In this paper, we design a novel local structural measurement matrix (LSMM) for block-based CS coding of natural images by utilizing the local smooth property of images. The proposed LSMM has two main advantages. First, LSMM is a highly sparse matrix, which can be easily implemented in hardware, and its reconstruction performance is even superior to GRM at low CS sampling sub rate. Second, the adjacent measurement elements generated by LSMM have high correlation, which can be exploited to greatly improve the coding efficiency. Furthermore, this paper presents a new framework with LSMM for block-based CS coding of natural images, including measurement generating, measurement coding and CS reconstruction. Experimental results show that the proposed framework with LSMM for block-based CS coding of natural images greatly enhances the existing CS coding performance when compared with other state-of-the-art image CS coding schemes.

In this paper, the hierarchical frame based video compressed sensing (CS) framework is proposed, which outperforms the traditional framework through the better exploitation of frames correlation with reference frames, the unequal sample subrates setting among frames in different layers and the reduction of the error propagation. By considering the spatial and temporal correlations of the video sequence, a spatial-temporal sparse representation based recovery is proposed for this framework. The similar blocks in both the current frame and these recovered reference frames are composed as a spatial-temporal group, which is defined as the unit of the sparse representation. By exploiting the low dimensional subspace description of each group, the video CS recovery is converted as a low-rank matrix approximation problem, which can be solved by exploiting the hard thresholding and the gradient descent. Experimental results show that the proposed method achieves better performance against both the state-of-art still-image CS recovery algorithms and the existing residual domain based video CS reconstruction approaches.

Different from general algorithms of haze removal and low lighting image enhancement, which only use the information of image to process, this paper adds a reference image to get more information for the algorithm and focuses on enhancing region of interest of an image based on the reference one. With the reference image, the haze one can be divided into Region of Interest (RoI) and Region of no Interest (non-RoI). Furthermore, the reference image can provide more useful information for computing the transmission map and atmospheric light. For the non-RoI region, a more robust transmission map and minimizing reconstruction error cost function based method to estimate atmospheric light has been proposed. Because the atmospheric light is a global variable, the optimized one is also suitable for the RoI region. With the global optimized atmospheric light, an optimized transmission map can be got for the RoI region. The RoI region can be enhanced via the optimal transmission map and atmosphere light. Theoretical analysis gives eloquent proof proving that the proposed method is definitely better than the traditional dark-channel-prior-based methods due to our better transmission map and atmosphere light. Extensive experiments also show the expected results.

Image interpolation is one of the most elementary imaging research topics. A number of image interpolation methods have been developed and tested on uncompressed images in the literature. However, a lot of videos have already been stored or have to be transmitted in compressed format due to the storage limitation or the bandwidth limitation. The existed image interpolation methods may not be efficient when directly applied to compressed images or videos. Inspired by the success of the intra prediction in HEVC and the edge-directed image interpolation methods, a directional intra frame interpolation for HEVC compressed video is proposed. The main idea is to use the directional prediction information in compressed low-resolution video bitstreams to estimate the associated high-resolution video. For intra frames, the prediction direction information is taken into account as context in the directional interpolation. When a pixel is decompressed with a small prediction residual, the interpolation is performed along its block direction. The interpolation weight for each block direction is off-line trained by the Wiener filter based on the representative video sequences. For each pixel with a large prediction residual, a piecewise autoregressive model is used as a regularization term into the interpolation function. Extensive experiments demonstrate that the proposed method achieves better performance than the traditional methods such as Bicubic, KR, LAZA, NEDI and SAI.

With the rapid development of digital video technology, frame rate up conversion is widely used. In this paper, a novel motion vector refinement method for frame rate up conversion on depth based 3D video is proposed. Our method involves two major stages in frame rate up conversion which are motion estimation and motion vector filtering. In the motion estimation process, the depth constraint to block matching algorithm is introduced in the bi-directional motion estimation method to obtain the motion vectors. In the motion vector filtering process, a depth-guided filter is designed to enhance the consistence of motions in the same depth plane. The refined motion vectors are used for frame interpolation. Experimental results show that the proposed method achieves 0.45 dB gain in terms of PSNR on average and improves the visual quality of the frame rate up-converted video.

In this paper, we propose a novel image interpolation algorithm, which is formulated via combining both the local autoregressive (AR) model and the nonlocal adaptive 3-D sparse model as regularized constraints under the regularization framework. Estimating the high-resolution image by the local AR regularization is different from these conventional AR models, which weighted calculates the interpolation coefficients without considering the rough structural similarity between the low-resolution (LR) and high-resolution (HR) images. Then the nonlocal adaptive 3-D sparse model is formulated to regularize the interpolated HR image, which provides a way to modify these pixels with the problem of numerical stability caused by AR model. In addition, a new Split-Bregman based iterative algorithm is developed to solve the above optimization problem iteratively. Experiment results demonstrate that the proposed algorithm achieves significant performance improvements over the traditional algorithms in terms of both objective quality and visual perception.

In this paper, a practical video coding scheme is developed to realize state-of-the-art video coding efficiency with lower encoder complexity at low bit-rate, while supporting standard compliance and error resilience. Such an architecture is particularly attractive for wireless visual communications. At the encoder, multiple descriptions of a video sequence are generated in the spatio-temporal domain by temporal multiplexing and spatial adaptive downsampling. The resulting side descriptions are interleaved with each other in temporal domain, and still with conventional square sample grids in spatial domain. As such, each side description can be compressed without any change to existing video coding standards. At the decoder, each side description is first decompressed, and then reconstructed to original resolution with the help of the other side description. In this procedure, the decoder recover the original video sequence in a constrained least squares regression process, using 2D or 3D piecewise autoregressive model according to different prediction modes. In this way, the spatial and temporal correlation is sufficiently explored to achieve superior quality. Experiment results demonstrate the proposed video coding scheme outperforms H.264 in rate-distortion performance at low bit-rates and achieves superior visual quality at medium bit-rates as well.

In this paper, a mode-dependent intra frame interpolation method is proposed for H.264/AVC compressed video. The intra prediction mode information is taken into account in the interpolation filter design. For each intra prediction mode, an optimal Wiener filter is trained based on the representative video sequences. Therefore the trained filter is adaptive to the intra prediction mode. Furthermore, the quantization parameter is also explored as context information for filter selection. Extensive experiments demonstrate that the proposed method achieves better performance than the traditional methods such as Bicubic, Bilinear, LAZA and NEDI, while keeping low computational complexity.