Deep neural networks using generative diffusion prior have provided the state-of-the-art performances for the task of blind image super resolution. Thanks to their powerful image generation capability, these deep networks are able to produce high-quality visual signals with realistic textures and structures. However, since these schemes employ a very large number of parameters, their training process is often difficult, and therefore, their performances can be limited. In order to address this, in this paper, we propose a diffusion-based blind image super resolution scheme, which by using a novel learning algorithm with invertible neural networks, is able to provide superior results. Specifically, we argue that because of the reversibility property of invertible neural networks, they are able to generate degraded low-quality images, whose super resolved versions are the upper bound of the image super resolution function space. The inclusion of such visual signals in the training process of our blind image super resolution network leads to facilitating the learning paradigm and achieving higher performances. We show that our proposed blind image super resolution scheme is able to outperform the state-of-the-art methods.

Federated Learning (FL) is a privacy-preserving algorithm that enables multiple clients to collaboratively train a global model without sharing their local data. This learning algorithm is particularly valuable in privacy-sensitive applications such as Human Activity Recognition (HAR), where users are reluctant to share their personal data. However, a conventional FL system suffers from data heterogeneity and communication overhead. To address these issues, we propose an efficient FL algorithm for image-based HAR using optimal server representations (OSR). OSR efficiently selects a representative set of privacy-preserved images for transmission to the server and improves the global model quality by training on privacy-preserved data. Our comprehensive experiments carried out on three public datasets, namely Stanford40, PPMI, and VOC2012, demonstrate the superiority of OSR in terms of performance and bandwidth usage compared to state-of-the-art approaches.

Diffusion models have provided the state-of-the-art performances for different computer vision tasks, including the task of underwater image enhancement. One of the challenges in the task of underwater image enhancement is that various spatial regions of the image require different restoration techniques. In order to address this, we propose a novel diffusion-based underwater image enhancement network, in which by employing the two ideas of uncertainty-aware learning and feature recalibration based on the color tones dominated in the underwater environments, it is able to provide superior performances. Specifically, the former idea strives to process various spatial regions of the underwater image based on their restoration uncertainty, while the latter technique recalibrates the features generated by the diffusion model by taking the various color tones in the underwater environments into consideration. The results of different experimentations show the superiority of the proposed diffusion-based model over the other state-of-the-art underwater image enhancement networks.

JPEG is one of the most popular image compression techniques , with numerous applications ranging from medical imaging to surveillance systems. Since JPEG introduces the blocking artifacts to the decompressed visual signals, enhancing the quality of these images is of paramount importance . Recently, various deep neural networks have been proposed for JPEG image deblocking that can effectively reduce the blocking artifacts produced by the JPEG compression technique. However, most of these schemes could only handle decompressed images generated by a set of specific JPEG quality factor (QF) values employed in the network training process. Therefore, when the images are obtained by the JPEG QF values other than those used in the network training process, the performance of deep learning-based JPEG image deblocking schemes drops significantly. To address this, in this paper, we propose a novel deep learning-based blind JPEG image deblocking method, which employs out-of-distribution detection to perform deblocking efficiently for various quality factor (QF) values. The proposed scheme can distinguish between the decompressed images using the QF values used in the training set and those using the QF values not used in the training set, and then, a suitable deblocking strategy for generating high-quality images is developed. The proposed scheme is shown to outperform the state-of-the-art JPEG image deblocking methods for various QF values.

High-frequency components are the most crucial parts of the visual signals for the task of image super resolution. The deep image super resolution networks that are able to process the high-frequency components efficiently can provide high performances. In view of this, in this paper, we develop a new residual block for image super resolution, in which the feature attention process is carried out by focusing on various high-frequency components of the feature tensors. Specifically, we design a novel multi-dimensional filter design technique for the task of image super resolution, and employ it for obtaining a finite impulse response (FIR) high-pass filter bank to be embedded in a deep super resolution network for the feature attention process. Moreover, we utilize two other feature attention processes in the proposed residual block, namely, multi-scale transformerbased and convolutional learnable feature attention mechanisms, to generate rich sets of feature maps for a deep super resolution network. The results of different experiments demonstrate the effectiveness of the various modules of the proposed residual block in enhancing the super resolution performance

Feature attention is a technique used in deep neural networks to provide a discriminative processing of the various regions in an image based on their significance for enhancing the image restoration performance. In this paper, we develop a novel image restoration network, in which the feature maps extracted by the network are recalibrated using a pixel-wise feature attention and the recalibration process is guided by the structural and textural information of the image resulting from the Otsu’s method for its segmentation. It is shown that using this segmentation guidance strategy for recalibrating feature maps is indeed helpful in enhancing the quality of the restored images. The proposed image restoration network outperforms the state-of-the-art light-weight image restoration networks on benchmark datasets.

Deep light-weight image super resolution networks that provide a high performance have numerous real-life applications, such as mobile devices and multimedia systems. Hence, analyzing the capability of such deep networks in providing a similar performance between the cases that they are applied to the images with and without distributions similar to that of the training is crucial. In this paper, we carry out the robustness analysis of the deep state-of-the-art light-weight super resolution networks by proposing and using three metrics that are based on the statistical information of the super resolved images in both pixel level and feature level. The results of our metrics for the deep state-of-the-art light-weight super resolution networks demonstrate the behavior of such networks against realistic distribution shift in the test dataset.

Morphological operations are nonlinear mathematical operations that are capable of performing signal processing tasks based on the structures and textures of the signals. With this motivation of the capability of morphological operations, in this paper, a novel residual block that can generate morphological features of images and fuse them with the conventional hierarchical features has been proposed. The proposed residual block is then used to design a light-weight deep neural network architecture in a residual framework for the task of image super resolution. It is shown that a fusion of morphological features of images with the conventional hierarchical features can improve the super resolution capability of a deep convolutional network. Experiments are performed to demonstrate the effectiveness of the proposed idea of using morphological operations and the superiority of the network designed based on this idea in super resolving low quality images.

The design of ultralight-weight super-resolution convolutional neural networks capable of providing images with high visual quality is crucial in many real-world applications with limited power and storage capacity, such as mobile devices and portable cameras. In this paper, a new ultralight-weight super-resolution network, based on the idea of using multiresolution level feature interpolation in a residual framework, is developed. In the proposed network, the multiple resolution level interpolated features generated are fused and the resulting feature maps are added to the residual features obtained from a shallow convolutional neural network. The proposed network is applied to various benchmark datasets and is shown to outperform the state-of-the-art ultralight-weight image super-resolution networks existing in the literature.

The JPEG compression scheme introduces blocking artifacts when the images are decompressed. JPEG image deblocking schemes based on deep neural networks map a JPEG decompressed image to its corresponding deblocked image. Employing a residual block that is capable of generating a rich set of high frequency residual features in a deep JPEG image deblocking network can improve its representational capability, and therefore, enhance the network performance. In this paper, we propose two residual blocks that generate rich high frequency residual features. The first residual block generates features from the high frequency component of its input signal in addition to generating conventional hierarchical residual features using convolutional operations. The second one is a fractal residual block that is developed by replacing the conventional convolutions in the first block by the block itself. The two proposed residual blocks are, respectively, used in recursive (non-fractal) and non-recursive (fractal) neural networks for the task of JPEG deblocking. The results of the experiments performed on the two proposed deblocking networks show their performance superiority over the respective state-of-the-art deblocking networks.

Residual blocks use skip connections in order to facilitate the flow of information in the network and thus, provide a good network performance. As different objects in a generic image appear at different scales, employing a multi-scale feature generation module in a residual block for image super resolution can further improve the network performance. In this paper, a new residual block that generates features at multiple scales is proposed for the task of image super resolution. In order to enhance the representational capability of the network while keeping its complexity low, the proposed residual block uses two different feature generation techniques, namely, multi-scale granular channel feature generation and uni-scale holistic channel feature generation, and fuses their output feature maps. It is shown that the network using the proposed residual block outperforms the state-of-the-art lightweight super resolution networks on four benchmark datasets with various scaling factors.

Deep neural networks use a nonlinear end-to-end mapping in order to transform a low resolution image to the high resolution one. Residual blocks facilitate the flow of the information in deep neural networks and enhance the network performance. In this paper, a new residual block that enhances the representational capability of a super resolution network is proposed. The proposed residual block combines the features generated in various receptive fields using different hierarchical levels of convolution operations or convolution operations in conjunction with the space-to-depth and depth-to-space operations in order to provide a rich set of residual features. The experimental results demonstrate the superiority of the super resolution network using the proposed residual block over the state-of-the-art light-weight super resolution networks in terms of objective and subjective metrics.

Deep convolutional networks provide very high quality super resolution images through a learning process by a nonlinear end-to-end mapping between low and high resolution images. Many of the state-of-the-art super resolution networks employ residual blocks in their network architectures, where in each residual block the high frequency residual signals are added to the feature maps input to the block. In this paper, a new residual block is proposed for the problem of image super resolution. The proposed residual block consists of three modules, namely, feature transformation module, nonlinear edge extraction module and feature fusion module. The feature transformation module produces high frequency residual signals and the nonlinear edge extraction module extracts the edges of the features input to the block. These generated high frequency features are then fused using the feature fusion module in order to produce a very rich set of high frequency residual features. The performance of the super resolution network using the proposed residual block is compared with that of the state-of-the-art light-weight super resolution schemes on four benchmark datasets. It is shown that the proposed super resolution scheme outperforms the state-of-the-art light-weight super resolution networks, when both the performance and number of parameters of the network are simultaneously taken into consideration.

Deep image super resolution networks use a nonlinear end-to-end mapping between the low and high resolution versions of an image and therefore, provide a good performance. As the different parts of a single image appear in different scales, developing a deep learning based image super resolution scheme that is capable of generating features at different scales and levels is essential. In this paper, a new residual block is proposed with a view of generating a rich set of features extracted at different scales and levels. The development of the proposed block is carried out using two distinct strategies, the first one focussing on generating features directly in two different scales, whereas the second one aims at generating multi-scale features indirectly by extracting them from two different hierarchical levels of abstraction. It is shown through experimental results that the proposed scheme of designing the residual block results in a network that provides a superior performance with reduced number of parameters than that provided by the light-weight networks using other types of residual blocks.

Restoration of a blurred and subsampled image is an ill-posed problem. In this paper, a two-stage convolutional network is proposed to carry out the processes of upsampling and deblurring to restore the original image. The main idea in the proposed scheme is that the deblurring process is attempted on a high PSNR image obtained after removing the ringing effect that is necessarily caused by the upsampling process. The evaluation of the proposed scheme is carried out using a benchmark dataset in terms of PSNR. The scheme is shown to outperform the state-of-the-art schemes, namely, the sparse coding network, the non-local means filters and the centralized sparse representation.

Image compression is a field in image processing that tries to remove redundant information in an image. Losing the information in lossy compression techniques such as JPEG results to artifacts in the decompressed image that necessitates the image restoration. In this work, a new image restoration scheme based on deep neural nets and maxout activation functions for the application of image deblocking is proposed. Experimental results are presented to demonstrate the superiority of the proposed method both in terms of subjective and objective metrics.

In this paper, a new scheme for single image super resolution using convolutional neural networks and subband reconstruction theory is proposed. In the design of the network, which is referred to as SRSubBandNet, each subband of the residual signal between the high and low resolution images is reconstructed from all the previous subbands. Skip connections between the first, middle and the last SRBs are utilized to address the gradient vanishing problem in the proposed network. SRSubBandNet provides competitive results in terms of both subjective and objective qualities when applied to various benchmark datasets.