Face super-resolution (FSR) aims to reconstruct high-resolution (HR) face images from the low-resolution (LR) inputs. The advancement of FSR has been significantly accelerated by the application of convolutional neural networks in recent years. However, most existing FSR methods are still unsatisfactory in recovering facial texture details. Particularly in the high-magnification face image reconstruction tasks, it is difficult to accurately restore the facial detail features. To address this problem, we propose a Hierarchical Texture Reconstruction Network for FSR with Facial Prior (HTR-FSRNet), which performs hierarchical processing of textures in different regions of face images by proposing new local texture reconstruction modules, thereby better restoring the texture details. The HTR-FSRNet consists of two stages. In the first stage, we introduce a Local-Global Feature Enhancement Module (LGEM) to establish contextual dependencies. Then, based on the traditional convolutional feature extraction method, we develop a new Local Variance Adaptive Convolution (LVAC) module, specifically designed to handle texture-rich regions in face images. In the second stage, we develop a Local Pixel Adjustment Module (LPAM), which not only alleviates artifacts generated during the image restoration process but also reconstructs face regions with relatively lower texture complexity from a new perspective. With this design, we can reconstruct the facial structure and more specifically restore different texture regions. Meanwhile, our method is applicable to the tasks of high-magnification face image reconstruction. Experimental results confirm the superior performance of the proposed HTR-FSRNet.

In low-light scenarios, images often suffer from issues like insufficient brightness, blurred details, and color distortion, failing subsequent tasks such as surveillance and object detection. Most existing Low-Light Image Enhancement(LLIE) methods focus on improving the brightness without consideration of brightness fidelity and texture reconstruction, leading to the processed images being either overexposed or too dark, as well as poor details. To solve this problem, we propose a novel LLIE framework based on hierarchical Discrete Cosine Transform (DCT), named DCTLLIE. The DCTLLIE consists of two stages: brightening and texture reconstruction. In the brightening stage, we design a DCT-Lighter (DCTL) block, which refines the values of low-frequency DCT coefficients to increase image brightness. As images' brightness information often resides in the low-frequency region of DCT, we design a hierarchical DCT mechanism to separate the low-frequency information for brightness and the high-frequency information for initial texture optimization. In the texture reconstruction, we employ dual parallel branches, the Hybrid Spatial-Frequency Processing (HSFP) block and the Long-Range Dependency Learning (LRDL) block, to further optimize the image texture. With this design, we can enhance the brightness of low-light images and optimize their texture simultaneously. Experimental results validate the superior performance of the proposed DCTLLIE.

Night photography often struggles with challenges like low light and blurring, stemming from dark environments and prolonged exposures. Current methods either disregard priors and directly fitting end-to-end networks, leading to inconsistent illumination, or rely on unreliable handcrafted priors to constrain the network, thereby bringing the greater error to the final result. We believe in the strength of data-driven high-quality priors and strive to offer a reliable and consistent prior, circumventing the restrictions of manual priors. In this paper, we propose Clearer Night Image Restoration with Vector-Quantized Codebook (VQCNIR) to achieve remarkable and consistent restoration outcomes on real-world and synthetic benchmarks. To ensure the faithful restoration of details and illumination, we propose the incorporation of two essential modules: the Adaptive Illumination Enhancement Module (AIEM) and the Deformable Bi-directional Cross-Attention (DBCA) module. The AIEM leverages the inter-channel correlation of features to dynamically maintain illumination consistency between degraded features and high-quality codebook features. Meanwhile, the DBCA module effectively integrates texture and structural information through bi-directional cross-attention and deformable convolution, resulting in enhanced fine-grained detail and structural fidelity across parallel decoders. Extensive experiments validate the remarkable benefits of VQCNIR in enhancing image quality under low-light conditions, showcasing its state-of-the-art performance on both synthetic and real-world datasets. The code is available at https://github.com/AlexZou14/VQCNIR.

Low-light image enhancement is a computer vision task that aims to improve the visual perceptual quality of images captured in poorly illuminated scenes. At present, deep learning-based low-light enhancement methods can obtain high-quality enhanced images. However, it does not consider the statistical characteristics of different regions, such as edge, structure, and texture. The uncertainty of image regions is not well characterized and utilized. To address this problem, we propose a novel UnCertainty-driven Cycle-Consistent Generative Adversarial Network (UTrCGAN) to improve the performance of low-light enhancement. UTrCGAN first decomposes the unpaired low/normal-light images into reflectance and illumination components based on the Retinex theory. Then a generative adversarial network guided by uncertainty constraint is proposed to enhance the illumination component, in which the quality of the enhanced image is further improved by the guidance of variance estimation. Experimental results on the widely-used LOL dataset show that UTrCGAN outperforms the state-of-the-art methods in terms of visual quality and quantitative metrics.

While single-image super-resolution (SISR) methods based on convolutional neural networks (CNNs) have made remarkable progress, the increasing number and width of convolutional layers pose challenges due to heightened demands on computing resources and memory. In response to this issue, researchers have introduced several lightweight CNN models, among which the feature distillation network has emerged as a prominent solution. Motivated by the success of feature distillation networks and convolutional modulation (Conv2Former), we propose a straightforward convolutional modulation feature distillation network (CFDN). Dilated convolution is employed to expand the receptive field, thereby enhancing the performance of Conv2Former. Simultaneously, we integrate channel attention with a modified convolutional modulation block. Additionally, to optimize network performance, we introduce a spatial attention block. In comparison to other lightweight CNN SR models, our CFDN demonstrates commendable performance across various SR benchmarks.

Semantic segmentation has recently achieved notable advances by exploiting “class-level” contextual information during learning, e.g., the Object Contextual Representation (OCR) and Context Prior (CPNet) approaches. However, these approaches simply concatenate class-level information to pixel features to boost pixel representation learning, which cannot fully utilize intra-class and inter-class contextual information. Moreover, these approaches learn soft class centers based on coarse mask prediction, which is prone to error accumulation. To better exploit class-level information, we propose a universal Class-Aware Regularization (CAR) approach to optimize the intra-class variance and inter-class distance during feature learning, motivated by the fact that humans can recognize an object by itself no matter which other objects it appears with. Moreover, we design a dedicated decoder for CAR (named CARD), which consists of a novel spatial token mixer and an upsampling module, to maximize its gain for existing baselines while being highly efficient in terms of computational cost. Specifically, CAR consists of three novel loss functions. The first loss function encourages more compact class representations within each class, the second directly maximizes the distance between different class centers, and the third further pushes the distance between inter-class centers and pixels. Furthermore, the class center in our approach is directly generated from ground truth instead of from the error-prone coarse prediction. CAR can be directly applied to most existing segmentation models during training, including OCR and CPNet, and can largely improve their accuracy at no additional inference overhead. Extensive experiments and ablation studies conducted on multiple benchmark datasets demonstrate that the proposed CAR can boost the accuracy of all baseline models by up to 2.23% mIOU with superior generalization ability. CARD outperforms state-of-the-art approaches on multiple benchmarks with a highly efficient architecture. The code will be available at https://github.com/edwardyehuang/CAR.

In this paper, we propose a Cross-Modal Face Super-Resolution (CMFSR) method to construct high-resolution (HR) facial images from low-resolution (LR) cross-modal facial images captured respectively by disjoint visible light (VIS) and near-infrared (NIR) cameras. Due to the coupling of modality transformation and information fusion, CMFSR is more difficult to obtain HR reconstructed results compared with traditional super-resolution. To solve this problem, a Quasi-Siamese Domain Transfer Fusion Network (QSDTFN) for CMFSR is proposed in this paper, whose two branches transfer two LR face modality to HR face modality by domain transfer respectively. Different from two completely independent branches in the traditional pseudo-siamese network, only the HR-to-LR face transfer processes of the two branches in our quasi-siamese network are independent, while the LR-to-HR face transfer processes are coupled. This coupled module called the Adaptive Weighted Domain Transfer Fusion Module (AWDTFM) disentangles the modality and identity information in the two LR faces, thus achieving modality transformation and identity information fusion simultaneously. In order to strengthen the optimization on the process of CMFSR, this method further introduces the backward QSDTFN to form a higher-level bidirectional structure with the forward QSDTFN, and specifically designs two types of losses: intra-network loss and inter-network loss, to constrain the modality and identity consistencies within one QSDTFN and between two QSDTFNs respectively. The experimental results on the challenging LR cross-modal face datasets demonstrate that the proposed method performs favorably against the state-of-the-art methods.

Since deep convolutional neural network (CNN) has achieved excellent results in single image super-resolution (SISR), an increasing number of methods based on CNN have been proposed. Most CNN-based methods are devoted to finding mapping based on pixel intensity while ignoring the importance of frequency information, which can reflect semantic information of images on different bands. This leads to less effectiveness in the reconstruction of high-frequency details. To address this problem, we propose a novel CNN-based super-resolution method named joint wavelet sub-bands guided network (JWSGN). We separate the different frequency information of the image by the WT and then recover this information by a multi-branch network. To recover finer edge details, we propose an edge extraction module, which estimates an edge feature map by using the similarity of all high-frequency sub-bands and then corrects the high-frequency features recovered from each branch by exploiting the edge feature map. Furthermore, we use the complementary relationship between different frequencies to calibrate the high-frequency sub-bands. Finally, the high-resolution image is obtained by inverse wavelet transform. Both qualitative and quantitative experiments show that our method performs excellent performance with the guidance of the edge extraction module.

In modern networks, Lawful Interception (LI) is one of the necessary means for security agencies to safeguard national security and prevent crimes. This article intends to explore the dynamic placement problem of intercept access points (IAPs) unique to SDNs, and their derived interception strategies. by selecting the optimal intercept access point, bring the shortest path among the three points-source, destination, and law enforcement agency (LEA). In order to reduce the redundant return intercepted traffic in the network, the operating cost of the LI system must be minimized. We also addressed the performance of the fastest access to the return traffic by the LEA, and the impact on the communication quality of the monitored users. Comparison among the three-interception strategies-source/destination interception model, blocking interception model, and bicast model. the simulation results expressed as a percentage that improved by using the different strategies compared with each other of the total costs of ISP, the performance of LEAs, and affected QoS of monitored users.

Blind deblurring has received considerable attention in recent years. However, state-of-the-art methods often fail to process saturated blurry images. The main reason is that pixels around saturated regions are not conforming to the commonly used linear blur model. Pioneer arts suggest excluding these pixels during the deblurring process, which sometimes simultaneously removes the informative edges around saturated regions and results in insufficient information for kernel estimation when large saturated regions exist. To address this problem, we introduce a new blur model to fit both saturated and unsaturated pixels, and all informative pixels can be considered during the deblurring process. Based on our model, we develop an effective maximum a posterior (MAP)-based optimization framework. Quantitative and qualitative evaluations on benchmark datasets and challenging real-world examples show that the proposed method performs favorably against existing methods.

Deblurring night blurry images is difficult, because the common-used blur model based on the linear convolution operation does not hold in this situation due to the influence of saturated pixels. In this paper, we propose a non-blind deblurring network (NBDN) to restore night blurry images. To mitigate the side effects brought by the pixels that violate the blur model, we develop a confidence estimation unit (CEU) to estimate a map which ensures smaller contributions of these pixels in the deconvolution steps which are optimized by the conjugate gradient (CG) method. Moreover, unlike the existing methods using manually tuned hyper-parameters in their frameworks, we propose a hyper-parameter estimation unit (HPEU) to adaptively estimate hyper-parameters for better image restoration. The experimental results demonstrate that the proposed network performs favorably against state-of-the-art algorithms both quantitatively and qualitatively.

Owing to the excellent visual quality of results, Gatys et al.'s Neural Style Transfer (NST) online algorithm is regarded as the gold-standard in the community of NST, but this algorithm is quite time-consuming especially for high-resolution (HR) image. In this letter, we propose “One-Shot” super-resolution (SR) for fast high-resolution style transfer. We first generate a low-resolution (LR) stylized image by NST, and then use “One-Shot” super-resolution to restore the HR stylized image by learning the mapping relations between HR-LR stylized images from HR-LR style images. However, due to the style loss is not eliminated, there are some subtle but important fine-grained style differences between LR stylized and style images. These differences lead to the poor visual quality of SR results. To reduce the style differences further, we adjust the texture of LR style image to approach LR stylized image by backward style transfer. The result of backward style transfer will be treated as the LR part of the “One-Shot” example pair, which leads to a better SR. The experimental results show that with good visual quality, our method reduces the time consumption by 81.6%. Especially in a specific application scenario of fixed style image and changed content image, our method reduces the time consumption by 89.3%.

Face Super-Resolution (FSR) aims to infer High-Resolution (HR) face images from the captured Low-Resolution (LR) face image with the assistance of external information. Existing FSR methods are less effective for the LR face images captured with serious low-quality since the huge imaging/degradation gap caused by the different imaging scenarios (i.e., the complex practical imaging scenario that generates test LR images, the simple manual imaging degradation that generates the training LR images) is not considered in these algorithms. In this paper, we propose an image homogenization strategy via re-expression to solve this problem. In contrast to existing methods, we propose a homogenization projection in LR space and HR space as compensation for the classical LR/HR projection to formulate the FSR in a multi-stage framework. We then develop a re-expression process to bridge the gap between the complex degradation and the simple degradation, which can remove the heterogeneous factors such as serious noise and blur. To further improve the accuracy of the homogenization, we extract the image patch set that is invariant to degradation changes as Robust Neighbor Resources (RNR), with which these two homogenization projections re-express the input LR images and the initial inferred HR images successively. Both quantitative and qualitative results on the public datasets demonstrate the effectiveness of the proposed algorithm against the state-of-the-art methods.

In this paper, we propose a face super-resolution (FSR) method to handle the decreasing face recognition rate caused by low-quality images. To better model the input images, we build a nearest neighbor network (NNN) which consists of nodes and paths by introducing the second-layer nearest neighbors (SLNNs), where the paths of the network represent the distance between nodes. As the SLNN is trained in the high-resolution (HR) space and is exponentially supplementary to the traditional first-layer nearest neighbors (FLNNs), the neighbor inadequacy problem can be effectively solved by enriching the neighbor candidate set via NNN. Furthermore, we solve the NNN for the optimal weights of neighbors. Finally, we fuse the refined weights and neighbors for better reconstruction results. The effectiveness of this fusion strategy is validated by both quantitative and qualitative experimental results. The extensive experimental results on the public face datasets and real-world challenging low-resolution (LR) images demonstrate that the proposed method performs favorably against the state-of-the-art methods.

Because Face Super-Resolution (FSR) tends to infer High-Resolution (HR) face image by breaking the given Low- Resolution (LR) image into individual patches and inferring the HR correspondence one patch by one separately, Super- Resolution (SR) of face images with serious degradation, especially with occlusion, is still a challenging problem of the computer vision field. To address this problem, we propose a patch-level face model for FSR, which we called the position relation model. This model consists of the mapping relationships in every face position to the rest of the face positions based on similarity. In other words, we build a constraint for each patch position via the relationship in this model from the global range of face. Once an individual input LR image patch is seriously deteriorated, the substitute patch in whole face range can be sought according to the relationship of the model at this position as the provider of the LR information. In this way, the lost facial structures can be compensated by knowledge located in remote pixels or structure information which leads to better high-resolution face images. The LR images with degradations, not only the serious low-quality degradation, e.g. noise, blur, but also the occlusions, can be effectively hallucinated into HR ones. Quantitative and qualitative evaluations on the public datasets demonstrate that the proposed algorithm performs favorably against state-of-theart methods.

In this paper, we focus on restoring high-resolution facial images under noisy low-resolution scenarios. This problem is a challenging problem as the most important structures and details of captured facial images are missing. To address this problem, we propose a novel local patch-based face super-resolution (FSR) method via the joint learning of the contextual model. The contextual model is based on the topology consisting of contextual sub-patches, which provide more useful structural information than the commonly used local contextual structures due to the finer patch size. In this way, the contextual models are able to recover the missing local structures in target patches. In order to further strengthen the structural compensation function of contextual topology, we introduce the recognition feature as additional regularity. Based on the contextual model, we formulate the super-resolved procedure as a contextual joint representation with respect to the target patch and its adjacent patches. The high-resolution image is obtained by weighting contextual estimations. Both quantitative and qualitative validations show that the proposed method performs favorably against state-of-the-art algorithms.

Face Super Resolution(FSR) is to infer High Resolution(HR) facial images from given Low Resolution(LR) ones with the assistance of LR and HR training pairs. Among existing methods, local patch based methods are superior in visual and objective quality than global based methods. These local patch based methods are based on the consistency assumption that the neighbors in HR/LR space form similar local geometry. But when LR images are with low quality, the LR space is seriously contaminated that even two distinct patches look similar, which means that the consistency assumption is not well held anymore. To this end, in this paper we introduce the contextual topological structure of target patch to improve the consistency. The contextual topological structure consists of the target patch as well as its adjacent patches, we explore the relationship between them based on statistical probability and apply the relationship for joint learning progress of mapping from LR to HR. By incorporating the contextual topological structure, the robustness to noise of approach is increased as well as the LR/HR consistency. The effectiveness of proposed method is verified both quantitatively and qualitatively.

Face Super Resolution(FSR) is to infer High Resolution(HR) facial images from given Low Resolution(LR) ones with the assistance of LR and HR training pairs. Among existing methods, local patch based methods are superior in visual and objective quality than global based methods. These local patch based methods are based on the consistency assumption that the neighbors in HR/LR space form similar local geometry. But when LR images are Very Low Quality(VLQ), the LR space is seriously contaminated that even two distinct patches look similar, which means that the consistency assumption is not well held anymore. To this end, in this paper we use the target patch as well as the surrounding pixels, which we called parent patch, to represent the target patch. By incorporating the peripheral information, the parent patch is much more robust to noise in the LR and HR consistency learning. The effectiveness of proposed method is verified both quantitatively and qualitatively.

Most state-of-the-art face hallucination approaches suffer from complicated learning patterns and highly intensive computation, which will lead to low efficiency and considerable computing resources. Therefore, how to restore real face image quickly and efficiently is still an important issue in this field. To solve or partially solve the problem, this paper proposed a novel facial standard deviation prior based approach which can provide superior results with high efficiency for real face images. The high frequency information of test image will be enhanced via a facial specific sharpening operator which is obtained through the learning of standard deviation correspondence of training set. Experiments in simulation and real world images verified the effectiveness of proposed approach, and the distinct advantage on runtime and resource requirement of proposed approach.

This paper describes a novel method for single-image super-resolution (SR) based on a neighbor embedding technique which uses coupled feature spaces under surveillance scenarios. For surveillance face images, traditional neighbor embedding SR approaches could not offer counterintuitive results because consistency between high resolution images and low resolution images is destroyed by serious noise which caused by environmental impact factors and large distance between the camera and objects. In order to reinforce the consistency, we extend the learning space from single to a coupled feature space that combine image intensity feature and contour model. The contour model describes facial contour information as images generated from original low resolution ones. Simulation experiments show that this proposed approach could provide competitive results in simulation experiments in subjective and objective quality. Even in surveillance scenario the proposed method outperforms the traditional methods.

As the face image captured by a surveillance camera is typically very low-resolution (LR), blurred and noisy, traditional neighbor embedding method considers only one manifold (the LR image manifold) and fails very often to reliably estimate the intention geometrical structure. In this paper, we introduce the notion of neighbor embedding from the LR image manifold and the high-resolution (HR) one simultaneously and propose a novel neighbor embedding model, termed the coupled-layer neighbor embedding (CLNE), for surveillance face hallucination. CLNE differs substantially from other neighbor embedding models in that the former has two layers: the LR layer and the the HR layer. The LR layer in this model is the local geometrical structure of the LR patch manifold, which is characterized by the reconstruction weights; the HR layer in this model is a set of HR training patches that guide the K-nearest neighbor (K-NN) searching and geometrically constrain the reconstruction weights. By this coupled constraint paradigm between the adaptation of the LR layer and the HR one, CLNE can achieve a more robust neighbor embedding through the significant degradation process. Indeed, the experimental results confirm that our method outperforms the related state-of-the-art methods by having better objective values as well as better visual results.