Imbalanced label distribution is usually the case for real-world data, which poses a challenge for training unbiased recognition model. In this paper, we study two underlying mismatches, i.e., distribution mismatch and probability space mismatch, present in class-imbalanced learning. Firstly, we analyze the label distribution mismatch between imbalanced training data and balanced test data, and introduce a distribution unified framework to unify the two distributions through probability conversion. Secondly, we analyze that the utilization of cross-entropy loss under the proposed framework may lead to probability space mismatch, where the conversion of the predictive probability is implemented in softmax probability space while the comparison with one-hot label is implemented in true probability space. To alleviate this dilemma, we involve a teacher model and formulate a teacher-student learning strategy, which contains two novel techniques. The Teacher Guided Label Smoothing (TGLS) is first proposed to relax the one-hot label to smoother pseudo softmax probability, which is more aligned with the softmax probability space. Additionally, we propose Distribution Unified Knowledge Distillation (DU-KD) under the proposed framework to further reduce both the mismatches. Experiments on several benchmarks confirm the top-level performance of the proposed method.

With the springing up of face synthesis techniques, it is prominent in need to develop powerful face forgery detection methods due to security concerns. Some existing methods attempt to employ auxiliary frequency-aware information combined with CNN backbones to discover the forged clues. Due to the inadequate information interaction with image content, the extracted frequency features are thus spatially irrelavant, struggling to generalize well on increasingly realistic counterfeit types. To address this issue, we propose a Spatial-Frequency Dynamic Graph method to exploit the relation-aware features in spatial and frequency domains via dynamic graph learning. To this end, we introduce three well-designed components: 1) Content-guided Adaptive Frequency Extraction module to mine the content-adaptive forged frequency clues. 2) Multiple Domains Attention Map Learning module to enrich the spatial-frequency contextual features with multiscale attention maps. 3) Dynamic Graph Spatial-Frequency Feature Fusion Network to explore the high-order relation of spatial and frequency features. Extensive experiments on several benchmark show that our proposed method sustainedly exceeds the state-of-the-arts by a considerable margin.

Large vocabulary object detectors are often faced with the long-tailed label distributions, seriously degrading their ability to detect rarely seen categories. On one hand, the rare objects are prone to be misclassified as frequent categories. On the other hand, due to the limitation on the total number of detections per image, detectors usually rank all the confidence scores globally and filter out the lower-ranking ones. This may result in missed detection during inference, especially for the rare categories that naturally come with lower scores. Existing methods mainly focus on the former problem and design various classification loss to enhance the object-level classification accuracy, but largely overlook the global-level ranking task. In this paper, we propose a novel framework that Reconciles Object-level and Global-level (ROG) objectives to address both problems. As a multi-task learning framework, ROG simultaneously trains the model with two tasks: classifying each object proposal individually and ranking all the confidence scores globally. Specifically, complementary to the object-level classification loss for model discrimination, we design a generalized average precision (GAP) loss to explicitly optimize the global-level score ranking across different objects. For each category, GAP loss generates balanced gradients to rectify the ranking errors. In experiments, we show that GAP loss is highly versatile to be plugged into various advanced methods and brings considerable benefits. Code is at https://github.com/EricZsy/ROG.

Causal inference in the field of infectious disease attempts to gain insight into the potential causal nature of an association between risk factors and diseases. Simulated causality inference experiments have shown preliminary promise in improving understanding of the transmission of infectious diseases but still lack sufficient quantitative causal inference studies based on real-world data. Here, we investigate the causal interactions between three different infectious diseases and related factors, using causal decomposition analysis, to characterize the nature of infectious disease transmission. We show that the complex interactions between infectious disease and human behavior have a quantifiable impact on transmission efficiency of infectious diseases. Our findings, by shedding light on the underlying transmission mechanism of infectious diseases, suggest that causal inference analysis is a promising approach to determine epidemiological interventions.

3D facial landmark detection is extensively used in many research fields such as face registration, facial shape analysis, and face recognition. Most existing methods involve traditional features and 3D face models for the detection of landmarks, and their performances are limited by the hand-crafted intermediate process. In this paper, we propose a novel 3D facial landmark detection method, which directly locates the coordinates of landmarks from 3D point cloud with a well-customized graph convolutional network. The graph convolutional network learns geometric features adaptively for 3D facial landmark detection with the assistance of constructed 3D heatmaps, which are Gaussian functions of distances to each landmark on a 3D face. On this basis, we further develop a local surface unfolding and registration module to predict 3D landmarks from the heatmaps. The proposed method forms the first baseline of deep point cloud learning method for 3D facial landmark detection. We demonstrate experimentally that the proposed method exceeds the existing approaches by a clear margin on BU-3DFE and FRGC datasets for landmark localization accuracy and stability, and also achieves high-precision results on a recent large-scale dataset.

Mixup is a popular data augmentation method, with many variants subsequently proposed. These methods mainly create new examples via convex combination of random data pairs and their corresponding one-hot labels. However, most of them adhere to a random sampling and mixing strategy, without considering the frequency of label occurrence in the mixing process. When applying mixup to long-tailed data, a label suppression issue arises, where the frequency of label occurrence for each class is imbalanced and most of the new examples will be completely or partially assigned with head labels. The suppression effect may further aggravate the problem of data imbalance and lead to a poor performance on tail classes. To address this problem, we propose Label-Occurrence-Balanced Mixup to augment data while keeping the label occurrence for each class statistically balanced. In a word, we employ two independent class-balanced samplers to select data pairs and mix them to generate new data. We test our method on several long-tailed vision and sound recognition benchmarks. Experimental results show that our method significantly promotes the adaptability of mixup method to imbalanced data and achieves superior performance compared with state-of-the-art long-tailed learning methods.

We classify the discontinuity of loss in both five-param and eight-param rotated object detection methods as rotation sensitivity error (RSE) which will result in performance degeneration. We introduce a novel modulated rotation loss to alleviate the problem and a rotation sensitivity detection network (RSDet) which consists of an eight-param single-stage rotated object detector and the modulated rotation loss. Our proposed RSDet has several advantages: 1) it reformulates the rotated object detection problem as predicting the corners of objects while most previous methods employ a five-param-based regression method with different measurement units. 2) modulated rotation loss achieves consistent improvement on both five-param and eight-param rotated object detection methods by solving the discontinuity of loss. To further improve the accuracy of our method on objects smaller than 10 pixels, we introduce a novel RSDet++ which consists of a point-based anchor-free rotated object detector and a modulated rotation loss. Extensive experiments demonstrate the effectiveness of both RSDet and RSDet++, which achieve competitive results on rotated object detection in the challenging benchmarks DOTA-v1.0, DOTA-v1.5, and DOTA-v2.0. We hope the proposed method can provide a new perspective for designing algorithms to solve rotated object detection and pay more attention to tiny objects. The codes and models are available at:https://github.com/yangxue0827/RotationDetection.

Mining sufficient discriminative information is vital for effective feature representation in vehicle re-identification. Traditional methods mainly focus on the most salient features and neglect whether the explored information is sufficient. This paper tackles the above limitation by proposing a novel Salience-Navigated Vehicle Re-identification Network (SVRN) which explores diverse salient features at multi-scales. For mining sufficient salient features, we design SVRN from two aspects: 1) network architecture: we propose a novel salience-navigated vehicle re-identification network, which mines diverse features under a cascaded suppress-and-explore mode. 2) feature space: cross-space constraint enables the diversity from feature space, which restrains the cross-space features by vehicle and image identifications (IDs). Extensive experiments demonstrate our method’s effectiveness, and the overall results surpass all previous state-of-the-arts in three widely-used Vehicle ReID benchmarks (VeRi-776, VehicleID, and VERI-WILD), i.e., we achieve an 84.5% mAP on VeRi-776 benchmark that outperforms the second-best method by a large margin (3.5% mAP).

Popular rotated detection methods usually use five parameters (coordinates of the central point, width, height, and rotation angle) or eight parameters (coordinates of four vertices) to describe the rotated bounding box and l1 loss as the loss function. In this paper, we argue that the aforementioned integration can cause training instability and performance degeneration. The main reason is the discontinuity of loss which is caused by the contradiction between the definition of the rotated bounding box and the loss function. We refer to the above issues as rotation sensitivity error (RSE) and propose a modulated rotation loss to dismiss the discontinuity of loss. The modulated rotation loss can achieve consistent improvement on the five parameter methods and the eight parameter methods. Experimental results using one stage and two stages detectors demonstrate the effectiveness of our loss. The integrated network achieves competitive performances on several benchmarks including DOTA and UCAS AOD. The code is available at https://github.com/yangxue0827/RotationDetection.

Image-based Vehicle ReID methods have suffered from limited information caused by viewpoints, illumination, and occlusion as they usually use a single image as input. Graph convolutional methods (GCN) can alleviate the aforementioned problem by aggregating neighbor samples' information to enhance the feature representation. However, it's uneconomical and computational for the inference processes of GCN-based methods since they need to iterate over all samples for searching the neighbor nodes. In this paper, we propose the first Pseudo-GCN Vehicle ReID method (PGVR) which enables a CNN-based module to performs competitively to GCN-based methods and has a faster and lightweight inference process. To enable the Pseudo-GCN mechanism, a two-branch network and a graph-based knowledge distillation are proposed. The two-branch network consists of a CNN-based student branch and a GCN-based teacher branch. The GCN-based teacher branch adopts a ReID-based GCN to learn the topological optimization ability under the supervision of ReID tasks during training time. Moreover, the graph-based knowledge distillation explicitly transfers the topological optimization ability from the teacher branch to the student branch which acknowledges all nodes. We evaluate our proposed method PGVR on three mainstream Vehicle ReID benchmarks and demonstrate that PGVR achieves state-of-the-art performance.

Glioblastoma (GBM) is a common malignant brain tumor which often presents as a comorbidity with central nervous system (CNS) disorders. Both CNS disorders and GBM cells release glutamate and show an abnormality, but differ in cellular behavior. So, their etiology is not well understood, nor is it clear how CNS disorders influence GBM behavior or growth. This led us to employ a quantitative analytical framework to unravel shared differentially expressed genes (DEGs) and cell signaling pathways that could link CNS disorders and GBM using datasets acquired from the Gene Expression Omnibus database (GEO) and The Cancer Genome Atlas (TCGA) datasets where normal tissue and disease-affected tissue were examined. After identifying DEGs, we identified disease-gene association networks and signaling pathways and performed gene ontology (GO) analyses as well as hub protein identifications to predict the roles of these DEGs. We expanded our study to determine the significant genes that may play a role in GBM progression and the survival of the GBM patients by exploiting clinical and genetic factors using the Cox Proportional Hazard Model and the Kaplan-Meier estimator. In this study, 177 DEGs with 129 upregulated and 48 downregulated genes were identified. Our findings indicate new ways that CNS disorders may influence the incidence of GBM progression, growth or establishment and may also function as biomarkers for GBM prognosis and potential targets for therapies. Our comparison with gold standard databases also provides further proof to support the connection of our identified biomarkers in the pathology underlying the GBM progression.

Most existing person re-identification methods compute pairwise similarity by extracting robust visual features and learning the discriminative metric. Owing to visual ambiguities, these content-based methods that determine the pairwise relationship only based on the similarity between them, inevitably produce a suboptimal ranking list. Instead, the pairwise similarity can be estimated more accurately along the geodesic path of the underlying data manifold by exploring the rich contextual information of the sample. In this paper, we propose a lightweight post-processing person re-identification method in which the pairwise measure is determined by the relationship between the sample and the counterpart's context in an unsupervised way. We translate the point-to-point comparison into the bilateral point-to-set comparison. The sample's context is composed of its neighbor samples with two different definition ways: the first order context and the second order context, which are used to compute the pairwise similarity in sequence, resulting in a progressive post-processing model. The experiments on four large-scale person re-identification benchmark datasets indicate that (1) the proposed method can consistently achieve higher accuracies by serving as a post-processing procedure after the content-based person re-identification methods, showing its state-of-the-art results, (2) the proposed lightweight method only needs about 6 milliseconds for optimizing the ranking results of one sample, showing its high-efficiency. Code is available at: https://github.com/123ci/PBCmodel.

Vehicle detection in traffic surveillance videos is a special subtask in object detection, where desired objects are vehicles moving on the road while the background is still within a sequence. The disparity of speed within each frame, i.e. moving and static, is consistent with the vehicle and background semantic to some extent, thus motions can be extracted to enhance the appearance of foreground. In this paper, we propose a motion prior embedded parallel architecture for vehicle detection, aiming at illuminating vehicles and suppressing false positives in the background. We further implement extensive experiments on the UA-DETRAC dataset to validate the effectiveness of our approach, and achieve promising performance in both accuracy and speed.

In the research field of person re-identification, deep metric learning that guides the efficient and effective embedding learning serves as one of the most fundamental tasks. Recent efforts of the loss function based deep metric learning methods mainly focus on the top rank accuracy optimization by minimizing the distance difference between the correctly matching sample pair and wrongly matched sample pair. However, it is more straightforward to count the occurrences of correct top-rank candidates and maximize the counting results for better top rank accuracy. In this paper, we propose a generalized logistic function based metric with effective practicalness in deep learning, namely the“deeptop-rankcountermetric”, to approximately optimize the counted occurrences of the correct top-rank matches. The properties that qualify the proposed metric as a well-suited deep re-identification metric have been discussed and a progressive hard sample mining strategy is also introduced for effective training and performance boosting. The extensive experiments show that the proposed top-rank counter metric outperforms other loss function based deep metrics and achieves the state-of-the-art accuracies.

Recent machine learning methods use increasingly large deep neural networks to achieve state-of-the-art results in various tasks. Network quantization can effectively reduce computation and memory costs without modifying network structures, facilitating the deployment of deep neural networks (DNNs) on cloud and edge devices. However, most of the existing methods usually need time-consuming training or fine-tuning and access to the original training dataset that may be unavailable due to privacy or security concerns. In this paper, we present a novel method to achieve low-precision quantization with limited data. Firstly, to reduce the complexity of per-channel quantization and degeneration of per-layer quantization, we introduce group quantization that separates the output channels into groups and processes each group independently. Secondly, to better distill knowledge from the pre-trained FP32 model with limited data, we introduce a two-stage knowledge distillation method that divides the optimization process into blockwise optimization and joint optimization to address the limitation of layer-wise supervision and global supervision. Extensive experiments on ImageNet2012 (ResNet18/50, ShuffleNetV2, and MobileNetV2) demonstrate that the proposed approach can significantly improve the quantization model's accuracy when only a few training samples are available. We further show that the method also extends to other computer vision architectures and tasks such as object detection.

Network quantization can effectively reduce computation and memory costs without modifying network structures, facilitating the deployment of deep neural networks (DNNs) on edge devices. However, most of the existing methods usually need time-consuming training or fine-tuning and access to the original training dataset that may be unavailable due to privacy or security concerns. In this paper, we introduce a novel method named EvoQ that employs evolutionary search to achieve mixed precision quantization with limited data, which can optimize the resource allocation without adding computation consumption. Considering the shortage of samples and expensive search costs, we use 50 samples to measure the output difference between the quantization model and the pre-trained model for the evaluation of quantization policy, which can save the time obviously while maintaining high accuracy. To improve the search efficiency, we analyze the quantization sensitivity of each layer and utilize the results to optimize the mutation operation. At last, we calibrate the outputs and intermediate features of the quantization model using the selected 50 samples to improve the performance further. We implement extensive experiments on a diverse set of models, including ResNet18/50/101, SqueezeNet, ShuffleNetV2, and MobileNetV2 on ImageNet, as well as SSD-VGG and SSD-ResNet50 on PASCAL VOC. Our method can improve the performance apparently and outperforms the existing post-training quantization methods, demonstrating the effectiveness of EvoQ.

Generative Adversarial Networks (GANs) have been employed for face super resolution but they bring distorted facial details easily and still have weakness on recovering realistic texture. To further improve the performance of GAN-based models on super-resolving face images, we propose PCA-SRGAN which pays attention to the cumulative discrimination in the orthogonal projection space spanned by PCA projection matrix of face data. By feeding the principal component projections ranging from structure to details into the discriminator, the discrimination difficulty will be greatly alleviated and the generator can be enhanced to reconstruct clearer contour and finer texture, helpful to achieve the high perception and low distortion eventually. This incremental orthogonal projection discrimination has ensured a precise optimization procedure from coarse to fine and avoids the dependence on the perceptual regularization. We conduct experiments on CelebA and FFHQ face datasets. The qualitative visual effect and quantitative evaluation have demonstrated the overwhelming performance of our model over related works.

Dense 3D face correspondence is a fundamental and challenging issue in the literature of 3D face analysis. Correspondence between two 3D faces can be viewed as a non-rigid registration problem that one deforms into the other, which is commonly guided by a few facial landmarks in many existing works. However, the current works seldom consider the problem of incoherent deformation caused by landmarks. In this paper, we explicitly formulate the deformation as locally rigid motions guided by some seed points, and the formulated deformation satisfies coherent local motions everywhere on a face. The seed points are initialized by a few landmarks, and are then augmented to boost shape matching between the template and the target face step by step, to finally achieve dense correspondence. In each step, we employ a hierarchical scheme for local shape registration, together with a Gaussian reweighting strategy for accurate matching of local features around the seed points. In our experiments, we evaluate the proposed method extensively on several datasets, including two publicly available ones: FRGC v2.0 and BU-3DFE. The experimental results demonstrate that our method can achieve accurate feature correspondence, coherent local shape motion, and compact data representation. These merits actually settle some important issues for practical applications, such as expressions, noise, and partial data.

Translating near-infrared (NIR) face into color (RGB) face, is helpful to improve the visual effect of images and the performance of face recognition. The model for unpaired image-to-image translation is suitable for this task due to the high cost of pixel-matched data. Because of the complexity difference between NIR and RGB image domains, the complexity inequality in bidirectional NIR-RGB translations is significant. We analyze the limitation of the original CycleGAN in asymmetric translation tasks, and propose an Asymmetric Cycle-GAN model with U-net-like generators of unequal sizes to adapt to the asymmetric need in NIR-RGB translations. The edge-retain loss between NIR and the generated RGB images is also introduced to enhance face visual quality. The qualitative visual evaluation and quantitative evaluation with face ID and skin color criteria show that our model achieves great improvements compared with state-of-the-art methods on three public datasets and a newly proposed dataset.

Person re-identification addresses the problem of matching individual images of the same person captured by different non-overlapping camera views. Distance metric learning plays an effective role in addressing the problem. With the features extracted on several regions of person image, most of distance metric learning methods have been developed in which the learnt cross-view transformations are region-generic, i.e all region-features share a homogeneous transformation. The spatial structure of person image is ignored and the distribution difference among different region-features is neglected. Therefore in this paper, we propose a novel region-specific metric learning method in which a series of region-specific sub-models are optimized for learning cross-view region-specific transformations. Additionally, we also present a novel feature pre-processing scheme that is designed to improve the features' discriminative power by removing weakly discriminative features. Experimental results on the publicly available VIPeR, PRID450S and QMUL GRID datasets demonstrate that the proposed method performs favorably against the state-of-the-art methods.

In this paper, we present a novel face spoofing detection method based on 3D lighting environment analysis of an image pair collected before and after the lighting environment change. Our idea is inspired from the unimpressive fact that the illumination distributions of the internal spoof face stays stable under the protection of the photo and screen plane, while that of a exposed genuine face changes accordingly to different lighting environment due to a natural response of 3D structure. After estimating two sets of lighting environment coefficients of client's face image pair with the hand of 3D Morphable Model (3DMM) and Sphere Harmonic Illumination Model (SHIM), robust liveness judgement is conducted by hypothesis tests. Experimental results show the effectiveness of proposed method on multiple kinds of face attacks including printed photo, screen photo, and video replay attack, and other advantages such as user cooperation free, loose using conditions, simple equipment demand, easy to camouflage and propitious to face recognition.

The operator-based signal separation approach uses an adaptive operator to separate a signal into a set of additive subcomponents. In this paper, we show that differential operators and their initial and boundary values can be exploited to derive corresponding integral operators. Although the differential operators and the integral operators have the same null space, the latter are more robust to noisy signals. Moreover, after expanding the kernels of Frequency Modulated (FM) signals via eigen-decomposition, the operator-based approach with the integral operator can be regarded as the matched filter approach that uses eigen-functions as the matched filters. We then incorporate the integral operator into the Null Space Pursuit (NSP) algorithm to estimate the kernel and extract the subcomponent of a signal. To demonstrate the robustness and efficacy of the proposed algorithm, we compare it with several state-of-the-art approaches in separating multiple-component synthesized signals and real-life signals.

3D morph able model (3DMM) is a powerful tool to recover 3D shape and texture from a single facial image. Its foundation consists of three models (i.e. face, camera, and illumination) which can simulate the formulation process of facial images. In this paper, we adopt a new illumination model, the Sphere Harmonic Illumination Model (SHIM), to the 3DMM fitting process. The new illumination model takes more lighting factors into consideration than the Phong's model. Then, we use a new optimization algorithm to optimize the shape and texture parameters simultaneously under SHIM. Compared with the the existing methods that used SHIM to recover only texture, both the shape and texture recovered by our algorithm are improved. The experiments on he CMU-PIE database also show that, compared to other state-of-the-art methods based on the Phong's model, the proposed approach enhances the robustness of the fitting of 3DMM against lighting variations.

In this paper, we propose a more practical and accurate method for calibrating the roadside camera used in traffic surveillance systems. Considering the characteristics of the traffic scenes, we propose a minimum calibration condition that consists of two vanishing points and a vanishing line, which can be easily satisfied in most traffic scenes. Based on the minimum calibration condition, we provide a calibration method to estimate camera intrinsic parameters and rotation angles, which employs least squares optimization instead of closed-form computation. Compared with the existing calibration methods, our method is suitable for more traffic scenes and is able to accurately determine more camera parameters including the principal point. By making full use of video information, multiple observations of the vanishing points are available from different objects. For more accurate calibration, we present a dynamic calibration method using these observations to correct camera parameters. As for the estimation of the camera translation vector, known lengths in the road or known heights above the road are exploited. The experimental results on synthetic data and real traffic images demonstrate the accuracy, robustness, and practicability of the proposed calibration method.

The operator-based signal separation approach uses an adaptive operator to separate a signal into additive subcomponents. And different types of operator can depict different properties of a signal. In this paper, we define a new kind of integral operator which can be derived from the second kind of Fredholm integral equation. Then, we analyze the properties of the proposed integral operator and discuss its relation to the second condition of Intrinsic Mode Function (IMF). To demonstrate the robustness and efficacy of the proposed operator, we incorporate it into the Null Space Pursuit algorithm to separate several multicomponent signals, including a real-life signal.

An operator-based and sparsity-based approach is proposed to adaptively separate a signal into additive subcomponents. The proposed approach can be formulated as an optimization problem. Since the design of the operator can be adaptively customized to the target signal, we can propose different types of operators for different types of signals. The subcomponents are a kind of local narrow band signals in the null space of an adaptive operator and a residual signal which is a sparse signal in some sense. Our experiments, including simulated signals and a real-life signal, demonstrate the efficacy and accuracy of the proposed approach.

How to deal with themotion blurred image is a common problem in our daily life. Restoring blurred images is challenging, especially when both the blur kernel and the sharp image are unknown. In this work, we present a new algorithm for removing motion blur from a single image, which incorporates the image decomposition into the image deblurring process. Most of the existing algorithms solving the blind deblurring problem use the alternate iterative mechanism, which alternative estimates the kernel and restores the sharp image. We find that the small gradients of image are not always helpful but sometimes harmful to this kind of iterative algorithm. So we decompose the blurred image into cartoon and texture components. And we only use the cartoon part of the image, which can improve the stability and robustness of the algorithm. Our experiments show that our algorithms can achieve good results in man-made and real-life photos.

State-of-the-art pansharpening methods generally inject the spatial details extracted from the panchromatic (Pan) image into the multispectral (MS) images by considering different injection models. The fusion performances severely rely on the accuracy of the modeling and the estimation of model parameters. In this letter, we propose an optimized approach to avoid explicitly modeling the detail injection process. The solution employs the gradient field of the Pan image for spatial enhancement. The low-pass (LP) version of the fused bands are constrained to be the most similar to the original MS bands to preserve the spectral characteristics. We use the local correlation coefficients between the MS band and the LP version of the Pan image to adjust the two sources of information based on a simple observation, and it is further optimized by considering the overall quality index Q4. Experimental results demonstrate that the proposed method outperforms the state-of-the-art multiresolution analysis-based methods.

This paper proposes a new approach to the image blind super-resolution (BSR) problem in the case of affine interframe motion. Although the tasks of image registration, blur identification, and high-resolution (HR) image reconstruction are coupled in the imaging process, when dealing with nonisometric interframe motion or without the exact knowledge of the blurring process, classic SR techniques generally have to tackle them (maybe in some combinations) separately. The main difficulty is that state-of-the-art deconvolution methods cannot be straightforwardly generalized to cope with the space-variant motion. We prove that the operators of affine warping and blur commute with some additional transforms and derive an equivalent form of the BSR observation model. Using this equivalent form, we develop an iterative algorithm to jointly estimate the triple-coupled variables, i.e., the motion parameters, blur kernels, and HR image. Experiments on synthetic and real-life images illustrate the performance of the proposed technique in modeling the space-variant degradation process and restoring local textures.

In this paper, we propose a multiple component predictive coding framework. We firstly separate the reconstructed image into several subcomponents; and then predict each subcomponent independently but encode them together. To separate image into multiple subcomponents, we also propose a fast operator-based image separation algorithm. With the help of multicomponent prediction strategy, our prediction results can achieve superior performance than the H.264/AVC intra frame prediction method for images containing rich textures. By adopting the residue coding method used in H.264/AVC, we compare the compression efficacy of our proposed algorithm with the state-of-art JPEG2000 and H.264/AVC intra frame compression algorithms in the experimental part. The numerical results show that our algorithm is better than both H.264/AVC intra frame coding algorithm and JPEG2000 algorithm for images with ample textures.

We present a new algorithm for removing the haze effects from a single color image. By introducing an additive noise argument in the degradation image model, we establish a unified probabilistic framework for the clear day image and the atmosphere transmission. Then we use an alternative optimization method to approximate the MAP estimators of these two variables iteratively. Experimental results demonstrate the efficiency of the proposed method on restoring the true scene colors and contrast.

The size of training set as well as the usage thereof is an important issue of learning-based super-resolution. In this paper, we presented an adaptive learning method for face hallucination using locality preserving projections (LPP). By virtue of the ability to reveal the non-linear structure hidden in the high-dimensional image space, LPP is an efficient manifold learning method to analyze the local intrinsic features on the manifold of local facial areas. By searching out patches online in the LPP sub-space, which makes the resultant training set tailored to the testing patch, our algorithm performed the adaptive sample selection and then effectively restored the lost high-frequency components of the low-resolution face image by patch-based eigen transformation using the dynamic training set. Finally, experiments fully demonstrated that the proposed method can achieve good performance of super-resolution reconstruction by utilizing a relative small sample.

The interscale Stein's unbiased risk estimator (SURE)-based approach introduced by Luisier is a recent state of the art in orthonormal wavelet denoising, but it is not very effective for those images that have substantial high-frequency contents. To solve this problem, we introduce an effective integration of the intrascale correlations within the interscale SURE-based approach. We show that the consideration of both the intrascale and interscale dependencies of wavelet coefficients brings more denoising gains than those obtained with the interscale SURE-based approach, especially for denoising of images that have substantial textures such as the Barbara image.

Empirical-mode decomposition (EMD) provides a powerful tool for adaptive multiscale analysis of nonstationary signals. Aiming at the intrinsic mode function (IMF) criteria in sifting process and the scale mixing problem in EMD, this paper proposes a bandwidth criterion for IMF. By analyzing the simulated signal, it is confirmed that the IMFs obtained with the bandwidth criterion approximate the real components better and reflect the intrinsic information of the analyzed signal. Furthermore, the criterion based on bandwidth can weaken the scale mixing problem.

Research on texture synthesis has made substantial progress in recent years, and many patch-based sampling algorithms now produce quality results in an acceptable computation time. However, when such algorithms are applied, whether they provide good results for specific textures, and why they do so, are questions that have yet to be answered. In this article, we deal specifically with the second question by modeling the synthesis problem as one of learning from incomplete data, and propose an algorithm that is a generalization of patch-work approach. Through this algorithm, we demonstrate that the solution of patch-based sampling approaches is an approximation of finding the maximum-likelihood optimum by the generalized expectation and maximization (GEM) algorithm.

A novel boundary processing technique is proposed for bidimensional empirical mode decomposition (BEMD). BEMD is a new form of multiscale decomposition and has superior quality in extracting intrinsic components of textures. Although the boundary effect of BEMD is one of its main questions, there have been no concrete discussions or effective methods for it. A modified algorithm of nonparametric sampling for texture synthesis is used to extend images before applying BEMD. Experiments indicate that the technique proposed in This work can form better decomposition compared with past algorithms not only for textures but also for natural images.

Several images to be fused can be taken as a multivalued image. From multiscale fundamental form (MFF), a multivalued image wavelet representation can be obtained. In order to avoid the enlargement of the wavelet coefficients, the weighted multiscale fundamental form (WMFF) is exploited in our fusion process. The mutual information and the conditioned entropy are introduced to evaluate the fused result. Compared with the multiscale fundamental form, the weighted one has a better performance on image fusion.

Image inpainting refers to the ill-posed problem of filling in the missing data in digital images by interpolating from the vicinity. It is shown in this paper how inpainting can be performed by means of random simulation of boundary integral, which we call the Monte Carlo method. Our method is computationally less taxing than the classical diffusion methods and yields a solution that may have strong edges.

In this paper the technique of directional empirical mode decomposition (DEMD) and its application to texture segmentation are presented. Empirical mode decomposition (EMD) decomposes signals by sifting and then analyzes the instantaneous frequency of the obtained components called intrinsic mode functions (IMF). As a new form of extending 1D EMD to the 2D case, DEMD considers the directional frequency and envelope at each point. One type of 2D Hilbert transform is introduced to compute the analytical functions for the frequency and envelope. The technique of selecting directions for DEMD based on texture's Wold theory is also presented. Experimental results indicate the effectiveness of the method for texture segmentation.

Recently, there has been a growing interest in investigating signal-adaptive multirate filter banks. In this paper, we propose a direct method of constructing signal-adaptive orthogonal wavelet filter banks with better energy compaction performance than that of Daubechies wavelet filters of same length. The speed of this method is fast, so it can be constructed in real time.

In this paper we propose an image super-resolution algorithm using wavelet-domain hidden Markov tree (HMT) model. Wavelet-domain HMT models the dependencies of multiscale wavelet coefficients through the state probabilities of wavelet coefficients, whose distribution densities can be approximated by the Gaussian mixture. Because wavelet-domain HMT accurately characterizes the statistics of real-world images, we reasonably specify it as the prior distribution and then formulate the image super-resolution problem as a constrained optimization problem. And the cycle-spinning technique is used to suppress the artifacts that may exist in the reconstructed high-resolution images. Quantitative error analyses are provided and several experimental images are shown for subjective assessment.