In this paper, we address the issue of visual data organization by recovering an intrinsic order from an unorganized dataset. The proposed method exploits the inherent nature of manifold. This new perspective, posing no hypothesis on local topology of observed data, is simply built on the smoothness prior of manifold geometry. Under the observation that strong relation exists among visual content, we assume a visual dataset lies on a manifold and thus changes smoothly from point to point. By exploiting the linearity within nearby data points, our goal becomes to visit all of the data points along a manifold-guided order and to characterize the specific manifold’s shape.

The challenge of single-image dehazing mainly comes from double uncertainty of scene radiance and scene transmission. Most existing methods focus on restoring the visibility of hazy images and tend to derive a rough estimate of scene transmission. Unlike previous work, in this paper we advocate the significance of accurate transmission estimation and recast our problem as deriving the optimal transmission map directly from the haze model under two scene priors. We introduce theoretic and heuristic bounds of scene transmission to guide the optimum and show that the proposed theoretic bound happens to justify the well-known dark channel prior of haze-free images. With the constraints on the solution space, we then incorporate two scene priors, including locally consistent scene radiance and context-aware scene transmission, to formulate a constrained minimization problem and solve it by quadratic programming. The global optimality is guaranteed. Simulations on synthetic data set quantitatively verify the accuracy and show that the transmission map successfully captures fine-grained depth boundaries. Experimental results on color/gray-level images demonstrate that our method outperforms most state of the arts in terms of both accurate transmission maps and realistic haze-free images.

Modern techniques rely on convex relaxation to derive tractable approximations for rank-sparsity decomposition. However, the resultant precision loss usually deteriorates the performance in real-world applications. In this paper, we focus on the topic of visual saliency detection and consider the inherent uncertainty existing in observations, which may originate from both low-rank and sparse components. We formulate the rank-sparsity model with an implicit weighting factor and show that this weighting factor characterizes the nature of visual saliency. The proposed model is generalized to solve saliency and co-saliency detection in a unified way. In addition, this model can easily incorporate center-prior or other top-down priors and can extend to multi-task learning to explore the interrelation between multiple features. Experimental results demonstrate that our method improves existing rank-sparsity decomposition, and also outperforms most state of the arts on two salient object databases.

The success of research on matrix completion is evident in a variety of real-world applications. Tensor completion, which is a high-order extension of matrix completion, has also generated a great deal of research interest in recent years. Given a tensor with incomplete entries, existing methods use either factorization or completion schemes to recover the missing parts. However, as the number of missing entries increases, factorization schemes may overfit the model because of incorrectly predefined ranks, while completion schemes may fail to interpret the model factors. In this paper, we introduce a novel concept: complete the missing entries and simultaneously capture the underlying model structure. To this end, we propose a method called simultaneous tensor decomposition and completion (STDC) that combines a rank minimization technique with Tucker model decomposition. Moreover, as the model structure is implicitly included in the Tucker model, we use factor priors, which are usually known a priori in real-world tensor objects, to characterize the underlying joint-manifold drawn from the model factors. By exploiting this auxiliary information, our method leverages two classic schemes and accurately estimates the model factors and missing entries. We conducted experiments to empirically verify the convergence of our algorithm on synthetic data and evaluate its effectiveness on various kinds of real-world data. The results demonstrate the efficacy of the proposed method and its potential usage in tensor-based applications. It also outperforms state-of-the-art methods on multilinear model analysis and visual data completion tasks.

Given a set of images, finding a compact and discriminative representation is still a big challenge especially when multiple latent factors are hidden in the way of data generation. To represent multifactor images, although multilinear models are widely used to parameterize the data, most methods are based on high-order singular value decomposition (HOSVD), which preserves global statistics but interprets local variations inadequately. To this end, we propose a novel method, called multilinear graph embedding (MGE), as well as its kernelization MKGE to leverage the manifold learning techniques into multilinear models. Our method theoretically links the linear, nonlinear, and multilinear dimensionality reduction. We also show that the supervised MGE encodes informative image priors for image regularization, provided that an image is represented as a high-order tensor. From our experiments on face and gait recognition, the superior performance demonstrates that MGE better represents multifactor images than classic methods, including HOSVD and its variants. In addition, the significant improvement in image (or tensor) completion validates the potential of MGE for image regularization.

In this paper, we propose a novel low-rank appearance model for removing rain streaks. Different from previous work, our method needs neither rain pixel detection nor time-consuming dictionary learning stage. Instead, as rain streaks usually reveal similar and repeated patterns on imaging scene, we propose and generalize a low-rank model from matrix to tensor structure in order to capture the spatio-temporally correlated rain streaks. With the appearance model, we thus remove rain streaks from image/video (and also other high-order image structure) in a unified way. Our experimental results demonstrate competitive (or even better) visual quality and efficient run-time in comparison with state of the art.

Existing forensic fingerprints mostly rely on robust statistical estimates, which usually hinder accurate image tampering detection at fine-grained level. To date, people still put a big question mark behind “what has been tampered?” In this paper, we try to answer this question from a counterfeiter's perspective, devil in the details, that image tampering is usually sparsely and delicately manipulated. Thanks to recently well-established rank-sparsity incoherence, we formulate the fine-grained tampering detection as a constrained minimization problem in order to discriminate the authentic areas (sharing similar feature behaviours) from the tampered areas (inconsistently and sparsely distributed) in a forensic feature space. Our formulation could incorporate with any applicable forensic features and, unlike existing methods, needs neither statistical analysis nor model factor estimation. Our experimental results show that the proposed method successfully locates various kinds of image tampering, including copy-move forgery, resampling and recompression, at fine-grained level.

Age is one of the important biometric traits for reinforcing the identity authentication. The challenge of facial age estimation mainly comes from two difficulties: (1) the wide diversity of visual appearance existing even within the same age group and (2) the limited number of labeled face images in real cases. Motivated by previous research on human cognition, human beings can confidently rank the relative ages of facial images, we postulate that the age rank plays a more important role in the age estimation than visual appearance attributes. In this paper, we assume that the age ranks can be characterized by a set of ranking features lying on a low-dimensional space. We propose a simple and flexible subspace learning method by solving a sequence of constrained optimization problems. With our formulation, both the aging manifold, which relies on exact age labels, and the implicit age ranks are jointly embedded in the proposed subspace. In addition to supervised age estimation, our method also extends to semi-supervised age estimation via automatically approximating the age ranks of unlabeled data. Therefore, we can successfully include more available data to improve the feature discriminability. In the experiments, we adopt the support vector regression on the proposed ranking features to learn our age estimators. The results on the age estimation demonstrate that our method outperforms classic subspace learning approaches, and the semi-supervised learning successfully incorporates the age ranks from unlabeled data under different scales and sources of data set.

Describing time-variant appearance of object surface is still an open problem. With intricate environmental factors and different material characteristics over time, no researcher did tackle the principal problem: how to formulate time-variant change on general surface appearance? In this paper, we attempt to solve this challenging issue. Using multilinear algebra representation, we propose a novel appearance model and characterize the surface-specific and time-variant properties. When given an unknown sample, we propose a robust method to estimate its aging degree. In addition, we also propose an approach to synthesize its realistic appearance changes even when the material of this given sample does not exist in our database. Experimental results demonstrate the feasibility and effectiveness of our proposed approach. To the best of our knowledge, this challenging issue is first explored in image processing applications.

This paper proposes an automatic color transfer method for processing images with complex content based on intrinsic component. Although several automatic color transfer methods has been proposed by including region information and/or using multiple references, these methods tend to become ineffective when processing images with complex content and lighting variation. In this paper, our goal is to incorporate the idea of intrinsic component to better characterize the local organization within an image and to reduce the color-bleeding artifact across complex regions. Using intrinsic information, we first represent each image in region level and determine the best-matched reference region for each target region. Next, we conduct color transfer between the best-matched region pairs and perform weighted color transfer for pixels across complex regions in a de-correlated color space. Both subjective and objective evaluation of our experiments demonstrates that the proposed method outperforms the existing methods.

Since JPEG has been a popularly used image compression standard, forgery detection in JPEG images now plays an important role. Forgeries on compressed images often involve recompression and tend to erase those forgery traces existed in uncompressed images. We could, however, try to discover new traces caused by recompression and use these traces to detect the recompression forgeries. Quantization is the critical step in lossy compression which maps the DCT coefficients in an irreversible way under the quantization constraint set (QCS) theorem. In this paper, we first derive that a doubly compressed image no longer follows the QCS theorem and then propose a novel quantization noise model to characterize single and doubly compressed images. In order to detect double compression forgery, we further propose to approximate the uncompressed ground truth image using image restoration techniques. We conduct a series of experiments to demonstrate the validity of the proposed quantization noise model and also the effectiveness of the forgery detection method with the proposed image restoration techniques.

Since JPEG image format has been a popularly used image compression standard, tampering detection in JPEG images now plays an important role. The artifacts introduced by lossy JPEG compression can be seen as an inherent signature for compressed images. In this paper, we propose a new approach to analyse the blocking periodicity by, 1) developing a linearly dependency model of pixel differences, 2) constructing a probability map of each pixel’s belonging to this model, and 3) finally extracting a peak window from the Fourier spectrum of the probability map. We will show that, for single and double compressed images, their peaks’ energy distribution behave very differently. We exploit this property and derive statistic features from peak windows to classify whether an image has been tampered by cropping and recompression. Experimental results demonstrate the validity of the proposed approach.