Intrinsic image decomposition is a low-level image processing task that extracts the reflectance and lighting components from an image. This process can improve the illumination robustness of perception tasks, such as object detection, recognition, and image understanding. Recently, deep image generation frameworks have been used to generate intrinsic images. However, the encoder and decoder lack prior knowledge constraints. This paper presents a quantized codebook for embedding intrinsic features that guide the extraction of intrinsic images. To enhance reconstruction accuracy, we propose a purification method to eliminate irrelevant elements from the codebook. Additionally, we propose self-attention and cross-attention modules to integrate the intrinsic features of the codebook into the input image features for reconstruction. The effectiveness of the algorithm is demonstrated through experiments conducted on several popular datasets.

Adversarial reprogramming allows repurposing a machine-learning model to perform a different task. For example, a model trained to recognize animals can be reprogrammed to recognize digits by embedding an adversarial program in the digit images provided as input. Recent work has shown that adversarial reprogramming may not only be used to abuse machine-learning models provided as a service, but also beneficially, to improve transfer learning when training data is scarce. However, the factors affecting its success are still largely unexplained. In this work, we develop a first-order linear model of adversarial reprogramming to show that its success inherently depends on the size of the average input gradient, which grows when input gradients are more aligned, and when inputs have higher dimensionality. The results of our experimental analysis, involving fourteen distinct reprogramming tasks, show that the above factors are correlated with the success and the failure of adversarial reprogramming.

Adversarial reprogramming allows stealing computational resources by repurposing machine learning models to perform a different task chosen by the attacker. For example, a model trained to recognize images of animals can be reprogrammed to recognize medical images by embedding an adversarial program in the images provided as inputs. This attack can be perpetrated even if the target model is a black box, supposed that the machine-learning model is provided as a service and the attacker can query the model and collect its outputs. So far, no defense has been demonstrated effective in this scenario. We show for the first time that this attack is detectable using stateful defenses, which store the queries made to the classifier and detect the abnormal cases in which they are similar. Once a malicious query is detected, the account of the user who made it can be blocked. Thus, the attacker must create many accounts to perpetrate the attack. To decrease this number, the attacker could create the adversarial program against a surrogate classifier and then fine-tune it by making a few queries to the target model. In this scenario, the effectiveness of the stateful defense is reduced, but we show that it is still effective.

Single image-based intrinsic image decomposition attempts to separate one input image into several intrinsic components, which is inherently an under-constrained problem. Some recent works have been proposed to estimate the intrinsic components using encoder-decoder structures. However, they generally lack exploration of the different component-oriented feature constraints and feature selection processes. In this paper, a non-local color compensation network (NCCNet) is proposed. Firstly, the hue and value channels of HSV color space are used as the complementary information for RGB images for the estimation of albedo and shading, respectively. The color space representation serves as an external constraint, which does not require expensive sensors or complicated computations. Secondly, an integrated non-local attention scheme is proposed to describe the relations of non-adjacent regions with a lower computational complexity compared to traditional methods. Then the non-local and local attention are combined to describe correlations among features and used as feature selectors between the encoder and decoder. Thirdly, the mutual constraint between albedo and shading is also explored in the network to further optimize the process. In order to train the network, a unified mutual exclusion loss function is proposed. Extensive experiments are conducted on several popular datasets, and the proposed NCCNet achieves improved performance with comparable computational cost compared to competing methods.

When new unknown samples are captured continually in the open-world environment, the concept diversity accumulation of existing classes and the identification/creation of new concept classes should be considered simultaneously. Since the initial training set of existent classes may be under-prepared, adhering to immediate decisions will inevitably lead to reduced open set recognition performance and higher costs of labeling/updating. Inspired by quantitative indicators in predictive reliability assessment and semi-supervised/active learning, the inclusive-consistency-based quantitative decision-making framework (ICQdm) is proposed for incremental automatic target recognition (ATR) to evaluate the identifiability and typicality of new unknown samples, which could give the decision-making guide of recognition and updating. For recognition decision-making, the first consistency indicator calculates the reliability of the unknown sample being included by one specific training class. The test samples with low reliability should be the wrongly classified samples and new unknown classes’ samples, which are difficult to be labeled by recognition models themselves. For updating decision-making, the second consistency indicator is designed to be the sample distribution density under the inclusive constraint, which could highlight the dense sample distributions of new unknown samples outside the known training distribution. Experiments verify that the proposed ICQdm outperforms other comparison methods on the open set recognition reliability evaluation and labeling/updating efficiency.

Intrinsic image analysis plays an important role for image understanding, since it can provide accurate reflectance, shape and illumination information of the scene. However, intrinsic image analysis is an ill-posed problem which need to apply extra constrains for the decomposition of reflectance image and shading image from a single image. Recently deep neural networks are introduced for intrinsic image analysis, which can produce two intrinsic components simultaneously. In fact, the mutually exclusive relationship between reflectance image and shading image is not only a constraint for decomposition but also can improve the decomposition results. However, this relationship is always omitted in the current networks. In order to address this problem, we propose a novel deep network called as Cascaded Quotient Network (CasQNet) for intrinsic image decomposition. The CasQNet consists of two sub-networks: a Pyramid Mini-U-Net (PyNet) that specifically extracts the reflectance image in multi-scale and a Shading Optimization Network (SoNet) that optimizes the resulting shading. These two sub-networks are cascaded by a quotient operation, which directly enforces the mutually exclusive relationship between reflectance image and shading image in the network architecture. In PyNet, the task of reconstructing reflectance image is achieved by a series of nested multi-scale U-Nets, which simplified the learning task for each U-Net. SoNet is designed to address the unsmooth and blur problems of extreme points caused by the quotient operation. PyNet and SoNet are trained alternately and finally jointed in cascaded structure. Furthermore, we combine multiple loss functions, which consist of data loss, correlation loss and reconstruction loss, for improving the learning effectiveness. To evaluate our proposed algorithm, extensive experiments are performed on three datasets, i.e., ShapeNet, BOLD Surface and MIT Intrinsic Image datasets. Qualitative and quantitative results show that our model achieves the best performance compared to the state-of-the-art methods.

Material is one of the intrinsic features of objects, and consequently material recognition plays an important role in image understanding. The same material may have various shapes and appearance, while keeping the same physical characteristic. This brings great challenges for material recognition. Besides suitable features, a powerful classifier also can improve the overall recognition performance. Due to the limitations of classical linear neurons, used in all shallow and deep neural networks, such as CNN, we propose to apply the generalized operational neurons to construct a classifier adaptively. These generalized operational perceptrons (GOP) contain a set of linear and nonlinear neurons, and possess a structure that can be built progressively. This makes GOP classifier more compact and can easily discriminate complex classes. The experiments demonstrate that GOP networks trained on a small portion of the data (4%) can achieve comparable performances to state-of-the-arts models trained on much larger portions of the dataset.

Point set registration presents unique significance in Lidar-based intelligent vehicle localization and mapping. It involves registering point sets of the same scene observed from different positions by determining their relative spatial transformation. However, due to the noise and outliers in the point sets and initial misalignment, existing methods suffer from the issues of low accuracy or large computational cost. In this paper, we propose a novel Bayesian state space model to describe the sequential point registration problem. Specifically, we specify the transformations to be the latent states and further assume that they vary smoothly across time. The point clouds are then represented as Gaussian mixture models that change accordingly with the transformation. We then develop a stochastic variational Bayesian inference algorithm to learning the distributions of the transformation, which automatically strike a balance between mapping every two consecutive point clouds and the temporal smoothness of the transformation. Experimental results based simulated data show that the proposed variational Bayesian point set registration (VB-PSR) algorithm achieves higher accuracy with comparable or less time and resources, in comparison with the state- of-the-art methods.

The simultaneous localization and mapping (SLAM) problem involves using the measurements of sensors to construct an environmental map, while simultaneously recovering the vehicle trajectory within this map. There are broadly two strategies for SLAM: on-line and off-line. In this paper, we focus on the off-line SLAM (a.k.a. full SLAM) problem and propose a variational Bayes inference algorithm to address it. Specifically, the intractable posterior distribution of the vehicle poses given the measurements is approximated by a tractable variational distribution, resulting in estimates of the vehicle poses as well as their uncertainties. In contrast with the existing off- line methods, the inverse variances of the additive noise are updated along with the posterior distribution instead of being fixed, thus, the proposed method is robust to unknown noises. Furthermore, the computational complexity of the proposed method is only linear in the number of frames and the computational bottleneck of the algorithm can be easily parallelized to achieve further acceleration. Numerical results show that the proposed method is insensitive to the selection of the noise parameters. More importantly, it is superior in efficiency to the state-of-the-art method, especially for large- scale SLAM problems.

Face presentation attacks are the main threats to face recognition systems, and many presentation attack detection (PAD) methods have been proposed in recent years. Although these methods have achieved significant performance in some specific intrusion modes, difficulties still exist in addressing replayed video attacks. That is because the replayed fake faces contain a variety of aliveness signals, such as eye blinking and facial expression changes. Replayed video attacks occur when attackers try to invade biometric systems by presenting face videos in front of the cameras, and these videos are often launched by a liquid-crystal display (LCD) screen. Due to the smearing effects and movements of LCD, videos captured from the real and replayed fake faces present different motion blurs, which are reflected mainly in blur intensity variation and blur width. Based on these descriptions, a motion blur analysis-based method is proposed to deal with the replayed video attack problem. We first present a 1D convolutional neural network (CNN) for motion blur intensity variation description in the time domain, which consists of a serial of 1D convolutional and pooling filters. Then, a local similar pattern (LSP) feature is introduced to extract blur width. Finally, features extracted from 1D CNN and LSP are fused to detect the replayed video attacks. Extensive experiments on two standard face PAD databases, i.e., relay-attack and OULU-NPU, indicate that our proposed method based on the motion blur analysis significantly outperforms the state-of-the-art methods and shows excellent generalization capability.

Convolutional neural networks (CNNs) have achieved excellent performance in the field of pattern recognition when huge amount of training data is available. However, training a CNN model is less obvious when only a limited amount of data is given such as in the case of face anti-spoofing problem. It is indeed not easy to collect very large sets of fake faces. Especially for the fully-connected layers, tens of thousands of parameters need to be learned. To tackle this problem of lack of training data in face anti-spoofing, we propose to explore the incorporation of hand-crafted features in the CNN framework. In our proposed approach, the color local binary patterns (LBP) features are extracted from the convolutional feature maps, which are fine tuned based on the VGG-face model. These features are then fed into support vector machine (SVM) classifier. Extensive experiments are conducted on two benchmark and publicly available databases showing very interesting performance compared to state-of-the-art methods.

In this paper we show how to improve the detection of shadows in natural scenes using a novel combination of colour and illumination features. Detecting shadows is useful because they provide information about both light sources and the shapes of objects thereby illuminated. Recent shadow detection methods use supervised machine learning techniques with input from colour and texture features extracted directly from the original images (e.g. Lalonde et al. ECCV 2010, Zhu et al. CVPR 2010). It seems sensible to augment these with estimates of scene illumination, as can be obtained with an intrinsic image extraction algorithm. Intrinsic image extraction separates the illumination and reflectance components in a scene, and the resulting illumination maps contain robust intensity change features at shadow boundaries. In this paper, we make two main contributions. First we improve upon existing methods for extracting illumination maps. Second we show how to use these illumination maps together with colour segmentation to extend the Lalonde’s approach to shadow detection. Illumination maps are extracted using a steerable filter framework based on global and local correlations in low and high frequency bands respectively. The illumination and colour features so extracted are then input to a decision tree trained to detect shadow edges using AdaBoost. We tested variations of our proposed approach on two public databases of natural scenes. This study showed that our approach improves on that of Lalonde both in terms of sensitivity to shadow edges and rejection of false positives. Following Lalonde we show that our detection results are further improved by imposing an edge continuity constraint via a conditional random field (CRF) model. 1