A simple method is presented for road scene anomaly detection, which is particularly important in autonomous driving. Without the prior knowledge about the appearance and size, anomaly detection is formulated as an OoD (Out-of-Distribution) detection task. Epistemic uncertainty indicates that when a model is trained with finite training data, it will fail to yield reliable prediction if the testing data is far from the training data distribution (OoD). Deviating from previous works, instead of modelling the uncertainty of a single model explicitly, this work casts the individual uncertainty into an ensemble uncertainty metric by measuring the discrepancy between the different models. The benefits are: 1) Following a meta-learning framework, outputs from the off-the-shelf models can be used directly, which results in a very small training cost; 2) With the ensemble approach, the model is especially robust to dataset shift. Experimental results show that the performance of the proposed method is competitive: 1) Its APs (Average precisions) are 3.5%, 3.1%, and 5.2% higher than the best SOTAs (State-of-the-arts) on LaF (Lost-and-Found), RA (Road Anomaly), and RO (Road Obstacles) respectively; 2) For all datasets with different road conditions, it obtains consistent performance. (Code is available at https://github.com/yazhouliu/EUG).

Fair face recognition is all about learning invariant feature that generalizes to unseen faces in any demographic group. Unfortunately, face datasets inevitably capture the imbalanced demographic attributes that are ubiquitous in real-world observations, and the model learns biased feature that generalizes poorly in the minority group. We point out that the bias arises due to the confounding demographic attributes, which mislead the model to capture the spurious demographic-specific feature. The confounding effect can only be removed by causal intervention, which requires the confounder annotations. However, such annotations can be prohibitively expensive due to the diversity of the demographic attributes. To tackle this, we propose to generate diverse data partitions iteratively in an unsupervised fashion. Each data partition acts as a self-annotated confounder, enabling our Invariant Feature Regularization (Inv-Reg) to deconfound. Inv-Reg is orthogonal to existing methods, and combining Inv-Reg with two strong baselines (Arcface and CIFP) leads to new state-of-the-art that improves face recognition on a variety of demographic groups. Code is available at https://github.com/milliema/InvReg.

One of the challenges in developing autonomous vehicles (AV) is the collection of suitable real environment data for the training and evaluation of machine learning algorithms for autonomous vehicles. Such environment data collection via various sensors mounted on AV is big data in nature which require massive time and money investment and in some specific scenarios could pose a significant danger to human lives. This necessitates the virtual scenarios simulator to simulate the real environment by generating big data images from a virtual fisheye lens that can mimic the field of view and radial distortion of commercial available camera lens of any manufacturer and model. In this paper, we proposed the novelty of developing a fisheye lens with distortion system to generate big data scenarios images to train and test imaged based sensing functions and to evaluate scenarios according to EuroNCAP standards. A total of 10,123 RGB, depth and segmentation images of varying road scenarios were generated by proposed system in approximately 14 hours as compared to existing methods of 20 hours, achieving 42.86% improvement.

Driver poses recognition contains three tasks such as body joint, head angle, and face landmark estimation, which is of paramount interest for the advanced driver assistance systems (ADAS). Recently proposed methods intend to use deeper and more complicated networks to achieve better performance, which leads to heavy models that are not feasible for the resource limited applications such as ADAS. To resolve this issue, we have worked on the following aspects: 1) a lightweight network model, which is referred to as recurrent multi-task thin net (RM-ThinNet), has been proposed which was especially designed for the computationally and memory limited devices; 2) a recurrent structure has been introduced to handle the scale difference and dependency between different tasks, and this recurrence ensures the different tasks are accomplished at different stages and their outputs can augment each other; and 3) a virtual data synthesization pipeline and a couple transfer learning method have been presented, by which network can be learnt effectively by relatively a small number of real data. Comparisons with the state-of-the-art methods reveal the superiority of the proposed method and competitive performance can be achieved with smaller model size and faster speed.

Pose variation is one key challenge in face recognition. As opposed to current techniques for pose invariant face recognition, which either directly extract pose invariant features for recognition, or first normalize profile face images to frontal pose before feature extraction, we argue that it is more desirable to perform both tasks jointly to allow them to benefit from each other. To this end, we propose a Pose Invariant Model (PIM) for face recognition in the wild, with three distinct novelties. First, PIM is a novel and unified deep architecture, containing a Face Frontalization sub-Net (FFN) and a Discriminative Learning sub-Net (DLN), which are jointly learned from end to end. Second, FFN is a well-designed dual-path Generative Adversarial Network (GAN) which simultaneously perceives global structures and local details, incorporated with an unsupervised cross-domain adversarial training and a "learning to learn" strategy for high-fidelity and identity-preserving frontal view synthesis. Third, DLN is a generic Convolutional Neural Network (CNN) for face recognition with our enforced cross-entropy optimization strategy for learning discriminative yet generalized feature representation. Qualitative and quantitative experiments on both controlled and in-the-wild benchmarks demonstrate the superiority of the proposed model over the state-of-the-arts.

Learning from synthetic faces, though perhaps appealing for high data efficiency, may not bring satisfactory performance due to the distribution discrepancy of the synthetic and real face images. To mitigate this gap, we propose a 3D-Aided Deep Pose-Invariant Face Recognition Model (3D-PIM), which automatically recovers realistic frontal faces from arbitrary poses through a 3D face model in a novel way. Specifically, 3D-PIM incorporates a simulator with the aid of a 3D Morphable Model (3D MM) to obtain shape and appearance prior for accelerating face normalization learning, requiring less training data. It further leverages a global-local Generative Adversarial Network (GAN) with multiple critical improvements as a refiner to enhance the realism of both global structures and local details of the face simulator’s output using unlabelled real data only, while preserving the identity information. Qualitative and quantitative experiments on both controlled and in-the-wild benchmarks clearly demonstrate superiority of the proposed model over state-of-the-arts.

Synthesizing realistic profile faces is promising for more efficiently training deep pose-invariant models for large-scale unconstrained face recognition, by populating samples with extreme poses and avoiding tedious annotations. However, learning from synthetic faces may not achieve the desired performance due to the discrepancy between distributions of the synthetic and real face images. To narrow this gap, we propose a Dual-Agent Generative Adversarial Network (DA-GAN) model, which can improve the realism of a face simulator's output using unlabeled real faces, while preserving the identity information during the realism refinement. The dual agents are specifically designed for distinguishing real v.s. fake and identities simultaneously. In particular, we employ an off-the-shelf 3D face model as a simulator to generate profile face images with varying poses. DA-GAN leverages a fully convolutional network as the generator to generate high-resolution images and an auto-encoder as the discriminator with the dual agents. Besides the novel architecture, we make several key modifications to the standard GAN to preserve pose and texture, preserve identity and stabilize training process: (i) a pose perception loss; (ii) an identity perception loss; (iii) an adversarial loss with a boundary equilibrium regularization term. Experimental results show that DA-GAN not only presents compelling perceptual results but also significantly outperforms state-of-the-arts on the large-scale and challenging NIST IJB-A unconstrained face recognition benchmark. In addition, the proposed DA-GAN is also promising as a new approach for solving generic transfer learning problems more effectively.

Pedestrian detection remains a popular and challenging problem due to large variation in appearance. A robust feature extraction method is highly desired for accurate pedestrian detection. In this paper, firstly, we propose a staggered multiscale LBP histogram. In order to exploit grayscale difference information in more directions, three scales with radius of 1, 3, and 5 pixels are utilized, and different scales are staggered. The Staggered Multi-scale LBP histogram is composed of three 256-bin histograms, each of which corresponds to one of the three scales. Secondly, dimensionality of the LBP histogram is reduced using a boosting learning method. Experimental results show that the proposed feature outperforms benchmarks such as Uniform-LBP, HOG and CoHOG on INRIA, Daimler Chrysler and our Panasonic night time datasets.

Being fast to compute and simple to implement, Local Binary Pattern (LBP) has also shown superior performance in texture classification and face detection. However, it is not well optimized for pedestrian detection. At night/dark environment, pedestrian detection typically needs to overcome problems of low contrast, image blur, and image noise. A novel feature extraction method, consisting of Weighted LBP, Multi-resolution LBP, and Multi-scale LBP, is proposed to solve them. Experimental results show that the proposed method improves upon the basic LBP significantly and outperforms benchmarks such as HOG and CoHOG.

The alteration of compressed video bitstream due to embedding of digital watermark tends to produce unpredictable video bit rate variations which may in turn lead to video playback buffer overflow/underflow or transmission bandwidth violation problems. This paper presents a novel bit rate control technique for real-time MPEG watermarking applications. The proposed bit rate control scheme evaluates the combined bit length of a set of multiple watermarked VLC codewords and successively replaces watermarked VLC codewords having the largest increase in bit length with their corresponding unmarked VLC codewords until a target bit length is achieved. The proposed method offers much flexibility and scalability compared to the pioneering scheme proposed by Hartung and Girod. Experimental results show that the proposed bit rate control scheme is effective in meeting the bit rate targets and capable of improving the watermark detection robustness for different video contents compressed at different bit rates.