Reliable indoor localization is crucial for location-based services.Unlike outdoor environments where the Global Navigation Satellite System (GNSS) is prevalent, indoor localization systems employ diverse methods to enhance the accuracy of individual devices. However, these methods face limitations, such as the dependence on pre-existing map data and the necessity of installing anchors. The advancement of the Internet of Things (IoT) and the increasing availability of smart devices have enabled the development of more flexible and dynamic indoor localization solutions. In this paper, we propose a novel method to enhance indoor localization through cooperative localization framework. The core concept involves utilizing existing robots as mobile robot anchors to enhance pedestrian localization accuracy through interaction with pedestrians, particularly in environments lacking fixed anchors. We employed a factor graph optimization approach to tightly couple intra-device and inter-device data. This integration dynamically adjusts the inclusion of anchor data based on its quality, thereby minimizing error propagation. The experimental results demonstrate that the localization accuracy of our proposed method better than extend Kalman filter algorithms, emphasizing the potential of mobile IoT devices in indoor localization systems.

In current deep learning tasks, Adam-style optimizers—such as Adam, Adagrad, RMSprop, Adafactor, and Lion—have been widely used as alternatives to SGD-style optimizers. These optimizers typically update model parameters using the sign of gradients, resulting in more stable convergence curves. The learning rate and the batch size are the most critical hyperparameters for optimizers, which require careful tuning to enable effective convergence. Previous research has shown that the optimal learning rate increases linearly (or follows similar rules) with batch size for SGD-style optimizers. However, this conclusion is not applicable to Adam-style optimizers. In this paper, we elucidate the connection between optimal learning rates and batch sizes for Adam-style optimizers through both theoretical analysis and extensive experiments. First, we raise the scaling law between batch sizes and optimal learning rates in the “sign of gradient” case, in which we prove that the optimal learning rate first rises and then falls as the batch size increases. Moreover, the peak value of the surge will gradually move toward the larger batch size as training progresses. Second, we conduct experiments on various CV and NLP tasks and verify the correctness of the scaling law.

Partitioning a large graph into smaller subgraphs by minimizing the number of cutting vertices and edges, namely cut size or replication factor, plays a crucial role in distributed graph processing tasks. However, many prior works have primarily focused on optimizing the cut size by considering only vertex balance or edge balance, leading to significant workload imbalance and consequently hindering the performance of downstream tasks. Therefore, in this paper, we address the dual-balanced graph partition problem that minimizes the cut size while simultaneously guaranteeing both vertex and edge balance. We propose a lightweight effective two-phase framework, namely fine-grained splitting and merging (FSM), which decomposes the graph into more and smaller partitions and then merges them. FSM offers the flexibility of integrating with various state-of-the-art single-balanced techniques. We develop two efficient algorithms Fast Merging and Precise Merging to enable trade-offs between computational efficiency and partitioning quality. Experimental results on large real-world graphs demonstrate that FSM achieves state-of-the-art cut size while maintaining dual balance. The runtime for downstream tasks PageRank, connected component, and diameter estimation, can be reduced by a large proportion, up to 9.43%, 11.35%, and 17.94%, respectively.

Machine learning models, especially neural network (NN) classifiers, have shown tremendous potential of being used in complex tasks such as image classification, object detection and video analytics. However, to be adopted in the real-world applications, there are still problems to be answered. One of these problems is that training machine learning models, especially NN models, requires a certain level of computation and data processing. Other problems are the limited bandwidth of the network and the possibility of exposing the privacy of the users to attacks if the training data (specially video) is going to be transferred through the network. To mitigate these problems, researchers recently proposed the concept of federated learning.In this paper, we build a video analytic application for traffic management and train it using federated learning. More specifically, each traffic surveillance camera combined with its co-located small PC are seen as the worker node in federated learning. In this way, the NN model in each node can be trained on data collected from all nodes without transmitting and sharing with a central server, which resolves all of the above mentioned problems. The performance of the trained NN model is evaluated via experiments under different open sourced datasets to demonstrate that the proposed work has the potential to enhance the detection accuracy (mAP) over 40%.

With the wide application of information and communication technologies, a great number of cyber vulnerabilities threaten the security and stability of the power system. Therefore, this article analyzed the intrusion paths, the invasion mechanisms, and the possible consequences of most potential attacks in power communication networks (PCNs); a method for classifying and evaluating defense measures considering security indexes is also proposed. First, the typical structure and hierarchy of PCNs are sorted out. Second, we study the potential cyber attacks that may occur against PCNs and classify them into hot-spot attacks and nonhot-spot attacks based on academic research and statistical analysis of historical cyber attacks. Then, the potential target devices, propagation paths, and possible consequences to PCNs are analyzed in detail for different cyber attacks. Third, a security index system to assess the resistibility, recognizability, and recoverability of different defense measures is proposed. Based on the capability of different measures against cyber attacks, the defensive measures are classified into resistibility enhancement measures, recognizability enhancement measures, and recoverability enhancement measures. Accordingly, the mechanisms of different defense measures are analyzed. After that, the effectiveness of each defense measure is evaluated based on the degree of correlation between the defense measure and the security indexes.

Traffic video analytics has become one of the core components in the evolution of transportation systems. Artificially intelligent traffic management systems apply computer vision techniques to alleviate the monotony of manually monitoring the video feeds from surveillance cameras. Object detection is the most important step in these systems, and much research has been done on identifying objects in traffic scenes. This paper reviews various algorithms used for object detection in traffic surveillance, in addition to the recent trends and future directions. Based on the approaches used in the related studies, the object detection methods are categorized into motion-based and appearance-based techniques. Each group of techniques is further classified into a number of subcategories and the advantages and disadvantages of each method are finally analyzed. The major challenges, limitations, and potential solutions are also discussed along with the future directions.

Scaling-out RDF processing to deal with graph size usually requires partitioning the RDF graph. Typical partitioning approaches minimize edge-cuts or vertex-cuts. In this paper we argue that these approaches do not avoid or reduce joins between different partitions (i.e., inter-partition join), and propose an approach based on minimizing the number of distinct crossing properties, which we call Minimum Property-Cut (MPC). This approach enables more queries to be independently evaluated without inter-partition join. However, the minimum property-cut partitioning is a NP-hard problem and we propose a heuristic greedy algorithm to address that. Extensive experiments over a variety of synthetic and real RDF graphs show that the proposed technique can significantly avoid joins and results in good performance.

Recent years have witnessed an exponential growth of model scale in deep learning-based recommender systems---from Google's 2016 model with 1 billion parameters to the latest Facebook's model with 12 trillion parameters. Significant quality boost has come with each jump of the model capacity, which makes us believe the era of 100 trillion parameters is around the corner. However, the training of such models is challenging even within industrial scale data centers. We resolve this challenge by careful co-design of both optimization algorithm and distributed system architecture. Specifically, to ensure both the training efficiency and the training accuracy, we design a novel hybrid training algorithm, where the embedding layer and the dense neural network are handled by different synchronization mechanisms; then we build a system called Persia (short for parallel recommendation training system with hybrid acceleration) to support this hybrid training algorithm. Both theoretical demonstrations and empirical studies with up to 100 trillion parameters have been conducted to justify the system design and implementation of Persia. We make Persia publicly available (at github.com/PersiaML/Persia) so that anyone can easily train a recommender model at the scale of 100 trillion parameters.

Traffic management systems capture tremendous video data and leverage advances in video processing to detect and monitor traffic incidents. The collected data are traditionally forwarded to the traffic management center (TMC) for in-depth analysis and may thus exacerbate the network paths to the TMC. To alleviate such bottlenecks, we propose to utilize edge computing by equipping edge nodes that are close to cameras with computing resources (e.g., cloudlets). A cloudlet, with limited computing resources as compared to TMC, provides limited video processing capabilities. In this paper, we focus on two common traffic monitoring tasks, congestion detection, and speed detection, and propose a two-tier edge computing based model that takes into account of both the limited computing capability in cloudlets and the unstable network condition to the TMC. Our solution utilizes two algorithms for each task, one implemented at the edge and the other one at the TMC, which are designed with the consideration of different computing resources. While the TMC provides strong computation power, the video quality it receives depends on the underlying network conditions. On the other hand, the edge processes very high-quality video but with limited computing resources. Our model captures this trade-off. We evaluate the performance of the proposed two-tier model as well as the traffic monitoring algorithms via test-bed experiments under different weather as well as network conditions and show that our proposed hybrid edge-cloud solution outperforms both the cloud-only and edge-only solutions.

Recent years have witnessed a growing list of systems for distributed data-parallel training. Existing systems largely fit into two paradigms, i.e., parameter server and MPI-style collective operations. On the algorithmic side, researchers have proposed a wide range of techniques to lower the communication via "system relaxations": quantization, decentralization, and communication delay. However, most, if not all, existing systems only rely on standard synchronous and asynchronous stochastic gradient (SG) based optimization, therefore, cannot take advantage of all possible optimizations that the machine learning community has been developing recently. Given this emerging gap between the current landscapes of systems and theory, we build Bagua, a MPI-style communication library, providing a collection of primitives, that is both flexible and modular to support state-of-the-art system relaxation techniques of distributed training. Powered by this design, Bagua has a great ability to implement and extend various state-of-the-art distributed learning algorithms. In a production cluster with up to 16 machines (128 GPUs), Bagua can outperform PyTorch-DDP, Horovod and BytePS in the end-to-end training time by a significant margin (up to 2X) across a diverse range of tasks. Moreover, we conduct a rigorous tradeoff exploration showing that different algorithms and system relaxations achieve the best performance over different network conditions.

Robust road region extraction plays a crucial role in many computer vision applications, such as automated driving and traffic video analytics. Various weather and illumination conditions like snow, fog, dawn, daytime, and nighttime often pose serious challenges to automated road region detection. This paper presents a new real-time road recognition method that is able to accurately extract the road region in traffic videos under adverse weather and illumination conditions. Specifically, the novel global foreground modeling (GFM) method is first applied to subtract the ever-changing background in the traffic video frames and robustly detect the moving vehicles which are assumed to drive in the road region. The initial road samples are then obtained from the subtracted background model in the location of the moving vehicles. The integrated features extracted from both the grayscale and the RGB and HSV color spaces are further applied to construct a probability map based on the standardized Euclidean distance between the feature vectors. Finally, the robust road mask is derived by integrating the initially estimated road region and the regions located by the flood-fill algorithm. Experimental results using a dataset of real traffic videos demonstrate the feasibility of the proposed method for automated road recognition in real-time.

A novel moving cast shadow detection method is presented in this paper to detect and remove the cast shadows from the foreground. First, the foreground is detected using the global foreground modeling (GFM) method. Second, the moving cast shadow is detected and removed from the foreground using a new moving cast shadow detection method that contains four hierarchical steps. In the first step, a set of new chromatic criteria is presented to detect the candidate shadow pixels in the HSV color space. In the second step, a new shadow region detection method is proposed to cluster the candidate shadow pixels into shadow regions. In the third step, a statistical shadow model, which uses a single Gaussian distribution to model the shadow class, is presented to classify shadow pixels. In the last step, an aggregated shadow detection method is presented for final shadow detection. Experiments using the public video data 'Highway-3' and the real traffic data from the New Jersey Department of Transportation (NJDOT) show the feasibility of the proposed method.

A new foreground segmentation method is presented in this paper for video analysis. Specifically, a new feature representation scheme is first proposed in different color spaces, namely, the RGB, the YIQ, and the YCbCr color spaces. The new feature vector, which integrates the color values in a particular color space, the horizontal and vertical Haar wavelet features, and the temporal difference features, enhances the discriminatory power. A new Global Foreground Modeling (GFM) method is then presented to improve upon the popular video analysis approaches. The Bayes classifier is finally applied for foreground segmentation in video. Experimental results using the New Jersey Department of Transportation (NJDOT) traffic video sequences show that the new foreground segmentation method achieves better performance than the popular video analysis methods.

This paper presents a novel multiple anthropological Fisher kernel (MAFK) framework for kinship verification. The proposed MAFK framework, which goes beyond the Mahalanobis distance metric learning, integrates multiple anthropology inspired features and derives semantically meaningful similarities between images. The major novelty of this paper comes from the following three aspects. First, three new anthropology inspired features (AIF) are derived by extracting the AIF-SIFT, AIF-WLD and AIF-DAISY features on images that are enhanced by an anthropology inspired similarity enhancement method extended from the SIFT flow method. Second, a novel multiple anthropological Fisher kernel framework (MAFK) is proposed which combines multiple features and their metrics between images in a unified paradigm. The MAFK is optimized as a constrained, non-negative, and weighted variant of the sparse representation problem regularized by the criterion of pushing away the nearby non-kinship samples and pulling close the kinship samples. Third, a novel normalized kernel similarity measure (NKSM) is proposed by normalizing the MAFK with the fractional power transformation and L2 normalization. The feasibility of the proposed MAFK framework is assessed on two representative kinship data sets, namely the KinFaceW-I and the KinFaceW-II data sets. The experimental results show the effectiveness of the proposed method.

This paper presents an enhanced sparse coding method by exploiting both the generative and discriminative information in sparse representation model. Specifically, the proposed generative and discriminative sparse representation (GDSR) method integrates two new criteria, namely a discriminative criterion and a generative criterion, into the conventional sparse representation criterion. The generative criterion reveals the class conditional probability of each dictionary item by using the dictionary distribution coefficients which are derived by representing each dictionary item as a linear combination of the training samples. To further enhance the discriminative ability of the proposed method, a discriminative criterion is also applied using new localized within-class and between-class scatter matrices. Moreover, a novel GDSR based classification (GDSRc) method is proposed by utilizing both the derived sparse representation and the dictionary distribution coefficients. This hybrid method provides new insights, and leads to an effective representation and classification schema for improving the classification performance. The largest step size for learning the sparse representation is theoretically derived to address the convergence issues in the optimization procedure of the GDSR method. Extensive experimental results and analysis on several public classification datasets show the feasibility and effectiveness of the proposed method.

This paper presents an innovative sparse representation model using the complete marginal Fisher analysis (CMFA) framework for different challenging visual recognition tasks. First, a complete marginal Fisher analysis method is presented by extracting the discriminatory features in both the column space of the local samples based within the class scatter matrix and the null space of its transformed matrix. The rationale of extracting features in both spaces is to enhance the discriminatory power by further utilizing the null space, which is not accounted for in the marginal Fisher analysis method. Second, a discriminative sparse representation model is proposed by integrating a representation criterion such as the sparse representation and a discriminative criterion for improving the classification capability. In this model, the largest step size for learning the sparse representation is derived to address the convergence issues in optimization, and a dictionary screening rule is presented to purge the dictionary items with null coefficients for improving the computational efficiency. Experiments on some challenging visual recognition tasks using representative datasets, such as the Painting-91 dataset, the 15 scene categories dataset, the MIT-67 indoor scenes dataset, the Caltech 101 dataset, the Caltech 256 object categories dataset, the AR face dataset, and the extended Yale B dataset, show the feasibility of the proposed method.

A locally linear K Nearest Neighbor (LLK) method is presented in this paper with applications to robust visual recognition. Specifically, the concept of an ideal representation is first presented, which improves upon the traditional sparse representation in many ways. The objective function based on a host of criteria for sparsity, locality, and reconstruction is then optimized to derive a novel representation, which is an approximation to the ideal representation. The novel representation is further processed by two classifiers, namely, an LLK-based classifier and a locally linear nearest mean-based classifier, for visual recognition. The proposed classifiers are shown to connect to the Bayes decision rule for minimum error. Additional new theoretical analysis is presented, such as the nonnegative constraint, the group regularization, and the computational efficiency of the proposed LLK method. New methods such as a shifted power transformation for improving reliability, a coefficients' truncating method for enhancing generalization, and an improved marginal Fisher analysis method for feature extraction are proposed to further improve visual recognition performance. Extensive experiments are implemented to evaluate the proposed LLK method for robust visual recognition. In particular, eight representative data sets are applied for assessing the performance of the LLK method for various visual recognition applications, such as action recognition, scene recognition, object recognition, and face recognition.

Anthropology studies show that genetic features are inherited by children from their parents resulting in visual resemblance between them. This paper presents a novel SIFT flow based genetic Fisher vector feature (SF-GFVF) which enhances the facial genetic features for kinship verification. The proposed SF-GFVF feature is derived by applying a novel similarity enhancement method based on SIFT flow and learning an inheritable transformation on the Fisher vector feature so as to enhance and encode the genetic features of parent and child image in kinship relations. In particular, the similarity enhancement method is first presented by applying the SIFT flow algorithm to the densely sampled SIFT features in order to intensify the genetic features. Further analysis shows the relation of the extracted genetic features to anthropological results and discovers interesting patterns in different kinship relations. Finally, an inheritable transformation is applied to the enhanced Fisher vector feature which is learned with the criterion of minimizing the distance between kinship samples and maximizing the distance between non-kinship samples. Experimental results on the two representative kinship databases, namely the KinFace W-I and the Kinship W-II data sets show that the proposed method is able to outperform other popular methods.

Anthropology studies discover that some genetic related facial features, which are inherited by children from their parents, can be used for kinship verification. This paper investigates an important inheritable feature - color and presents a novel inheritable color space (InCS) and a generalized InCS (GInCS) framework with application to kinship verification. Specifically, a novel color similarity measure (CSM) is first defined. Second, based on this similarity measure, a new inheritable color space (InCS) is derived by balancing the criterion of minimizing the distance between kinship pairs and the criterion of maximizing the distance between non-kinship pairs. Unlike conventional color spaces, e.g. the RGB color space, the proposed InCS, which is learned automatically from the data, captures the inheritable information between parent and child. Third, theoretical and empirical analysis show that the proposed InCS exhibits the decorrelation property, which is positively related to the performance of kinship verification. Robustness to the illumination variation is also discussed. Fourth, a generalized InCS framework is presented to extend the InCS from the pixel level to the feature level for improving the performance and the robustness to illumination variation. The proposed InCS is evaluated on several popular datasets, namely the KinFaceW-I dataset, the KinFaceW-II dataset, the UB KinFace dataset, and the Cornell KinFace dataset. Experimental results show that the proposed InCS is able to (i) improve the conventional color spaces such as RGB, YUV, YIQ color spaces by a large margin, (ii) achieve robustness to the illumination variation, and (iii) outperforms other popular methods.

This paper presents a novel set of image features that encode the local, color, spatial, relative intensity information and gradient orientation of the painting image for painting artist classification, style classification as well as artist and style influence analysis. In particular, a new color DAISY Fisher vector (CD-FV) feature is first created by computing Fisher vectors on densely sampled DAISY features. Second, a color WLD-SIFT Fisher vector (CWS-FV) feature is developed by fusing Weber local descriptors (WLD) with Scale Invariant Feature Transform (SIFT) descriptors and Fisher vectors are computed on the fused WLD-SIFT features. Finally, an innovative color multi-fusion Fisher vector (CMFFV) feature is developed by integrating the Principal Component Analysis (PCA) features of CD-FV, CWS-FV and color SIFT-FV features. The effectiveness of the proposed CMFFV feature is assessed on the challenging Painting-91 dataset. Experimental results show that the proposed CMFFV feature is able to (i) achieve the state-of-the-art performance for painting artist classification, (ii) outperform other popular image descriptors, as well as (iii) discover the artist and style influence to understand their connections and evolution in different art movement periods.

This paper presents a novel locally linear KNN model with the goal of not only developing efficient representation and classification methods, but also establishing a relation between them so as to approximate some classification rules, e.g. the Bayes decision rule. Towards that end, first, the proposed model represents the test sample as a linear combination of all the training samples and derives a new representation by learning the coefficients considering the reconstruction, locality and sparsity constraints. The theoretical analysis shows that the new representation has the grouping effect of the nearest neighbors, which is able to approximate the “ideal representation”. And then the locally linear KNN model based classifier (LLKNNC), which shows its connection to the Bayes decision rule for minimum error in the view of kernel density estimation, is proposed for classification. Besides, the locally linear nearest mean classifier (LLNMC), whose relation to the LLKNNC is just like the nearest mean classifier to the KNN classifier, is also derived. Furthermore, to provide reliable kernel density estimation, the shifted power transformation and the coefficients cut-off method are applied to improve the performance of the proposed method. The effectiveness of the proposed model is evaluated on several visual recognition tasks such as face recognition, scene recognition, object recognition and action recognition. The experimental results show that the proposed model is effective and outperforms some other representative popular methods.

This paper presents a novel general k nearest neighbour classifier (GKNNc) and a novel general nearest mean classifier (GNMc) for visual classification. Instead of treating the data equally, both GKNNc and GNMc assign a weight coefficient to each data. To achieve good performance, the conditions and properties of the weight coefficients for GKNNc and GNMc are further analysed. Then a sparse representation based method is proposed to derive the weight coefficients for both GKNNc and GNMc. Experimental results on several representative data sets, such as the Caltech 101 dataset and the MIT-67 indoor scenes dataset demonstrate the feasibility of the proposed methods.

This paper presents two novel discriminative dictionary learning models for sparse representation, namely the Fisher discriminative sparse model (FDSM) and the marginal Fisher discriminative sparse model (MFDSM). To learn the FDSM and the MFDSM efficiently and homogeneously, a general Fisher regularized model is further derived so that both of them can be learned without much modification. Experimental results on four popular databases, namely the extended Yale face database B, the AR face database, the 15 scenes dataset and the MIT-67 indoor scenes dataset show that the proposed method can improve upon other popular methods.

This paper presents a novel locally linear KNN method with an improved marginal Fisher analysis for image classification. First, the discriminating color space (DCS), which is derived by discriminant analysis of the red, green, and blue primary colors, is integrated into the proposed method. Second, an improved marginal Fisher analysis (IMFA) applies an eigenvalue spectrum analysis to improve the generalization performance of the marginal Fisher analysis method. Third, a new locally linear KNN classifier (LLKNN), which represents the test image as a linear combination of its k nearest training images and assigns it to the class with the largest sum of weights, is presented to improve upon the traditional KNN approach. The effectiveness of the proposed method is evaluated on two representative datasets, namely the AR face image data set and the ETH-80 image data set. Experimental results show that the proposed method performs better than some representative state-of-the-art methods.

This paper introduces a new local feature description method to categorize scene images. We encode local image information by exploring the pseudo-Wigner distribution of images and the Local Binary Patterns (LBP) technique and make four major contributions. In particular, we first define a multi-neighborhood LBP for small image blocks. Second, we combine the multi-neighborhood LBP with the pseudo-Wigner distribution of images for feature extraction. Third, we derive the innovative WLBP feature vector by utilizing the frequency domain smoothing, the bag-of-words model and spatial pyramid representations of an image. Finally, we perform extensive experiments to evaluate the performance of the proposed WLBP descriptor. Specifically, we test our descriptor for classification performance using a Support Vector Machine (SVM) classifier on three fairly challenging publicly available image datasets, namely the UIUC Sports Event dataset, the Fifteen Scene Categories dataset and the MIT Scene dataset. Experimental results reveal that the proposed WLBP descriptor outperforms the traditional LBP technique and yields results better than some other popular image descriptors.

This paper proposes a novel fully automatic method to model the low-level human and object interactions for action recognition in the static images. Specifically, we exploit both the superpixels and the grid patches of an image to construct a hierarchical interaction graph and then develop an HITS map learning algorithm to learn the human-object interactions for recognizing the human actions. The major contributions of the paper are three-fold. First, a novel two-layer hierarchical interaction graph based on the superpixels and the grid patches is presented to model the low-level human-object interactions. Second, the novel HITS map, which is derived by the weighted HITS algorithm on the hierarchical interaction graph, assigns heavy weights to the important superpixels and grid patches that reveal more meaningful interactions. Third, the novel weighted image representation is derived from the learned HITS map for action recognition. Extensive experimental results show the feasibility of the proposed method using three representative datasets, namely, the Willow Action dataset, the UIUC Sports Event dataset and the CMU Sports dataset. In particular, the proposed method is able to (i) automatically model the human-object interactions without extensive manual annotations or numerous error-prone detections, and (ii) improve upon other popular methods in terms of action recognition performance.

This paper proposes three clustering-based discriminant analysis (CDA) models to address the problem that the Fisher linear discriminant may not be able to extract adequate features for satisfactory performance, especially for two class problems. The first CDA model, CDA-1, divides each class into a number of clusters by means of the k-means clustering technique. In this way, a new within-cluster scatter matrix Sw(c) and a new between-cluster scatter matrix Sb(c) are defined. The second and the third CDA models, CDA-2 and CDA-3, define a nonparametric form of the between-cluster scatter matrices N-Sb(c). The nonparametric nature of the between-cluster scatter matrices inherently leads to the derived features that preserve the structure important for classification. The difference between CDA-2 and CDA-3 is that the former computes the between-cluster matrix N-Sb(c) on a local basis, whereas the latter computes the between-cluster matrix N-Sb(c) on a global basis. This paper then presents an accurate CDA-based eye detection method. Experiments on three widely used face databases show the feasibility of the proposed three CDA models and the improved eye detection performance over some state-of-the-art methods.

The Histograms of Oriented Gradients (HOG) descriptor represents shape information by storing the local gradients in an image. The Haar wavelet transform is a simple yet powerful technique that can separately enhance the horizontal and vertical local features in an image. In this paper, we enhance the HOG descriptor by subjecting the image to the Haar wavelet transform and then computing HOG from the result in a manner that enriches the shape information encoded in the descriptor. First, we define the novel HaarHOG descriptor for grayscale images and extend this idea for color images. Second, we compare the image recognition performance of the HaarHOG descriptor with the traditional HOG descriptor in four different color spaces and grayscale. Finally, we compare the image classification performance of the HaarHOG descriptor with some popular descriptors used by other researchers on four grand challenge datasets.

This paper introduces several novel color, shape and texture-based image descriptors for scene image classification with applications to image search and retrieval. Specifically, first, a new 3-Dimensional Local Binary Pattern (3DLBP) descriptor is proposed for color image local feature extraction. Second, a new shape descriptor (HaarHOG) is introduced by combining Haar wavelet transformation and Histogram of Oriented Gradients (HOG). Third, these descriptors are fused using an optimal feature representation technique to generate a robust 3-Dimensional LBP-HaarHOG (3DLH) descriptor that can perform well on different scene image categories. Finally, the Enhanced Fisher Model (EFM) is applied for discriminatory feature extraction and the nearest neighbor classification rule is used for image classification. The proposed descriptors and fusion technique are evaluated using three grand challenge datasets: the MIT Scene dataset, the UIUC Sports Event dataset, and a part of the Caltech 256 dataset.

For complicated classification problems, the standard Support Vector Machine (SVM) is likely to be complex and thus the classification efficiency is low. In this paper, we propose a new efficient SVM (eSVM), which is based on the idea of minimizing the margin of misclassified samples. Compared with the conventional SVM, the eSVM is defined on fewer support vectors and thus can achieve much faster classification speed and comparable or even higher classification accuracy. We then present a real-time accurate eye localization system using the eSVM together with color information and 2D Haar wavelet features. Experiments on some public data sets show that (i) the eSVM significantly improves the efficiency of the standard SVM without sacrificing its accuracy and (ii) the eye localization system has real-time speed and higher detection accuracy than some state-of-the-art approaches.

This paper presents a fast method for detecting the center of the eye in color face images using different color spaces. Specifically, this method consists of three stages. First, a color face image is transformed from the RGB color space to the YUV color space to extract the U color component image, whose binary image is utilized by projection functions to roughly locate the eye boundaries. Second, the center of the eye is identified within the eye boundaries through two different approaches: one approach converts the RGB image to a gray scale image and pinpoints the center of the eye with the lowest intensity value, while the other approach transforms the color image from the RGB color space to the HSV color space and singles out the center of the eye with the largest intensity variation compared with its 8-neighbors in the H color component image. Note that the better result due to these two approaches is chosen as the center of the eye. Finally, the center of the eye is adjusted based on the prior knowledge of anthropometry for further improving the accuracy of eye detection. Experiments using 974 randomly chosen Face Recognition Grand Challenge (FRGC) images show the feasibility of our eye detection method. In particular, the eye detection rate of both eye centers being accurately detected is 95.4%.

This correspondence presents a novel face recognition method that extracts multiple features in the color image discriminant (CID) color space, where three new color component images, D1, D2, and D3, are derived using an iterative algorithm. As different color component images in the CID color space display different characteristics, three different image encoding methods are presented to effectively extract features from the component images for enhancing pattern recognition performance. To further improve classification performance, the similarity scores due to the three color component images are fused for the final decision making. Experimental results using two large-scale face databases, namely, the face recognition grand challenge (FRGC) version 2 database and the FERET database, show the effectiveness of the proposed method.

This paper presents a novel independent component analysis (ICA) color space method for pattern recognition. The novelty of the ICA color space method is twofold: 1) deriving effective color image representation based on ICA, and 2) implementing efficient color image classification using the independent color image representation and an enhanced Fisher model (EFM). First, the ICA color space method assumes that each color image is defined by three independent source images, which can be derived by means of a blind source separation procedure, such as ICA. Unlike the RGB color space, where the R , G, and B component images are correlated, the new ICA color space method derives three component images C(1) , C(2) , and C(3) that are independent and hence uncorrelated. Second, the three independent color component images are concatenated to form an augmented pattern vector, whose dimensionality is reduced by principal component analysis (PCA). An EFM then derives the discriminating features of the reduced pattern vector for pattern recognition. The effectiveness of the proposed ICA color space method is demonstrated using a complex grand challenge pattern recognition problem and a large scale database. In particular, the face recognition grand challenge (FRGC) and the biometric experimentation environment (BEE) reveal that for the most challenging FRGC version 2 Experiment 4, which contains 12,776 training images, 16,028 controlled target images, and 8014 uncontrolled query images, the ICA color space method achieves the face verification rate (ROC III) of 73.69% at the false accept rate (FAR) of 0.1%, compared to the face verification rate (FVR) of 67.13% of the RGB color space (using the same EFM) and 11.86% of the FRGC baseline algorithm at the same FAR.

In a wide range of color-related computer vision applications, researchers tried to select one of the conventional color spaces as the optimum one. This paper, however, addresses the problem of how to learn an optimum color space from the given training sample set. We seek a set of optimal coefficients to combine the R, G and B components based on a discriminant criterion and then gain one discriminant color component for representing color image for recognition purposes. Further, we can obtain three sets of optimal combination coefficients and use them to generate a three-dimensional discriminant color space (DCS). The proposed DCS method was assessed on Experiment 4 of the Face Recognition Grand Challenge (FRGC) database and the experimental results show the proposed discriminant color space significantly outperforms the RGB and Ig(r-g) color spaces.

This paper first clarifies Assumption 3 (which misses a constant) and Assumption 4 (where the whitened pattern vectors refer to the whitened means) in paper "The Bayes Decision Rule Induced Similarity Measures" (IEEE Transactions on Pattern Analysis and Machine Intelligence), vol. 29, no. 6, pp. 1086-1090, 2007), and then provides examples to show that the assumptions after the clarification are consistent.

This paper presents learning the uncorrelated color space (UCS), the independent color space (ICS), and the discriminating color space (DCS) for face recognition. The new color spaces are derived from the RGB color space that defines the tristimuli R, G, and B component images. While the UCS decorrelates its three component images using principal component analysis (PCA), the ICS derives three independent component images by means of blind source separation, such as independent component analysis (ICA). The DCS, which applies discriminant analysis, defines three new component images that are effective for face recognition. Effective color image representation is formed in these color spaces by concatenating their component images, and efficient color image classification is achieved using the effective color image representation and an enhanced Fisher model (EFM). Experiments on the face recognition grand challenge (FRGC) and the biometric experimentation environment (BEE) show that for the most challenging FRGC version 2 Experiment 4, which contains 12 776 training images, 16 028 controlled target images, and 8014 uncontrolled query images, the ICS, DCS, and UCS achieve the face verification rate (ROC III) of 73.69%, 71.42%, and 69.92%, respectively, at the false accept rate of 0.1%, compared to the RGB color space, the 2-D Karhunen-Loeve (KL) color space, and the FRGC baseline algorithm with the face verification rate of 67.13%, 59.16%, and 11.86%, respectively, with the same false accept rate.

This correspondence presents a novel hybrid Color and Frequency Features (CFF) method for face recognition. The CFF method, which applies an Enhanced Fisher Model (EFM), extracts the complementary frequency features in a new hybrid color space for improving face recognition performance. The new color space, the RIQ color space, which combines the R component image of the RGB color space and the chromatic components I and Q of the YIQ color space, displays prominent capability for improving face recognition performance due to the complementary characteristics of its component images. The EFM then extracts the complementary features from the real part, the imaginary part, and the magnitude of the R image in the frequency domain. The complementary features are then fused by means of concatenation at the feature level to derive similarity scores for classification. The complementary feature extraction and feature level fusion procedure applies to the I and Q component images as well. Experiments on the Face Recognition Grand Challenge (FRGC) version 2 Experiment 4 show that i) the hybrid color space improves face recognition performance significantly, and ii) the complementary color and frequency features further improve face recognition performance.

This paper presents a basic color image discriminant (CID) model and its general version for color image recognition. The CID models seek to unify the color image representation and recognition tasks into one framework. The proposed models, therefore, involve two sets of variables: a set of color component combination coefficients for color image representation and one or multiple projection basis vectors for color image discrimination. An iterative basic CID algorithm and its general version are designed to find the optimal solution of the proposed models. The general CID (GCID) algorithm is further extended to generate three color components (such as the three color components of the RGB color images) for further improvement of the recognition performance. Experiments using the face recognition grand challenge (FRGC) database and the biometric experimentation environment (BEE) system show the effectiveness of the proposed models and algorithms. In particular, for the most challenging FRGC version 2 Experiment 4, which contains 12 776 training images, 16 028 controlled target images, and 8014 uncontrolled query images, the proposed method achieves the face verification rate (ROC III) of 78.26% at the false accept rate (FAR) of 0.1%.

This paper presents a general discriminant model (GDM) for color face recognition. The GDM model involves two sets of variables: a set of color component combination coefficients for color image representation and a set of projection basis vectors for image discrimination. An iterative whitening-maximization (IWM) algorithm is designed to find the optimal solution of the model. The proposed algorithm is further extended to generate three color components (like the three color components of RGB color images) for further improving the face recognition performance. Experiments using the face recognition grand challenge (FRGC) database and the biometric experimentation environment (BEE) system show the effectiveness of the proposed model and algorithm. In particular, for the most challenging FRGC version 2 Experiment 4, which contains 12,776 training images, 16,028 controlled target images, and 8,014 uncontrolled query images, the proposed method achieves the face verification rate (ROC III) of 74.91% at the false accept rate of 0.1%.

This paper first shows that the popular whitened cosine similarity measure is related to the Bayes decision rule under specific assumptions and then presents two new similarity measures: the PRM Whitened Cosine (PWC) similarity measure and the Within-Class Whitened Cosine (WWC) similarity measure. Experiments on face recognition using the Face Recognition Grand Challenge (FRGC) version 2 database show the effectiveness of the new measures.

This paper first discusses some theoretical properties of 2D principal component analysis (2DPCA) and then presents a horizontal and vertical 2DPCA-based discriminant analysis (HVDA) method for face verification. The HVDA method, which applies 2DPCA horizontally and vertically on the image matrices (2D arrays), achieves lower computational complexity than the traditional PCA and Fisher linear discriminant analysis (LDA)-based methods that operate on high dimensional image vectors (1D arrays). The horizontal 2DPCA is invariant to vertical image translations and vertical mirror imaging, and the vertical 2DPCA is invariant to horizontal image translations and horizontal mirror imaging. The HVDA method is therefore less sensitive to imprecise eye detection and face cropping, and can improve upon the traditional discriminant analysis methods for face verification. Experiments using the face recognition grand challenge (FRGC) and the biometric experimentation environment system show the effectiveness of the proposed method. In particular, for the most challenging FRGC version 2 Experiment 4, which contains 12\thinspace776 training images, 16 028 controlled target images, and 8014 uncontrolled query images, the HVDA method using a color configuration across two color spaces, namely, theYIQand theYCbCrcolor spaces, achieves the face verification rate (ROC III) of 78.24% at the false accept rate of 0.1%.

This paper presents a method that applies color information to improve face recognition performance of the Face Recognition Grand Challenge (FRGC) baseline algorithm, also known as the Biometric Experimentation Environment (BEE) baseline algorithm. In particular, we empirically assess the face recognition performance of the BEE baseline algorithm by applying color configurations in the YIQ and the YCbCr color spaces. The color configuration is defined as an individual or a combination of color component images. Experimental results using an FRGC ver1.0 dateset containing 1,126 images demonstrate that the YQCr color configuration improves the rank-one face recognition rate of the BEE baseline algorithm from 37% to 70%; when experimenting with an FRGC ver2.0 dataset consisting of 30,702 images, theYQCr color configuration achieves 65% verification rate comparing to the FRGC baseline performance of 12%.

This paper presents a novel pattern recognition framework by capitalizing on dimensionality increasing techniques. In particular, the framework integrates Gabor image representation, a novel multiclass Kernel Fisher Analysis (KFA) method, and fractional power polynomial models for improving pattern recognition performance. Gabor image representation, which increases dimensionality by incorporating Gabor filters with different scales and orientations, is characterized by spatial frequency, spatial locality, and orientational selectivity for coping with image variabilities such as illumination variations. The KFA method first performs nonlinear mapping from the input space to a high-dimensional feature space, and then implements the multiclass Fisher discriminant analysis in the feature space. The significance of the nonlinear mapping is that it increases the discriminating power of the KFA method, which is linear in the feature space but nonlinear in the input space. The novelty of the KFA method comes from the fact that 1) it extends the two-class kernel Fisher methods by addressing multiclass pattern classification problems and 2) it improves upon the traditional Generalized Discriminant Analysis (GDA) method by deriving a unique solution (compared to the GDA solution, which is not unique). The fractional power polynomial models further improve performance of the proposed pattern recognition framework. Experiments on face recognition using both the FERET database and the FRGC (Face Recognition Grand Challenge) databases show the feasibility of the proposed framework. In particular, experimental results using the FERET database show that the KFA method performs better than the GDA method and the fractional power polynomial models help both the KFA method and the GDA method improve their face recognition performance. Experimental results using the FRGC databases show that the proposed pattern recognition framework improves face recognition performance upon the BEE baseline algorithm and the LDA-based baseline algorithm by large margins.

Content-based face image retrieval is concerned with computer retrieval of face images (of a given subject) based on the geometric or statistical features automatically derived from these images. It is well known that color spaces provide powerful information for image indexing and retrieval by means of color invariants, color histogram, color texture, etc. This paper assesses comparatively the performance of content-based face image retrieval in different color spaces using a standard algorithm, the Principal Component Analysis (PCA), which has become a popular algorithm in the face recognition community. In particular, we comparatively assess 12 color spaces (RGB, HSV, YUV, YCbCr, XYZ, YIQ, L*a*b*, U*V*W*, L*u*v*, I1I2I3, HSI, and rgb) by evaluating seven color configurations for every single color space. A color configuration is defined by an individual or a combination of color component images. Take the RGB color space as an example, possible color configurations are R, G, B, RG, RB, GB and RGB. Experimental results using 600 FERET color images corresponding to 200 subjects and 456 FRGC (Face Recognition Grand Challenge) color images of 152 subjects show that some color configurations, such as YV in the YUV color space and YI in the YIQ color space, help improve face retrieval performance.

This paper presents a novel Gabor-based kernel Principal Component Analysis (PCA) method by integrating the Gabor wavelet representation of face images and the kernel PCA method for face recognition. Gabor wavelets first derive desirable facial features characterized by spatial frequency, spatial locality, and orientation selectivity to cope with the variations due to illumination and facial expression changes. The kernel PCA method is then extended to include fractional power polynomial models for enhanced face recognition performance. A fractional power polynomial, however, does not necessarily define a kernel function, as it might not define a positive semidefinite Gram matrix. Note that the sigmoid kernels, one of the three classes of widely used kernel functions (polynomial kernels, Gaussian kernels, and sigmoid kernels), do not actually define a positive semidefinite Gram matrix either. Nevertheless, the sigmoid kernels have been successfully used in practice, such as in building support vector machines. In order to derive real kernel PCA features, we apply only those kernel PCA eigenvectors that are associated with positive eigenvalues. The feasibility of the Gabor-based kernel PCA method with fractional power polynomial models has been successfully tested on both frontal and pose-angled face recognition, using two data sets from the FERET database and the CMU PIE database, respectively. The FERET data set contains 600 frontal face images of 200 subjects, while the PIE data set consists of 680 images across five poses (left and right profiles, left and right half profiles, and frontal view) with two different facial expressions (neutral and smiling) of 68 subjects. The effectiveness of the Gabor-based kernel PCA method with fractional power polynomial models is shown in terms of both absolute performance indices and comparative performance against the PCA method, the kernel PCA method with polynomial kernels, the kernel PCA method with fractional power polynomial models, the Gabor wavelet-based PCA method, and the Gabor wavelet-based kernel PCA method with polynomial kernels.

This paper describes an enhanced independent component analysis (EICA) method and its application to content based face image retrieval. EICA, whose enhanced retrieval performance is achieved by means of generalization analysis, operates in a reduced principal component analysis (PCA) space. The dimensionality of the PCA space is determined by balancing two competing criteria: the representation criterion for adequate data representation and the magnitude criterion for enhanced retrieval performance. The feasibility of the new EICA method has been successfully tested for content-based face image retrieval using 1,107 frontal face images from the FERET database. The images are acquired from 369 subjects under variable illumination, facial expression, and time (duplicated images). Experimental results show that the independent component analysis (ICA) method has poor generalization performance while the EICA method has enhanced generalization performance; the EICA method has better performance than the popular face recognition methods, such as the Eigenfaces method and the Fisherfaces method.

This paper presents a novel Bayesian discriminating features (BDF) method for multiple frontal face detection. The BDF method, which is trained on images from only one database, yet works on test images from diverse sources, displays robust generalization performance. The novelty of this paper comes from the integration of the discriminating feature analysis of the input image, the statistical modeling of face and nonface classes, and the Bayes classifier for multiple frontal face detection. First, feature analysis derives a discriminating feature vector by combining the input image, its 1D Harr wavelet representation, and its amplitude projections. While the Harr wavelets produce an effective representation for object detection, the amplitude projections capture the vertical symmetric distributions and the horizontal characteristics of human face images. Second, statistical modeling estimates the conditional probability density functions, or PDFs, of the face and nonface classes, respectively. While the face class is usually modeled as a multivariate normal distribution, the nonface class is much more difficult to model due to the fact that it includes "the rest of the world." The estimation of such a broad category is, in practice, intractable. However, one can still derive a subset of the nonfaces that lie closest to the face class, and then model this particular subset as a multivariate normal distribution.

We present an independent Gabor features (IGFs) method and its application to face recognition. The novelty of the IGF method comes from 1) the derivation of independent Gabor features in the feature extraction stage and 2) the development of an IGF features-based probabilistic reasoning model (PRM) classification method in the pattern recognition stage. In particular, the IGF method first derives a Gabor feature vector from a set of downsampled Gabor wavelet representations of face images, then reduces the dimensionality of the vector by means of principal component analysis, and finally defines the independent Gabor features based on the independent component analysis (ICA). The independence property of these Gabor features facilitates the application of the PRM method for classification. The rationale behind integrating the Gabor wavelets and the ICA is twofold. On the one hand, the Gabor transformed face images exhibit strong characteristics of spatial locality, scale, and orientation selectivity. These images can, thus, produce salient local features that are most suitable for face recognition. On the other hand, ICA would further reduce redundancy and represent independent features explicitly. These independent features are most useful for subsequent pattern discrimination and associative recall. Experiments on face recognition using the FacE REcognition Technology (FERET) and the ORL datasets, where the images vary in illumination, expression, pose, and scale, show the feasibility of the IGF method. In particular, the IGF method achieves 98.5% correct face recognition accuracy when using 180 features for the FERET dataset, and 100% accuracy for the ORL dataset using 88 features.

This paper introduces a novel Gabor-Fisher (1936) classifier (GFC) for face recognition. The GFC method, which is robust to changes in illumination and facial expression, applies the enhanced Fisher linear discriminant model (EFM) to an augmented Gabor feature vector derived from the Gabor wavelet representation of face images. The novelty of this paper comes from 1) the derivation of an augmented Gabor feature vector, whose dimensionality is further reduced using the EFM by considering both data compression and recognition (generalization) performance; 2) the development of a Gabor-Fisher classifier for multi-class problems; and 3) extensive performance evaluation studies. In particular, we performed comparative studies of different similarity measures applied to various classifiers. We also performed comparative experimental studies of various face recognition schemes, including our novel GFC method, the Gabor wavelet method, the eigenfaces method, the Fisherfaces method, the EFM method, the combination of Gabor and the eigenfaces method, and the combination of Gabor and the Fisherfaces method. The feasibility of the new GFC method has been successfully tested on face recognition using 600 FERET frontal face images corresponding to 200 subjects, which were acquired under variable illumination and facial expressions. The novel GFC method achieves 100% accuracy on face recognition using only 62 features.

This paper describes a novel Gabor feature classifier (GFC) method for face recognition. The GFC method employs an enhanced Fisher discrimination model on an augmented Gabor feature vector, which is derived from the Gabor wavelet transformation of face images. The Gabor wavelets, whose kernels are similar to the 2D receptive field profiles of the mammalian cortical simple cells, exhibit desirable characteristics of spatial locality and orientation selectivity. As a result, the Gabor transformed face images produce salient local and discriminating features that are suitable for face recognition. The feasibility of the new GFC method has been successfully tested on face recognition using 600 FERET frontal face images, which involve different illumination and varied facial expressions of 200 subjects. The effectiveness of the novel GFC method is shown in terms of both absolute performance indices and comparative performance against some popular face recognition schemes such as the eigenfaces method and some other Gabor wavelet based classification methods. In particular, the novel GFC method achieves 100% recognition accuracy using only 62 features.

This paper introduces a new face coding and recognition method, the enhanced Fisher classifier (EFC), which employs the enhanced Fisher linear discriminant model (EFM) on integrated shape and texture features. Shape encodes the feature geometry of a face while texture provides a normalized shape-free image. The dimensionalities of the shape and the texture spaces are first reduced using principal component analysis, constrained by the EFM for enhanced generalization. The corresponding reduced shape and texture features are then combined through a normalization procedure to form the integrated features that are processed by the EFM for face recognition. Experimental results, using 600 face images corresponding to 200 subjects of varying illumination and facial expressions, show that (1) the integrated shape and texture features carry the most discriminating information followed in order by textures, masked images, and shape images, and (2) the new coding and face recognition method, EFC, performs the best among the eigenfaces method using L(1) or L(2) distance measure, and the Mahalanobis distance classifiers using a common covariance matrix for all classes or a pooled within-class covariance matrix. In particular, EFC achieves 98.5% recognition accuracy using only 25 features.

This paper advances an integrated learning and evolutionary computation methodology for approaching the task of learning the face space. The methodology is geared to provide a framework whereby enhanced and robust face coding and classification schemes can be derived and evaluated using both machine and human benchmark studies. In particular we take an interdisciplinary approach, drawing from the accumulated and vast knowledge of both the computer vision and psychology communities, and describe how evolutionary computation and statistical learning can engage in mutually beneficial relationships in order to define an exemplar (absolute)-based coding of multidimensional face space representation for successfully coping with changing population (face) types, and to leverage past experience for incremental face space definition.

Introduces evolutionary pursuit (EP) as an adaptive representation method for image encoding and classification. In analogy to projection pursuit, EP seeks to learn an optimal basis for the dual purpose of data compression and pattern classification. It should increase the generalization ability of the learning machine as a result of seeking the trade-off between minimizing the empirical risk encountered during training and narrowing the confidence interval for reducing the guaranteed risk during testing. It therefore implements strategies characteristic of GA for searching the space of possible solutions to determine the optimal basis. It projects the original data into a lower dimensional whitened principal component analysis (PCA) space. Directed random rotations of the basis vectors in this space are searched by GA where evolution is driven by a fitness function defined by performance accuracy (empirical risk) and class separation (confidence interval). Accuracy indicates the extent to which learning has been successful, while separation gives an indication of expected fitness. The method has been tested on face recognition using a greedy search algorithm. To assess both accuracy and generalization capability, the data includes for each subject images acquired at different times or under different illumination conditions. EP has better recognition performance than PCA (eigenfaces) and better generalization abilities than the Fisher linear discriminant (Fisherfaces).

This paper introduces two new coding schemes, probabilistic reasoning models (PRM) and enhanced FLD (Fisher linear discriminant) models (EFM), for indexing and retrieval of large image databases with applications to face recognition. The unifying theme of the new schemes is that of lowering the space dimension ("data compression") subject to increased fitness for the discrimination index.

We introduce in this paper a new face coding and recognition method which employs the Enhanced FLD (Fisher Linear Discrimimant) Model (EFM) on integrated shape (vector) and texture ('shape-free' image) information. Shape encodes the feature geometry of a face while texture provides a normalized shape-free image by warping the original face image to the mean shape, i.e., the average of aligned shapes. The dimensionalities of the shape and the texture spaces are first reduced using Principal Component Analysis (PCA). The corresponding but reduced shape find texture features are then integrated through a normalization procedure to form augmented features. The dimensionality reduction procedure, constrained by EFM for enhanced generalization, maintains a proper balance between the spectral energy needs of PCA for adequate representation, and the FLD discrimination requirements, that the eigenvalues of the within-class scatter matrix should not include small trailing values after the dimensionality reduction procedure as they appear in the denominator.

This paper introduces a new face coding scheme which employs an enhanced Fisher classifier (EFC) operating on integrated shape and intensity features. The dimensionalities of the shape and the intensity image spaces are first reduced using the principal component analysis, constrained by the EFC for enhanced generalization. The reduced shape and the intensity features are then integrated through a normalization procedure to form integrated features. Experiments using 600 face images from the FERET database of varying illumination and corresponding to 200 subjects, whose facial expression can vary, show the feasibility of the new face coding scheme. In particular, the EFC achieves 98.5% recognition rate using only 25 features. Our experiments also show that the integrated shape and intensity features carry the most discriminating information followed in order by textures, shape vectors, masked images and shape images.

We introduce in this paper two probabilistic reasoning models (PPM-1 and PRM-2) which combine the Principal Component Analysis (PCA) technique and the Bayes classifier and show their feasibility on the face recognition problem. The conditional probability density function for each class is modeled using the within class scatter and the Maximum A Posteriori (MAP) classification rule is implemented in the reduced PCA subspace. Experiments carried out using 1107 facial images corresponding to 369 subjects (with 169 subjects having duplicate images) from the FERET database show that the PRM approach compares favorably against the two well-known methods for face recognition-the Eigenfaces and Fisherfaces.

This paper introduces a Bayesian framework for face recognition which unifies popular methods such as the eigenfaces and Fisherfaces and can generate two novel probabilistic reasoning models (PRM) with enhanced performance. The Bayesian framework first applies principal component analysis (PCA) for dimensionality reduction with the resulting image representation enjoying noise reduction and enhanced generalization abilities for classification tasks. Following data compression, the Bayes classifier which yields the minimum error when the underlying probability density functions (PDF) are known, carries out the recognition in the reduced PCA subspace using the maximum a posteriori (MAP) rule, which is the optimal criterion for classification because it measures class separability. The PRM models are described within this unified Bayesian framework and shown to yield better performance against both the eigenfaces and Fisherfaces methods.

We introduce two enhanced Fisher linear discriminant (FLD) models (EFM) in order to improve the generalization ability of the standard FLD based classifiers such as Fisherfaces. Similar to Fisherfaces, both EFM models apply first principal component analysis (PCA) for dimensionality reduction before proceeding with FLD type of analysis. EFM-1 implements the dimensionality reduction with the goal to balance between the need that the selected eigenvalues account for most of the spectral energy of the raw data and the requirement that the eigenvalues of the within-class scatter matrix in the reduced PCA subspace are not too small. EFM-2 implements the dimensionality reduction as Fisherfaces do. It proceeds with the whitening of the within-class scatter matrix in the reduced PCA subspace and then chooses a small set of features (corresponding to the eigenvectors of the within-class scatter matrix) so that the smaller trailing eigenvalues are not included in further computation of the between-class scatter matrix. Experimental data using a large set of faces-1,107 images drawn from 369 subjects and including duplicates acquired at a later time under different illumination-from the FERET database shows that the EFM models outperform the standard FLD based methods.