Facial expression recognition has been studied broadly, and several works using local micro-pattern descriptors have obtained significant results. There are, however, open questions: how to design a discriminative and robust feature descriptor?, how to select expression-related most influential features?, and how to represent the face descriptor exploiting the most salient parts of the face? In this article, we address these three issues to achieve better performance in recognizing facial expressions. First, we propose a new feature descriptor, namely Local Shape Pattern (LSP), that describes the local shape structure of a pixel’s neighborhood based on the prominent directional information by analyzing the statistics of the neighborhood gradient, which allows it to be robust against subtle local noise and distortion. Furthermore, we propose a selection strategy for learning the influential codes being active in the expression affiliated changes by selecting them exhibiting statistical dominance and high spatial variance. Lastly, we learn the size of the salient facial blocks to represent the facial description with the notion that changes in expressions vary in size and location. We conduct person-independent experiments in existing datasets after combining above three proposals, and obtain an improved performance for the facial expression recognition task.

Currently available local feature descriptors used in facial expression recognition at times suffer from unstable feature descriptions, especially in the presence of weak and distorted edges due to noise, limiting their performances. We propose a novel local descriptor named Neighborhood-aware Edge Directional Pattern (NEDP) to overcome such limitations. Instead of relying solely on the local neighborhood to describe the feature around a pixel, as done by the existing local descriptors, NEDP examines the gradients at the target (center) pixel as well as its neighboring pixels to explore a wider neighborhood for the consistency of the feature in spite of the presence of subtle distortion and noise in local region. We introduce template-orientations for the neighboring pixels, which give importance to the gradients in consistent edge directions, prioritizing the specific neighbors falling in the direction of the local edge to represent the shape of the local textures, unambiguously. Moreover, due to the effective management of the featureless regions, no such region is erroneously encoded as a feature by NEDP. Experiments of the performances for person-independent recognition on benchmark expression datasets also show that NEDP performs better than other existing descriptors, and thereby, improves the overall performance of facial expression recognition.

The following topics are dealt with: feature extraction; object detection; learning (artificial intelligence); video signal processing; video surveillance; object tracking; neural nets; image classification; image sequences; image motion analysis.

An appropriate aging description from face image is the prime influential factor in human age recognition, but still there is an absence of a specially engineered aging descriptor, which can characterize discernible facial aging cues (e.g., craniofacial growth, skin aging) from a detailed and more finer point of view. To address this issue, we propose a local face descriptor, directional age-primitive pattern (DAPP), which inherits discernible aging cue information and is functionally more robust and discriminative than existing local descriptors. We introduce three attributes for coding the DAPP description. First, we introduce Age-Primitives encoding aging related to the most crucial texture primitives, yielding a reasonable and clear aging definition. Second, we introduce an encoding concept dubbed as Latent Secondary Direction, which preserves compact structural information in the code avoiding uncertain codes. Third, a globally adaptive thresholding mechanism is initiated to facilitate more discrimination in a flat and textured region. We apply DAPP on separate age group recognition and age estimation tasks. Applying the same approach to both of these tasks is seldom explored in the literature. Carefully conducted experiments show that the proposed DAPP description outperforms the existing approaches by an acceptable margin.

This paper presents a new face descriptor, local directional ternary pattern (LDTP), for facial expression recognition. LDTP efficiently encodes information of emotion-related features (ı.e., eyes, eyebrows, upper nose, and mouth) by using the directional information and ternary pattern in order to take advantage of the robustness of edge patterns in the edge region while overcoming weaknesses of edge-based methods in smooth regions. Our proposal, unlike existing histogram-based face description methods that divide the face into several regions and sample the codes uniformly, uses a two-level grid to construct the face descriptor while sampling expression-related information at different scales. We use a coarse grid for stable codes (highly related to non-expression), and a finer one for active codes (highly related to expression). This multi-level approach enables us to do a finer grain description of facial motions while still characterizing the coarse features of the expression. Moreover, we learn the active LDTP codes from the emotion-related facial regions. We tested our method by using person-dependent and independent cross-validation schemes to evaluate the performance. We show that our approaches improve the overall accuracy of facial expression recognition on six data sets.

In this paper, we propose a hybrid background model that relies on edge and non-edge features of the image to produce the model. We encode these features into a coding scheme, that we called Local Hybrid Pattern (LHP), that selectively models edges and non-edges features of each pixel. Furthermore, we model each pixel with an adaptive code dictionary to represent the background dynamism, and update it by adding stable codes and discarding unstable ones. We weight each code in the dictionary to enhance its description of the pixel it models. The foreground is detected as the incoming codes that deviate from the dictionary. We can detect (as foreground or background) and classify (as edge or inner region) each pixel simultaneously. We tested our proposed method in existing databases with promising results.

Unattended object detection is an important task in surveillance. Thus, we propose a new method to detect unattended object by modeling the objects as newly learned temporal background. We use edge-segments to model the structural changes in the scene. Specifically, we construct distributions of these edge-segments to analyze the scene, and to segment its different components: background, fore-ground, and the interesting new objects. Additionally, we propose a clustering algorithm to recover the unattended objects from a set of edges based on the assumption that spatially close edges come from the same object. Our experiments on several datasets validate our proposed method.

For background-subtraction-based moving object detection, reliable background modeling is the most important component. Pixel-based methods are sensitive to illumination change, and edge-based methods can solve illumination-related problems, but have shape distortion problems. In this paper, we propose an edge-segment-based statistical background modeling algorithm and an online update mechanism to detect moving objects from consecutive frames, which creates a balance between the pixel- and edge-based methods. Our background modeling method uses a statistical map to model the frequency of the background-edges, as distributions that comprise support regions approximated with a quadratic function and enhanced with color and gradient information, to overcome the edge-distortion problem by matching the edge-segments to the modeled distributions. To adjust the changing background in the scenes, we propose an online-background update step for every incoming frame that updates the statistical map and enhances the information held by the distributions support regions. Furthermore, our experiments show that the proposed method obtains better results and detects moving edges efficiently.

The current contrast enhancement algorithms occasionally result in artifacts, overenhancement, and unnatural effects in the processed images. These drawbacks increase for images taken under poor illumination conditions. In this paper, we propose a content-aware algorithm that enhances dark images, sharpens edges, reveals details in textured regions, and preserves the smoothness of flat regions. The algorithm produces an ad hoc transformation for each image, adapting the mapping functions to each image's characteristics to produce the maximum enhancement. We analyze the contrast of the image in the boundary and textured regions, and group the information with common characteristics. These groups model the relations within the image, from which we extract the transformation functions. The results are then adaptively mixed, by considering the human vision system characteristics, to boost the details in the image. Results show that the algorithm can automatically process a wide range of images-e.g., mixed shadow and bright areas, outdoor and indoor lighting, and face images-without introducing artifacts, which is an improvement over many existing methods.