Facial expression recognition has been studied broadly, and several works using local micro-pattern descriptors have obtained significant results. There are, however, open questions: how to design a discriminative and robust feature descriptor?, how to select expression-related most influential features?, and how to represent the face descriptor exploiting the most salient parts of the face? In this article, we address these three issues to achieve better performance in recognizing facial expressions. First, we propose a new feature descriptor, namely Local Shape Pattern (LSP), that describes the local shape structure of a pixel’s neighborhood based on the prominent directional information by analyzing the statistics of the neighborhood gradient, which allows it to be robust against subtle local noise and distortion. Furthermore, we propose a selection strategy for learning the influential codes being active in the expression affiliated changes by selecting them exhibiting statistical dominance and high spatial variance. Lastly, we learn the size of the salient facial blocks to represent the facial description with the notion that changes in expressions vary in size and location. We conduct person-independent experiments in existing datasets after combining above three proposals, and obtain an improved performance for the facial expression recognition task.

Currently available local feature descriptors used in facial expression recognition at times suffer from unstable feature descriptions, especially in the presence of weak and distorted edges due to noise, limiting their performances. We propose a novel local descriptor named Neighborhood-aware Edge Directional Pattern (NEDP) to overcome such limitations. Instead of relying solely on the local neighborhood to describe the feature around a pixel, as done by the existing local descriptors, NEDP examines the gradients at the target (center) pixel as well as its neighboring pixels to explore a wider neighborhood for the consistency of the feature in spite of the presence of subtle distortion and noise in local region. We introduce template-orientations for the neighboring pixels, which give importance to the gradients in consistent edge directions, prioritizing the specific neighbors falling in the direction of the local edge to represent the shape of the local textures, unambiguously. Moreover, due to the effective management of the featureless regions, no such region is erroneously encoded as a feature by NEDP. Experiments of the performances for person-independent recognition on benchmark expression datasets also show that NEDP performs better than other existing descriptors, and thereby, improves the overall performance of facial expression recognition.

Detection of Betacam dropout defects that can occur in the digitisation process of old archived media has importance in the restoration of degraded data to a higher quality. Most of the existing methods rely on the temporal information of multiple consecutive frames to detect Betacam dropouts, which sometimes may not work well as several successive frames may contain a Betacam error at the same position. In this study, an automatic method of Betacam dropout error detection is proposed based on vertical patterns in a single frame. Hence, it is also applicable when temporal information‐based detectors fail. The results of performance tests done in real working environments demonstrate that the proposed Betacam dropout detection method performs much better than the existing methods.

In this paper, we propose Local Adaptive Hybrid Pattern (LAHP) to use intrinsic properties, both edge and color, of each pixel adaptively (inspired from LHP based methods) while using a single feature representation (inspired from LOBSTER). The proposed LAHP encodes edge and color information, and an adaptive factor based on gradient magnitude together as a single feature. We introduce a way to calculate the code distance of LNESP to reduce sensitive to edge distortions. Furthermore, we modify ADM (Adaptive Dictionary Model) to manage LAHP and extend a feature matching scheme of ADM to adjacent background models to reduce sensitivity to background motions. Results show promising performance than other methods in standard datasets.

The following topics are dealt with: feature extraction; object detection; learning (artificial intelligence); video signal processing; video surveillance; object tracking; neural nets; image classification; image sequences; image motion analysis.

An appropriate aging description from face image is the prime influential factor in human age recognition, but still there is an absence of a specially engineered aging descriptor, which can characterize discernible facial aging cues (e.g., craniofacial growth, skin aging) from a detailed and more finer point of view. To address this issue, we propose a local face descriptor, directional age-primitive pattern (DAPP), which inherits discernible aging cue information and is functionally more robust and discriminative than existing local descriptors. We introduce three attributes for coding the DAPP description. First, we introduce Age-Primitives encoding aging related to the most crucial texture primitives, yielding a reasonable and clear aging definition. Second, we introduce an encoding concept dubbed as Latent Secondary Direction, which preserves compact structural information in the code avoiding uncertain codes. Third, a globally adaptive thresholding mechanism is initiated to facilitate more discrimination in a flat and textured region. We apply DAPP on separate age group recognition and age estimation tasks. Applying the same approach to both of these tasks is seldom explored in the literature. Carefully conducted experiments show that the proposed DAPP description outperforms the existing approaches by an acceptable margin.

This paper presents a new face descriptor, local directional ternary pattern (LDTP), for facial expression recognition. LDTP efficiently encodes information of emotion-related features (ı.e., eyes, eyebrows, upper nose, and mouth) by using the directional information and ternary pattern in order to take advantage of the robustness of edge patterns in the edge region while overcoming weaknesses of edge-based methods in smooth regions. Our proposal, unlike existing histogram-based face description methods that divide the face into several regions and sample the codes uniformly, uses a two-level grid to construct the face descriptor while sampling expression-related information at different scales. We use a coarse grid for stable codes (highly related to non-expression), and a finer one for active codes (highly related to expression). This multi-level approach enables us to do a finer grain description of facial motions while still characterizing the coarse features of the expression. Moreover, we learn the active LDTP codes from the emotion-related facial regions. We tested our method by using person-dependent and independent cross-validation schemes to evaluate the performance. We show that our approaches improve the overall accuracy of facial expression recognition on six data sets.

In this paper, we propose a novel edge descriptor method for background modeling. In comparison to previous edge-based local-pattern methods, it is more robust to noise and illumination variations due to the use of principal gradient information in a local neighborhood. For the background modeling problem, we combined the proposed method with the Local Hybrid Pattern and experimented with an adaptive-dictionary-model based background modeling method. We show in the quantitative evaluations that the proposed methods is better than other local edge descriptors when applied to the same framework. Furthermore, we show that our proposed method is more powerful than other state of the art methods on standard datasets for the background modeling problem.

In this paper, we propose a hybrid background model that relies on edge and non-edge features of the image to produce the model. We encode these features into a coding scheme, that we called Local Hybrid Pattern (LHP), that selectively models edges and non-edges features of each pixel. Furthermore, we model each pixel with an adaptive code dictionary to represent the background dynamism, and update it by adding stable codes and discarding unstable ones. We weight each code in the dictionary to enhance its description of the pixel it models. The foreground is detected as the incoming codes that deviate from the dictionary. We can detect (as foreground or background) and classify (as edge or inner region) each pixel simultaneously. We tested our proposed method in existing databases with promising results.

A spatial detection technique, based on the extrema of current pixel's neighborhood, is proposed for Digital Dropout error, evident in the archived media. Digital dropout is a major type of damage occurred due to the physical error in original tape and tends to occur in block by block basis. As long as the presence of fast moving object is observed in video sequences where current state-of-the-art methods fail to detect accurate error information, such spatial detection offers a much better solution in terms of quality and complexity. Experiments are performed on video archives to evaluate the efficacy of the proposed technique.

Spatiotemporal image descriptors are gaining attention in the image research community for better representation of dynamic textures. In this paper, we introduce a dynamic-micro-texture descriptor, i.e., spatiotemporal directional number transitional graph (DNG), which describes both the spatial structure and motion of each local neighborhood by capturing the direction of natural flow in the temporal domain. We use the structure of the local neighborhood, given by its principal directions, and compute the transition of such directions between frames. Moreover, we present the statistics of the direction transitions in a transitional graph, which acts as a signature for a given spatiotemporal region in the dynamic texture. Furthermore, we create a sequence descriptor by dividing the spatiotemporal volume into several regions, computing a transitional graph for each of them, and represent the sequence as a set of graphs. Our results validate the robustness of the proposed descriptor in different scenarios for expression recognition and dynamic texture analysis.

Unattended object detection is an important task in surveillance. Thus, we propose a new method to detect unattended object by modeling the objects as newly learned temporal background. We use edge-segments to model the structural changes in the scene. Specifically, we construct distributions of these edge-segments to analyze the scene, and to segment its different components: background, fore-ground, and the interesting new objects. Additionally, we propose a clustering algorithm to recover the unattended objects from a set of edges based on the assumption that spatially close edges come from the same object. Our experiments on several datasets validate our proposed method.

For background-subtraction-based moving object detection, reliable background modeling is the most important component. Pixel-based methods are sensitive to illumination change, and edge-based methods can solve illumination-related problems, but have shape distortion problems. In this paper, we propose an edge-segment-based statistical background modeling algorithm and an online update mechanism to detect moving objects from consecutive frames, which creates a balance between the pixel- and edge-based methods. Our background modeling method uses a statistical map to model the frequency of the background-edges, as distributions that comprise support regions approximated with a quadratic function and enhanced with color and gradient information, to overcome the edge-distortion problem by matching the edge-segments to the modeled distributions. To adjust the changing background in the scenes, we propose an online-background update step for every incoming frame that updates the statistical map and enhances the information held by the distributions support regions. Furthermore, our experiments show that the proposed method obtains better results and detects moving edges efficiently.

Background modeling is challenging due to background dynamism. Most background modeling methods fail in the presence of intensity changes, because the model cannot handle sudden changes. A solution to this problem is to use intensity-robust features. Despite the changes of an edge's shape and position among frames, edges are less sensitive than a pixel's intensity to illumination changes. Furthermore, background models in the presence of moving objects produce ghosts in the detected output, because high quality models require ideal backgrounds. In this paper, we propose a robust statistical edge-segment-based method for background modeling of non-ideal sequences. The proposed method learns the structure of the scene using the edges' behaviors through the use of kernel-density distributions. Moreover, it uses segment features to overcome the shape and position variations of the edges. Hence, the use of segments gives us local information of the scene, and that helps us to predict the objects and background precisely. Furthermore, we accumulate segments to build edge distributions, which allow us to perform unconstrained training and to overcome the ghost effect. In addition, the proposed method uses adaptive thresholding (in the segments) to detect the moving objects. Therefore, this approach increases the accuracy over previous methods, which use fixed thresholds.

This paper proposes a novel local feature descriptor, local directional number pattern (LDN), for face analysis, i.e., face and expression recognition. LDN encodes the directional information of the face's textures (i.e., the texture's structure) in a compact way, producing a more discriminative code than current methods. We compute the structure of each micro-pattern with the aid of a compass mask that extracts directional information, and we encode such information using the prominent direction indices (directional numbers) and sign-which allows us to distinguish among similar structural patterns that have different intensity transitions. We divide the face into several regions, and extract the distribution of the LDN features from them. Then, we concatenate these features into a feature vector, and we use it as a face descriptor. We perform several experiments in which our descriptor performs consistently under illumination, noise, expression, and time lapse variations. Moreover, we test our descriptor with different masks to analyze its performance in different face analysis tasks.

In this paper, we propose a novel local feature descriptor, Local Sign Directional Pattern (LSDP), for face expression recognition. LSDP encodes the directional information of the face's textures—i.e., the texture's structure—in a compact way, producing a more discriminating code than other state-of-the-art methods. The structure of each micro-pattern is encoded by using its prominent directions and sign—which allows it to distinguish among similar structural patterns that have different intensity transitions. We divide the face into several regions, from which we extract the distributions of the LSDP features. These features are concatenated into a feature vector, and used as a face descriptor, and the expression recognition is obtained with the aid of Support Vector Machine classifiers.

Deriving an effective facial feature from original face images is a vital step for a successful automatic facial expression recognition. In this paper, we proposed a new feature descriptor, Local Principal Texture Pattern (LPTP), for expression recognition. We compute the LPTP feature, at each pixel, by extracting the principal directions of the local neighborhood, and we code the intensity differences on these directions. The mixture of direction and contrast information makes our descriptor robust against rotation and illumination changes. Consequently, we represent each expression as a distribution of LPTP codes. Our experiments demonstrate the superiority of the proposed feature, on two facial expression databases, over the existing methods.

The current contrast enhancement algorithms occasionally result in artifacts, overenhancement, and unnatural effects in the processed images. These drawbacks increase for images taken under poor illumination conditions. In this paper, we propose a content-aware algorithm that enhances dark images, sharpens edges, reveals details in textured regions, and preserves the smoothness of flat regions. The algorithm produces an ad hoc transformation for each image, adapting the mapping functions to each image's characteristics to produce the maximum enhancement. We analyze the contrast of the image in the boundary and textured regions, and group the information with common characteristics. These groups model the relations within the image, from which we extract the transformation functions. The results are then adaptively mixed, by considering the human vision system characteristics, to boost the details in the image. Results show that the algorithm can automatically process a wide range of images-e.g., mixed shadow and bright areas, outdoor and indoor lighting, and face images-without introducing artifacts, which is an improvement over many existing methods.

The detection of moving objects depends on the accuracy of the model used to represent the background. Common pixel-based and naive edge-based approaches have many drawbacks in dynamic environments, e.g., false detections with noise. We propose a novel background model that encodes the background as edges, building a statistical distribution per segment that represents the edge behavior. We build the background distributions using a kernel-based approach; the moving objects are detected as the edges that deviate from the distributions. The method does adaptive thresholding to the edges, which maintains their shape and boosts the detection accuracy. Sets of gradient distributions are incorporated into the model, to determine edges that lie within the distributions, but are moving edges. The number of distributions is handled dynamically, allowing them to increase and decrease accordingly to the situation. The experiments show that the proposed method improves the detection rates, due to its robustness against illumination changes.

This paper presents a novel local feature descriptor, the Local Directional Pattern (LDP), for describing local image feature. A LDP feature is obtained by computing the edge response values in all eight directions at each pixel position and generating a code from the relative strength magnitude. Each bit of code sequence is determined by considering a local neighborhood hence becomes robust in noisy situation. A rotation invariant LDP code is also introduced which uses the direction of the most prominent edge response. Finally an image descriptor is formed to describe the image (or image region) by accumulating the occurrence of LDP feature over the whole input image (or image region). Experimental results on the Brodatz texture database show that LDP impressively outperforms the other commonly used dense descriptors (e.g.,Gabor-wavelet and LBP).

Automatic facial expression recognition is a challengingproblem in computer vision, and has gained significantimportance in applications of human-computer interaction.This paper presents a new appearance-based feature descriptor,the Local Directional Pattern Variance (LDPv), torepresent facial components for human expression recognition.In contrast with LDP, the proposed LDPv introducesthe local variance of directional responses to encodethe contrast information within the descriptor. Here,the LDPv represenation characterizes both spatial structureand contrast information of each micro-patterns. Templatematching and Support Vector Machine (SVM) classifierare used to classify the LDPv feature vector of differentprototypic expression images. Experimental results usingthe Cohn-Kanade database show that the LDPv descriptoryields an improved recognition rate, as compared to existingappearance-based feature descriptors, such as the Gaborwaveletand Local Binary Pattern (LBP).

We propose an edge segment based statistical backgroundmodeling algorithm and a moving edge detectionframework for the detection of moving objects. We analyzethe performance of the proposed segment based statisticalbackground model with traditional pixel based, edge pixelbased and edge segment based approaches. Existing edgebased moving object detection algorithms fetches difficultydue to the change in background motion, object shape, illuminationvariation and noise. The proposed algorithmmakes efficient use of statistical background model usingthe edge-segment structure. Experiments with natural imagesequences show that our method can detect moving objectsefficiently under the above mentioned environments.

A robust face descriptor is an essential component for a good facial expression recognition system. In this paper, we analyze the performance of a new feature descriptor, Local Directional Pattern (LDP), for the representation of facial expressions. LDP features are obtained by computing the edge response values in all eight directions at each pixel position and then a code is generated according to the relative magnitude's strength. Thus each expression is represented as a distribution of LDP codes. Different machine learning techniques are compared using Cohn-Kanade facial expression database for classification. Extensive experiments explicate the superiority of the proposed LDP based descriptor over other existing well known descriptors.

In this paper, we present a novel texture descriptor Local Directional Pattern (LDP) to represent facial image for gender classification. The face area is divided into small regions, from which LDP histograms are extracted and concatenated into a single vector to efficiently represent the face image. The classification is performed by using support vector machines (SVMs), which had been shown to be superior to traditional pattern classifiers in gender classification problem. Experimental results show the superiority of the proposed method on the images collected from FERET face database and achieved 95.05% accuracy.

A new approach for skin region segmentation is proposed. It uses color distance map (CDM) and an algorithm based on the property of flow of water. The CDM itself is a grayscale image, which makes the algorithm very simple. However, it is still capable of providing color information based on which some skin and non-skin seed regions can be determined reliably. Then a water-flow based procedure determines skin and non-skin segments completely. The color distance map is robust against variations in imaging conditions and the water-flow procedure efficiently uses the region information to extract solid skin segments without generating much noisy segments.

In this paper, we propose an approach to collaboratively track motion of a moving target in a wide area utilizing camera-equipped visual sensor networks, which are expected to play an essential role in a variety of applications such as surveillance and monitoring. A genetic fitting method for efficient contour extraction is used as inter-scene approach to detect and track the target. We also considered the existence of faulty sensors in the network which deteriorate the difficulty of target tracking problem, and proposed a robust sensor collaboration method. The experimental results have shown that the proposed target tracking approach produces very successful target tracking compared with the existing method especially in case that the target is adjacent to neighboring objects of background