The interest in automatic emotion recognition and the larger field of Affective Computing has recently gained momentum. The current emergence of large, video-based affect datasets offering rich multi-modal inputs facilitates the development of deep learning-based models for automatic affect analysis that currently holds the state of the art. However, recent approaches to process these modalities cannot fully exploit them due to the use of oversimplified fusion schemes. Furthermore, the efficient use of temporal information inherent to these huge data are also largely unexplored hindering their potential progress. In this work, we propose a multi-modal, sequence-based neural network with gating mechanisms for Valence and Arousal based affect recognition. Our model consists of three major networks: Firstly, a latent-feature generator that extracts compact representations from both modalities that have been artificially degraded to add robustness. Secondly, a multi-task discriminator that estimates both input identity and a first step emotion quadrant estimation. Thirdly, a sequence-based predictor with attention and gating mechanisms that effectively merges both modalities and uses this information through sequence modelling. In our experiments on the SEMAINE and SEWA affect datasets, we observe the impact of both proposed methods with progressive increase in accuracy. We further show in our ablation studies how the internal attention weight and gating coefficient impact our models’ estimates quality. Finally, we demonstrate state of the art accuracy through comparisons with current alternatives on both datasets.

Facial alignment is an essential task for many higher level facial analysis applications, such as animation, human activity recognition and human - computer interaction. Although the recent availability of big datasets and powerful deep-learning approaches have enabled major improvements on the state of the art accuracy, the performance of current approaches can severely deteriorate when dealing with images in highly unconstrained conditions, which limits the real-life applicability of such models. In this paper, we propose a composite recurrent tracker with internal denoising that jointly address both single image facial alignment and deformable facial tracking in the wild. Specifically, we incorporate multilayer LSTMs to model temporal dependencies with variable length and introduce an internal denoiser which selectively enhances the input images to improve the robustness of our overall model. We achieve this by combining 4 different sub-networks that specialize in each of the key tasks that are required, namely face detection, bounding-box tracking, facial region validation and facial alignment with internal denoising. These blocks are endowed with novel algorithms resulting in a facial tracker that is both accurate, robust to in-the-wild settings and resilient against drifting. We demonstrate this by testing our model on 300-W and Menpo datasets for single image facial alignment, and 300-VW dataset for deformable facial tracking. Comparison against 20 other state of the art methods demonstrates the excellent performance of the proposed approach.

There is a growing interest in affective computing research nowadays given its crucial role in bridging humans with computers. This progress has recently been accelerated due to the emergence of bigger dataset. One recent advance in this field is the use of adversarial learning to improve model learning through augmented samples. However, the use of latent features, which is feasible through adversarial learning, is not largely explored, yet. This technique may also improve the performance of affective models, as analogously demonstrated in related fields, such as computer vision. To expand this analysis, in this work, we explore the use of latent features through our proposed adversarial-based networks for valence and arousal recognition in the wild. Specifically, our models operate by aggregating several modalities to our discriminator, which is further conditioned to the extracted latent features by the generator. Our experiments on the recently released SEWA dataset suggest the progressive improvements of our results. Finally, we show our competitive results on the Affective Behavior Analysis in-the-Wild (ABAW) challenge dataset.

In recent years, affective computing and its applications have become a fast-growing research topic. Furthermore, the rise of deep learning has introduced significant improvements in the emotion recognition system compared to classical methods. In this work, we propose a multi-modal emotion recognition model based on deep learning techniques using the combination of peripheral physiological signals and facial expressions. Moreover, we present an improvement to proposed models by introducing latent features extracted from our internal Bio Auto-Encoder (BAE). Both models are trained and evaluated on AMIGOS datasets reporting valence, arousal, and emotion state classification. Finally, to demonstrate a possible medical application in affective computing using deep learning techniques, we applied the proposed method to the assessment of anxiety therapy. To this purpose, a reduced multimodal database has been collected by recording facial expressions and peripheral signals such as electrocardiogram (ECG) and galvanic skin response (GSR) of each patient. Valence and arousal estimates were extracted using our proposed model across the duration of the therapy, with successful evaluation to the different emotional changes in the temporal domain.

Automatic kinship recognition using Computer Vision, which aims to infer the blood relationship between individuals by only comparing their facial features, has started to gain attention recently. The introduction of large kinship datasets, such as Family In The Wild (FIW), has allowed large scale dataset modeling using state of the art deep learning models. Among other kinship recognition tasks, family classification task is lacking any significant progress due to its increasing difficulty in relation to the family member size. Furthermore, most current state of-the-art approaches do not perform any data pre-processing (which try to improve models accuracy) and are trained without a regularizer (which results in models susceptible to overfitting). In this paper, we present the Deep Family Classifier (DFC), a deep learning model for family classification in the wild. We build our model by combining two sub-networks: internal Image Feature Enhancer which operates by removing the image noise and provides an additional facial heatmap layer and Family Class Estimator trained with strong regularizers and a compound loss. We observe progressive improvement in accuracy during the validation phase, with a state of the art results of 16.89% is obtained for the track 2 in the RFIW2019 challenge and 17.08% of familly classification task on FIW dataset.

Human facial tracking is an important task in computer vision, which has recently lost pace compared to other facial analysis tasks. The majority of current available tracker possess two major limitations: their little use of temporal information and the widespread use of handcrafted features, without taking full advantage of the large annotated datasets that have recently become available. In this paper we present a fully end-to-end facial tracking model based on current state of the art deep model architectures that can be effectively trained from the available annotated facial landmark datasets. We build our model from the recently introduced general object tracker Re3, which allows modeling the short and long temporal dependency between frames by means of its internal Long Short Term Memory (LSTM) layers. Facial tracking experiments on the challenging 300-VW dataset show that our model can produce state of the art accuracy and far lower failure rates than competing approaches. We specifically compare the performance of our approach modified to work in tracking-by-detection mode and showed that, as such, it can produce results that are comparable to state of the art trackers. However, upon activation of our tracking mechanism, the results improve significantly, confirming the advantage of taking into account temporal dependencies.

We propose a Multi-Instance-Learning (MIL) approach for weakly-supervised learning problems, where a training set is formed by bags (sets of feature vectors or instances) and only labels at bag-level are provided. Specifically, we consider the Multi-Instance Dynamic-Ordinal-Regression (MI-DOR) setting, where the instance labels are naturally represented as ordinal variables and bags are structured as temporal sequences. To this end, we propose Multi-Instance Dynamic Ordinal Random Fields (MI-DORF). In this framework, we treat instance-labels as temporally-dependent latent variables in an Undirected Graphical Model. Different MIL assumptions are modelled via newly introduced high-order potentials relating bag and instance-labels within the energy function of the model. We also extend our framework to address the Partially-Observed MI-DOR problems, where a subset of instance labels are available during training.We show on the tasks of weakly-supervised facial behavior analysis, Facial Action Unit (DISFA dataset) and Pain (UNBC dataset) Intensity estimation, that the proposed framework outperforms alternative learning approaches. Furthermore, we show that MIDORF can be employed to reduce the data annotation efforts in this context by large-scale.

An essential issue when training and validating computer vision systems for affect analysis is how to obtain reliable ground-truth labels from a pool of subjective annotations. In this paper, we address this problem when labels are given in an ordinal scale and annotated items are structured as temporal sequences. This problem is of special importance in affective computing, where collected data is typically formed by videos of human interactions annotated according to the Valence and Arousal (V-A) dimensions. Moreover, recent works have shown that inter-observer agreement of V-A annotations can be considerably improved if these are given in a discrete ordinal scale. In this context, we propose a novel framework which explicitly introduces ordinal constraints to model the subjective perception of annotators. We also incorporate dynamic information to take into account temporal correlations between ground-truth labels. In our experiments over synthetic and real data with V-A annotations, we show that the proposed method outperforms alternative approaches which do not take into account either the ordinal structure of labels or their temporal correlation.

This paper proposes a novel imaging biomarker of lung cancer relapse from 3-D texture analysis of CT images. Three-dimensional morphological nodular tissue properties are described in terms of 3-D Riesz-wavelets. The responses of the latter are aggregated within nodular regions by means of feature covariances, which leverage rich intra- and inter-variations of the feature space dimensions. When compared to the classical use of the average for feature aggregation, feature covariances preserve spatial co-variations between features. The obtained Riesz-covariance descriptors lie on a manifold governed by Riemannian geometry allowing geodesic measurements and differentiations. The latter property is incorporated both into a kernel for support vector machines (SVM) and a manifold-aware sparse regularized classifier. The effectiveness of the presented models is evaluated on a dataset of 110 patients with non-small cell lung carcinoma (NSCLC) and cancer recurrence information. Disease recurrence within a timeframe of 12 months could be predicted with an accuracy of 81.3-82.7%. The anatomical location of recurrence could be discriminated between local, regional and distant failure with an accuracy of 78.3-93.3%. The obtained results open novel research perspectives by revealing the importance of the nodular regions used to build the predictive models.

Limited annotated training data is a challenging problem in Action Unit recognition. In this paper, we investigate how the use of large databases labelled according to the 6 universal facial expressions can increase the generalization ability of Action Unit classifiers. For this purpose, we propose a novel learning framework: Hidden-Task Learning. HTL aims to learn a set of Hidden-Tasks (Action Units) for which samples are not available but, in contrast, training data is easier to obtain from a set of related Visible-Tasks (Facial Expressions). To that end, HTL is able to exploit prior knowledge about the relation between Hidden and Visible-Tasks. In our case, we base this prior knowledge on empirical psychological studies providing statistical correlations between Action Units and universal facial expressions. Additionally, we extend HTL to Semi-Hidden Task Learning (SHTL) assuming that Action Unit training samples are also provided. Performing exhaustive experiments over four different datasets, we show that HTL and SHTL improve the generalization ability of AU classifiers by training them with additional facial expression data. Additionally, we show that SHTL achieves competitive performance compared with state-of-the-art Transductive Learning approaches which face the problem of limited training data by using unlabelled test samples during training.

In this paper we propose a novel covariance-based framework for the robust characterization and classification of human gestures in 3D depth sequences. The proposed 4DCov descriptor uses the notion of covariance to create compact representations of complex interactions between variations of 3D features in the spatial and temporal domain, instead of using the absolute features themselves. Despite the compactness of this representation, it still offers discriminative power for human-gesture classification. The codification of feature variations along a scene makes our descriptor robust to inter-subject and intra-class variations, periodic motions and different speeds during gesture executions, compared to other key point or histogram-based descriptor approaches. Furthermore, a sparse collaborative classification method is also presented, taking advantage of our descriptor laying on a specific manifold topology and observing that similar motions are geometrically clustered in the descriptor space. Classification accuracy results are presented against state-of-the-art approaches on top of four public human gesture datasets acquired with 3D depth sensor devices, including complex gestures from different natures.

In this paper we propose MCOV, a covariance-based descriptor for the fusion of shape and color information of 3D surfaces with associated texture aiming at a robust characterization and matching of areas in 3D point clouds. The proposed descriptor is based on the notion of covariance in order to create compact representations of the variations of texture and surface features in a radial neighbourhood, instead of using the absolute features themselves. Even if this representation is compact and low dimensional, it still offers discriminative power for complex scenes. The codification of feature variations in a close environment of a point provides invariance to rigid spatial transformations and robustness to changes in noise and scene resolution from a simple formulation perspective. Results on 3D points discrimination are validated by testing this approach performance on top of a selected database, corroborating the adequacy of our approach on the posed challenging conditions and outperforming other state-of-the-art 3D point descriptor methods. A qualitative test application on matching objects on scenes acquired with a common depth-sensor device is also provided.

We propose a real-time, robust to outliers and accurate solution to the Perspective-n-Point (PnP) problem. The main advantages of our solution are twofold: first, it in- tegrates the outlier rejection within the pose estimation pipeline with a negligible computational overhead, and sec- ond, its scalability to arbitrarily large number of correspon- dences. Given a set of 3D-to-2D matches, we formulate pose estimation problem as a low-rank homogeneous sys- tem where the solution lies on its 1D null space. Outlier correspondences are those rows of the linear system which perturb the null space and are progressively detected by projecting them on an iteratively estimated solution of the null space. Since our outlier removal process is based on an algebraic criterion which does not require computing the full-pose and reprojecting back all 3D points on the image plane at each step, we achieve speed gains of more than 100× compared to RANSAC strategies. An extensive exper- imental evaluation will show that our solution yields accu- rate results in situations with up to 50% of outliers, and can process more than 1000 correspondences in less than 5ms.

With the availability of large and heterogeneous corpora of untranscribed speech we have recently seen regained interest for algorithms to perform automatic segmentation of such data into acoustically homogeneous or phonetic units. In this paper, we face the problem of phonetic segmentation under a hierarchical clustering (HC) framework. Concretely, we focus on the task of automatically estimating the optimum number of segments in speech data. For this purpose we present aRiemannian stopping criterion that is able to automatically stop the HC processing when it is close to the underlying number of phonetic segments while providing a lower variance(robust) estimation of the optimal number of segments. We test the proposed criterion using TIMIT data and show that it outperforms previous approaches obtaining a significantly lower over/under-segmentation variance by 46, 1% relative and average improvement of 0:14 compared to a previously proposed approach. We also show that the proposed method is robust in automatically finding the correct number of segments under data source variations.

In this paper we focus on unsupervised discovering of acoustic classes suitable for use in pattern recognition applications. Our approach is based on a two-level clustering of an initial acoustic segmentation of the audio data in order to allow for discovery and correct modeling of complex acoustic classes. Initially, in a first-level, the acoustic space is densely clustered in order to provide a first layer of acoustic variance reduction. In a second-level clustering we use the acoustic segmentation to infer a smaller number of super-clusters taking advantage of the intra-segment relationships between the first-level clusters. In this paper we compare three possible clustering methods to obtain super-clusters as sub-sets or linear combinations of first-level clusters. Results indicate that the proposed two-level approach improves the balance between Purity and inverse Purity evaluation measures while significantly improving the stability of the transcriptions obtained when using the resulting models to transcribe the same acoustic events in different spoken utterances.

We propose a new algorithm to detect facial points in frontal and near-frontal face images. It combines a regression-based approach with a probabilistic graphical model-based face shape model that restricts the search to anthropomorphically consistent regions. While most regression-based approaches perform a sequential approximation of the target location, our algorithm detects the target location by aggregating the estimates obtained from stochastically selected local appearance information into a single robust prediction. The underlying assumption is that by aggregating the different estimates, their errors will cancel out as long as the regressor inputs are uncorrelated. Once this new perspective is adopted, the problem is reformulated as how to optimally select the test locations over which the regressors are evaluated. We propose to extend the regression-based model to provide a quality measure of each prediction, and use the shape model to restrict and correct the sampling region. Our approach combines the low computational cost typical of regression-based approaches with the robustness of exhaustive-search approaches. The proposed algorithm was tested on over 7,500 images from five databases. Results showed significant improvement over the current state of the art.

Real-time creation of video mosaics needs fast and accurate motion computation. While most mosaicing methods can use 2D image motion, the creation of multi view stereo mosaics needs more accurate 3D motion computation. Fast and accurate computation of 3D motion is challenging in the case of unstabilized cameras moving in 3D scenes, which is always the case when stereo mosaics are used. Efficient blending of the mosaic strip is also essential. Most cases of stereo mosaicing satisfy the assumption of limited camera motion, with no forward motion and no change in internal parameters. Under these assumptions uniform sideways motion creates straight epipolar lines. When the 3D motion is computed correctly, images can be aligned in space-time volume to give straight epipolar lines, a method which is depth invariant. We propose to align the video sequence in a space-time volume based on efficient feature tracking, and in this paper we used Kernel Tracking. Computation is fast as the motion in computed only for a few regions of the image, yet giving accurate 3D motion. This computation is faster and more accurate than the previously used direct approach. We also present "Barcode Blending", a new approach for using pyramid blending in video mosaics, which is very efficient. Barcode Blending overcomes the complexity of building pyramids for multiple narrow strips, combining all strips in a single blending step. The entire stereo mosaicing process is highly efficient in computation and in memory, and can be performed on mobile devices.

In this paper, we present a framework for real-time mosaicing from video sequences recorded from an uncalibrated pan tilt zoom camera based on multiframe registration. To this end, a new frame alignment algorithm, the direct local indirect global (DLIG), is presented. The key idea of the DLIG alignment is to divide the frame alignment problem into the problem of registering a set of spatially related image patches. The registration is iteratively computed by sequentially imposing a good local match and global spatial coherence. The patch registration is performed using a tracking algorithm, so a very efficient local matching can be achieved. We use the patch-based registration to obtain multiframe registration, using the mosaic coordinates to relate the current frame to patches from different frames that partially share the current field of view. Multiframe registration prevents the error accumulation problem, one of the most important problems in mosaicing. We also show how to embed a kernel tracking algorithm in order to obtain a precise and extremely efficient mosaicing algorithm. Finally, we perform a quantitative evaluation of our algorithm, including a comparison with other alignment approaches, and studying its performance against interlaced videos and illumination changes.

Finding fiducial facial points in any frame of a video showing rich naturalistic facial behaviour is an unsolved problem. Yet this is a crucial step for geometric-feature-based facial expression analysis, and methods that use appearance-based features extracted at fiducial facial point locations. In this paper we present a method based on a combination of Support Vector Regression and Markov Random Fields to drastically reduce the time needed to search for a point's location and increase the accuracy and robustness of the algorithm. Using Markov Random Fields allows us to constrain the search space by exploiting the constellations that facial points can form. The regressors on the other hand learn a mapping between the appearance of the area surrounding a point and the positions of these points, which makes detection of the points very fast and can make the algorithm robust to variations of appearance due to facial expression and moderate changes in head pose. The proposed point detection algorithm was tested on 1855 images, the results of which showed we outperform current state of the art point detectors.

In this paper, we introduce a new method to estimate a parametric description of the dominant motion existing in a video sequence, a key task needed to face more complex video analysis problems. In order to do so, we use motion data provided by the MPEG streams. We propose a method based on imaginary straight line tracking to retrieve the projective transformations that describe the dominant motion of a sequence by estimating 2-D homographies. Our method takes advantage not only of the MPEG motion data, but also of its structure. In order to overcome the noise introduced by the MPEG compression scheme, we also employ several robust estimators to attain reliability. We demonstrate its performance by displaying several image mosaics derived from the motion in real time.

In this paper we present a novel way to analyze LADAR images and model its data. Having an aerial LADAR image as data source, our aim is to extract a parametric description of the ground of our scenario in order to discern between the data samples that belong to the ground and those that belong to vehicles, objects or clutter. In the second part of this paper we estimate the pose of the interesting objects by building its corresponding oriented 3D bounding box. Our method uses robust statistics in order to extract proper descriptions of both the ground and the oriented bounding boxes of the objects. Specifically, we use two robust parameter estimators : The least median squares and the Helmolth tradeoff estimator, part of our prior work, depending on the percentage of outliers that may be present in the different steps of our approach.

In tracking tasks, representing a target region as a weighted histogram has opened possibilities which led to excellent results, as mean shift or camshift algorithms. This representation is extracted from the image by giving weights with kernels and it depends on the properties of the kernels. By a first order Taylor approximation of the histograms it is possible to perform a tracking using several kernels, interpreted as different sources of information. This representation improves the possibilities and gives more flexibility when facing problems of tracking, as occlusions, model variance or projective deformations of the image. In this paper we use this multi-kernel model representation to perform a simultaneous tracking of the entire object and also of each different part individually. This is performed in a new two-step process. In the first step we perform the multikernel estimation and in a second step we update the model representation taking into account single kernel estimations, representing the local movement of each part. From a probabilistic view of the Matusita metric we analyze the usefulness of this method against partial occlusions and some projective transformations like zooms or 3D rotations and articulated movements.

In this paper we present a novel parameter estimator that follows a general perception law as an objective function, the Helmoltz principle. We present an interpretation of this postulate in statistical terms and apply it along with low order statistics in order to obtain a flexible parameter estimator with a high breakdown point, capable of suiting the specific needs of each problem. The performance of the algorithm is tested applying the estimator to find lines in synthetic data with up to 90% of noise and real LADAR images.

Dominant motion estimation in video sequence is a task that must be often be solved in computer vision problems but involves a high computational cost due to the overwhelming amount of data to be treated when working in image domain. In this paper we introduce a novel technique to perform motion analysis in video sequences taking advantage of the motion information of MPEG streams and its structure, using imaginary line tracking and robust statistics to overcome the noise present in compressed domain information. In order to demonstrate the reliability of our new approach, we also show the results of its application to mosaic image construction problem.

We propose the use of attentional cascades based on the DCT and motion information contained in an MPEG coded stream. An attentional cascade is a sequence of very efficient classifiers that reject a large number of negative candidate regions, while keeping all the positive candidates. Working directly on the compressed domain has two main advantages: computationally expensive features are already computed, and the stream is only partially decoded without the additional cost of full decompression, which will be reached by a very small number of the initial candidate regions. We have applied these concepts to skin color detection, as a pre-attentive filtering prior to face detection, and to text region detection with particular focus on license plates for vehicle identification. In both cases, a reduction of the number of candidate regions close to 95% is achieved, which turns into an enormous performance increase in video indexing processes

In this paper, we present a new technique for separating different types of periodic motions in a video sequence. We consider different motions those that have different periodic patterns with one or many fundamental frequencies. We select the temporal Fourier transform for each pixel to be the representation space for a sequence of images. The classification is performed using nonnegative matrix factorization (NNMF) over the power spectra data set. The paper we present can be applied on a wide range of applications for video sequences analysis, such as: background subtraction on nonstatic backgrounds framework, object segmentation and classification. We point out the fact that no registration technique is applied in the method that we introduce. Nevertheless, this method can be used as a cooperative tool for the existing techniques based on camera motion models (motion segmentation, layer classification, tracking of moving objects, etc).

We present a general object detection system for video sequences based on the characterization of the temporal behavior of multiple image features, as well as any dependency relationships that may exist between them. This characterization is based on the coupling of multiple Markov chains and its generalization to coupled Markov random fields. The model automatically learns the temporal behavior of the features that better characterize an object from a training sequence. We show experiments on generic news videos with cluttered backgrounds and we propose a multi-scale process with automatic selection of the optimal scale of the object in the scene.

This paper presents a framework that creates background, foreground and a temporal summarization of the motions in a scene. The method is based on the Dominant Motion Assumption (DMA), where the background has a parametric motion and occupies the main part of the scene. Under this assumption, we present a robust optical flow based method to extract the moving parts of the scene using the clustering capabilities of mixtures of Gaussians. A general mosaicing method to summarize the background, the foreground and the trajectories of objects in the scene is also presented.

In this paper, we address the visual video summarization problem in a Bayesian framework in order to detect and describe the underlying temporal transformation symmetries in a video sequence. Given a set of time correlated frames, we attempt to extract a reduced number of image-like data structures which are semantically meaningful and that have the ability of representing the sequence evolution. To this end, we present a generative model which involves jointly the representation and the evolution of appearance. Applying Linear Dynamical System theory to this problem, we discuss how the temporal information is encoded yielding a manner of grouping the iconic representations of the video sequence in terms of invariance. The formulation of this problem is driven in terms of a probabilistic approach, which affords a measure of perceptual similarity taking both learned appearance and time evolution models into account.

This paper is about two topics. First, given a video sequence, we introduce a Bayesian framework where background and moving objects are segmented into layers. The model that describes the different layer evolutions in a sequence of images uses the results of a multi-frame optical flow estimation (MFOFE); we present a new technique based on the fact that each pixel in the frame of reference produces a trajectory in the mosaic absolute coordinate system. The second topic is video summarization. A general moisaicing method is presented for describing the background and the trajectories of moving objects in a sequence of frames. Combining layer segmentation and mosaicing, we show different manners of encoding and visualizing temporal information, where the key point is the selection of a certain object in the images as reference in the evolution.

In this paper we introduce a statistic snake that learns and tracks image features by means of statistic learning techniques. Using probabilistic principal component analysis a feature description is obtained from a training set of object profiles. In our approach a sound statistical model is introduced to define a likelihood estimate of the grey-level local image profiles together with their local orientation. This likelihood estimate allows to define a probabilistic potential field of the snake where the elastic curve deforms to maximise the overall probability of detecting learned image features. To improve the convergence of snake deformation, we enhance the likelihood map by a physics-based model simulating a dipole-dipole interaction. A new extended local coherent interaction is introduced defined in terms of extended structure tensor of the image to give priority to parallel coherence vectors.

We introduce chromatic aberration as a source of visual information that can be useful for autofocus and depth estimation. A color video camera equipped with a lens with chromatic aberration has been used to take images of both step and occlusion edges at several distances from the camera. The defocus measures obtained in the three different RGB color channels of each image are different. We suggest the way this information can be exploited in order to design an autofocus sensor, and also how depth information can be derived.

We address the object recognition problem in a probabilistic framework to detect and describe object appearance through image features organized by means of active contour models. We consider the formulation of saliency in terms of visual similarity embedded in the probabilistic principal component analysis framework. A likelihood of object structure detection is obtained using the relation between the visual field and the internal object representation. Deformable models are employed introducing a computational methodology for a perceptual organisation of image features as an abstract understanding of the integration between structure and constraints of the visual information-processing problem. A specific application of the integrated approach for vessels segmentation in angiography is considered and the results are encouraging.

Colour image variations produced by the change of video acquisition devices strongly affect colour appearance based recognition tasks in digital video applications. Up to date, many algorithms have been developed in order to achieve independence from different image formation factors like intensity, orientation and colour of the illumination source. We check the capability of three of these approaches to generate device independent colour representations in order to be used as a normalization step previous to the recognition process. Our previously developed color correction approach for video imagery based on Gaussian mixture models has also been tested. Experimental results show that a simple algorithm like the grayworld correction is enough for appearance based recognition under this particular kind of color variability. Similar recognition rates are obtained using our correction, with the advantage that the visual quality of normalized images is preserved. In digital video library applications, where not only recognition tasks are performed, this approach becomes much more useful.

We introduce a new deformable model, called eigensnake, for segmentation of elongated structures in a probabilistic framework. Instead of snake attraction by specific image features extracted independently of the snake, our eigensnake learns an optimal object description and searches for such image feature in the target image. This is achieved applying principal component analysis on image responses of a bank of Gaussian derivative filters. Therefore, attraction by eigensnakes is defined in terms of classification of image features. The potential energy for the snake is defined in terms of likelihood in the feature space and incorporated into a new energy minimising scheme. Hence, the snake deforms to minimise the mahalanobis distance in the feature space. A real application of segmenting and tracking coronary vessels in angiography is considered and the results are very encouraging.

In this paper a new criterion for determining the optimal focus setting during the image acquisition process is presented. It is based upon a new representation of the focal variation of contrast between images: the dynamic contrast histogram (DCH). The DCH of a point is a scaled joint probability density function between the gray levels of two images of the same scene, acquired with different focus settings in a neighbourhood of that point. The dynamic focusing criterion (DFC) developed here computes the slope of the principal axis of the DCH as a focusing criterion. The results of this technique are compared to other methods based on a quality measure of its focusing ability, its sensitivity to noise, and its computational complexity. Results are presented from microscopic images.