Bayesian inference is the standard for providing full predictive distributions with well calibrated uncertainty estimates. However, scaling to a modern, overparameterized deep learning setting typically comes at the cost of severe and restrictive approximations, sacrificing model predictive strength. With our approach, we factor model parameters as a function of deterministic and probabilistic components; the model is solved by combining maximum a posteriori estimation of the former, with inference over a low-dimensional, Implicit Neural Representation of the latter. This results in a solution that combines both predictive accuracy and good calibration, as it entails inducing stochasticity over the full set of model weights while being comparatively cheap to compute. Experimentally, our approach compares favorably to the state of the art, including much more expensive methods as well as less expressive posterior approximations over full network parameters.

In this work, we solve the problem of motion representation in videos, according to local transformations applied to specific keypoints extracted from static the images. First, we compute the co-ordinates of the keypoints of the body or face through a pre-trained model, and then we introduce a convolutional neural network to estimate a dense motion field through optical flow. Next, we train a generative adversarial network that exploits the previous information to generate new images that resemble as much as possible the target frames. To reduce trembling and extract smooth movements, our model incorporates a low-pass spatio-temporal Gaussian filter. Results indicate that our method provides high performance and the movement of objects is accurate and robust.

Recent architectures for object detection adopt a Feature Pyramid Network as a backbone for deep feature extraction. Many works focus on the design of pyramid networks which produce richer feature representations. In this work, we opt to learn a dataset-specific architecture for Feature Pyramid Networks. With the proposed method, the network fuses features at multiple scales, it is efficient in terms of parameters and operations, and yields better results across a variety of tasks and datasets. Starting by a complex network, we adopt Variational Inference to prune redundant connections. Our model, integrated with standard detectors, outperforms the state-of-the-art feature fusion networks.

Image deformations under simple geometric restrictions are crucial for Handwriting Text Recognition (HTR), since different writing styles can be viewed as simple geometrical deformations of the same textual elements. In this respect, the usefulness of including deformation invariance to an HTR system is indisputable. We explore different existing strategies for ensuring deformation invariance, including spatial transformers and deformable convolutions, under the context of text recognition, as well as introduce a new deformation-based algorithm, inspired by adversarial learning, which aims to reduce character output uncertainty during evaluation time. The resulting HTR system is shown to achieve state-of-the-art performance on the IAM and RIMES datasets.

In this paper, we present Wasserstein Inception Distance (WInD), a novel metric for evaluating performance of Generative Adversarial Networks (GANs). The proposed metric extends on the rationale of the previously proposed Frechet Inception Distance (FID), in the sense that GAN performance is quantified in terms of data and model distribution divergence. We extend FID by relaxing the Gaussian hypothesis of the related inception features and extend it for non-Gaussian, multimodal distributions. Gaussian Mixture Models (GMMs) are used to model data and model inception features, and the Wasserstein distance is employed as a pdf matching metric. We show that the proposed WInD metric inherits the desirable features of FID and correlates well with actual GAN performance. Furthermore, WInD can correctly evaluate cases were data and model distribution erroneously would appear as well peforming using FID. Numerical experiments on synthetic and real datasets validate our claim.

Detection of cell nuclei in microscopy images is a challenging research topic due to limitations in acquired image quality as well as due to the diversity of nuclear morphology. This has been a topic of enduring interest with promising success shown by deep learning methods. Recently, attention gating methods have been proposed and employed successfully in a diverse array of pattern recognition tasks. In this work, we introduce a novel attention module and integrate it with feature pyramid networks and the state-of-the-art Mask R-CNN network. We show with numerical experiments that the proposed model outperforms the state-of-the-art baseline.

Recent advances in deep learning have spurred 3D hand pose estimation, as convolutional network (ConvNet) based methods outperformed random forests. However, in the state of the art, ConvNet based methods employ only depth images of the hand without leveraging color and texture information from the RGB domain. In this paper, we investigate whether ConvNets can learn more rich and discriminative em-beddings, by combining RGB and depth information. To answer this question, we propose the fusion of RGB and depth information in a double-stream architecture. More specifically, RGB and depth images are fed into two separate networks by extracting features, which are subsequently fused at an intermediate layer of the ConvNet, implementing input-level fusion, feature-level fusion and score-level fusion. The double-stream scheme is coupled with a deep ConvNet, contrary to the shallow networks that are mostly proposed in the literature. Experimental results show that while the depth of the network is crucial for hand pose estimation, the double-stream nets perform very similarly with the net trained only with depth images. This may suggest that training double-stream architectures purely with supervision may be insufficient for hand pose estimation with RGB-D fusion.

Classification of cervical cells in Pap smear images is a challenging task due to the limitations these images exhibit and the complexity of the morphological changes in the structural parts of the cells. This procedure is very important as it provides fundamental information for the detection of cancerous or precancerous lesions. For this reason several algorithms have been proposed in order to classify normal and abnormal cells in such images. However, it is a common phenomenon that each research group usually creates its own dataset of images, as well-established datasets are not publicly available. To overcome this obstacle and to assist the research progress in this field, we present an annotated image database of Pap smear images, in which the cells are categorized in five different classes, depending on their cytomorphological features. The area of the cytoplasm and the nucleus in each image is manually defined by experts and salient features of intensity, texture and shape are calculated for each region of interest. Several experiments have been performed for the classification of these images and they include feature and image based classification schemes. In this direction, methods based on support vector machines and deep neural networks are tested and the performance of each classifier is presented in order to constitute a reference point for the evaluation of future classification techniques.

In this paper we present a semi-supervised, attribute-based model suitable for keyword spotting (KWS) in document images. Our model can take advantage of available non-annotated segmented word images, as well as string annotations without a matching word image. We build our model by extending on the probabilistic interpretation of Canonical Correlation Analysis (CCA), solved using Expectation-Maximization (EM). On test-time, we back-project the query and database images to the embedded space by calculating the embedding space posterior density given the observations. Keyword spotting is then efficiently performed by computing query nearest neighbours in the embedded Euclidean space. We validate that our model offers superior performance given the presence of partially-labelled data, with keyword spotting trials on the Bentham and George Washington datasets.

In this paper, a human behavior recognition method using multimodal features is presented. We focus on modeling individual and social behaviors of a subject (e.g., friendly/aggressive or hugging/kissing behaviors) with a hidden conditional random field (HCRF) in a supervised framework. Each video is represented by a vector of spatio-temporal visual features (STIP, head orientation and proxemic features) along with audio features (MFCCs). We propose a feature pruning method for removing irrelevant and redundant features based on the spatio-temporal neighborhood of each feature in a video sequence. The proposed framework assumes that human movements are highly correlated with sound emissions. For this reason, canonical correlation analysis (CCA) is employed to find correlation between the audio and video features prior to fusion. The experimental results, performed in two human behavior recognition datasets including political speeches and human interactions from TV shows, attest the advantages of the proposed method compared with several baseline and alternative human behavior recognition methods.

In this work, we investigate the problem of predicting gender from still images using human metrology. Since the values of the anthropometric measurements are difficult to be estimated accurately from state-of-the-art computer vision algorithms, ratios of anthropometric measurements were used as features. Additionally, since several measurements will not be available at test time in a real-life scenario, we opted for the Learning Using Privileged Information (LUPI) paradigm. During training, we used as features, ratios from all the available anthropometric measurements, whereas at test time only ratios of measurable (i.e., observable) quantities were used. We show that by using the LUPI framework, the estimation of soft biometric characteristics such as gender is possible. Gender classification from human metrology is also tested on real images with promising results.

In many human activity recognition systems the size of the unlabeled training data may be significantly large due to expensive human effort required for data annotation. Moreover, the insufficient data collection process from heterogenous sources may cause dissimilarities between training and testing data. To address these limitations, a novel probabilistic approach that combines learning using privileged information (LUPI) and active learning is proposed. A pool-based privileged active learning approach is presented for semi-supervising learning of human activities from multimodal labeled and unlabeled data. Both uncertainty and distance from the decision boundary are used as query inference strategies for selecting an unlabeled observation and querying its label. Experimental results in four publicly available datasets demonstrate that the proposed method can identify complex human activities with high accuracy.

In this paper, we propose a novel regression-based method for employing privileged information to estimate the height using human metrology. The actual values of the anthropometric measurements are difficult to estimate accurately using state-of-the-art computer vision algorithms. Hence, we use ratios of anthropometric measurements as features. Since many anthropometric measurements are not available at test time in real-life scenarios, we employ a learning using privileged information (LUPI) framework in a regression setup. Instead of using the LUPI paradigm for regression in its original form (i.e., ε-SVR+), we train regression models that predict the privileged information at test time. The predictions are then used, along with observable features, to perform height estimation. Once the height is estimated, a mapping to classes is performed. We demonstrate that the proposed approach can estimate the height better and faster than the ε-SVR+ algorithm and report results for different genders and quartiles of humans.

In this paper, we address the problem of word spotting using a shape-based matching scheme between segmented word images represented by local contour features. As in a typical query-by-example (QBE) paradigm, a user selects an instance of the query word from the collection of interest and a ranked list of images is returned, based on their similarity with the query. This is accomplished in two steps. The query image is firstly aligned with the test image according to a similarity measure defined on their descriptors and then the aligned images are matched through a deformable non-rigid point matching algorithm. Experiments are carried out on historical handwritten text, written in Greek and English, respectively. Moreover, comparisons with other QBE methods show the efficiency of our system as well as its flexibility in adapting to different scripts.

A method for the efficient encoding and decoding of a shape with application to edge map image compression is proposed. The method relies on the modeling of the manifold of a shape by line segments. Then, encoding is performed by collecting the characteristic features of each line segment, namely the starting and ending points and the number of points contributing to the computation of the corresponding segment. The reconstruction of the shape may be obtained by uniform sampling of points from each line segment computed in the encoding process. Experimental analysis demonstrates that in case of employing a robust and efficient line segment detection algorithm, the proposed encoding/decoding scheme exhibits high compression rates, compared to widely used lossless compression methods, while providing low distortion values.

A spatially varying Gaussian mixture model (SVGMM) prior is employed to ensure the preservation of region boundaries in penalized likelihood tomographic image reconstruction. Spatially varying Gaussian mixture models are characterized by the dependence of their mixing proportions on location (contextual mixing proportions) and they have been successfully used in image segmentation. The proposed model imposes a Student's t-distribution on the local differences of the contextual mixing proportions and its parameters are automatically estimated by a variational Expectation-Maximization (EM) algorithm. The tomographic reconstruction algorithm is an iterative process consisting of alternating between an optimization of the SVGMM parameters and an optimization for updating the unknown image using also the EM algorithm. Numerical experiments on various photon limited image scenarios show that the proposed model is more accurate than the widely used Gibbs prior.

A method for modeling and removing outliers from 2-D sets of scattered points is presented. The method relies on a principle due to Helmholtz stating that every large deviation from uniform noise should be perceptible, provided that the deviation is generated by an a contrario model of geometric structures. By assuming local linearity, we first employ a robust algorithm to model the local manifold of the corrupted data by local line segments. Our rationale is that long line segments should not be expected in a noisy set of points. This assumption leads to the modeling of the lengths of the line segments by a Pareto distribution, which is the adopted a contrario model for the observations. The model is successfully evaluated on two problems in computer vision: shape recovery and linear regression.

In this paper, we propose a method for spotting keywords in images of handwritten text. Relying on an object detection system in real images, local contour features are extracted from segmented word images in order to obtain a representative shape of a word-class. Thus, word spotting is cast following a query-by-word-class scenario where class models are generated using a random subset of the images belonging to that class. Cumbersome multi-writer conditions are tackled with a statistical model of intra-class deformations using principal component analysis (PCA). Novel word instances are detected through a combination of a Hough-style voting scheme with a non-rigid point matching algorithm. Finally, we evaluate the system's performance for word spotting as a classification task, using a vocabulary of word models.

An algorithm for sampling image edge points is presented. At first, the image edges are summarized by line segments, which represent the long axis of highly eccentric ellipses. Then, each line segment is partitioned into a number of bins and the point which is closer to the center of the bin is selected. Experiments on widely used databases demonstrate that the proposed method is accurate and provides samples that preserve the coherence of the initial information of the edge map, which is of importance in image retrieval applications.

In this paper, we propose a method to estimate the density of a data space represented by a geometric transformation of an initial Gaussian mixture model. The geometric transformation is hierarchical, and it is decomposed into two steps. At first, the initial model is assumed to undergo a global similarity transformation modeled by translation, rotation, and scaling of the model components. Then, to increase the degrees of freedom of the model and allow it to capture fine data structures, each individual mixture component may be transformed by another, local similarity transformation, whose parameters are distinct for each component of the mixture. In addition, to constrain the order of magnitude of the local transformation (LT) with respect to the global transformation (GT), zero-mean Gaussian priors are imposed onto the local parameters. The estimation of both GT and LT parameters is obtained through the expectation maximization framework. Experiments on artificial data are conducted to evaluate the proposed model, with varying data dimensionality, number of model components, and transformation parameters. In addition, the method is evaluated using real data from a speech recognition task. The obtained results show a high model accuracy and demonstrate the potential application of the proposed method to similar classification problems.

In this work, we propose an adaptive M-estimation scheme for robust image super-resolution. The proposed algorithm relies on a maximum a posteriori (MAP) framework and addresses the presence of outliers in the low resolution images. Moreover, apart from the robust estimation of the high resolution image, the contribution of the method is twofold: (i) the robust computation of the regularization parameters controlling the relative strength of the prior with respect to the data fidelity term and (ii) the robust estimation of the optimal step size in the update of the high resolution image. Experimental results demonstrate that integrating these estimations into a robust framework leads to significant improvement in the accuracy of the high resolution image.

A probabilistic real time tracking algorithm is proposed. The distribution of the target is represented by a Gaussian mixture model (GMM) and the weighted likelihood of the target is maximized in order to localize it in an image sequence. The role of the weight is important as it allows gradient based optimization to be performed, which would not be feasible in a context of standard likelihood representations. The algorithm models both the object to be tracked and local background elements and handles scale changes in target's appearance. It is experimentally demonstrated that the algorithm runs in real time, and it is at least at the same performance level with the mean shift algorithm while it provides more accurate target localization in non trivial scenarios (e.g. shadows).

A method for the segmentation of overlapping nuclei is presented, which combines local characteristics of the nuclei boundary and a priori knowledge about the expected shape of the nuclei. A deformable model whose behavior is driven by physical principles is trained on images containing a single nuclei, and attributes of the shapes of the nuclei are expressed in terms of modal analysis. Based on the estimated modal distribution and driven by the image characteristics, we develop a framework to detect and describe the unknown nuclei boundaries in images containing two overlapping nuclei. The problem of the estimation of an accurate nucleus boundary in the overlapping areas is successfully addressed with the use of appropriate weight parameters that control the contribution of the image force in the total energy of the deformable model. The proposed method was evaluated using 152 images of conventional Pap smears, each containing two overlapping nuclei. Comparisons with other segmentation methods indicate that our method produces more accurate nuclei boundaries which are closer to the ground truth.

A new Bayesian model for image segmentation based on a Gaussian mixture model is proposed. The model structure allows the automatic determination of the number of segments while ensuring spatial smoothness of the final output. This is achieved by defining two separate mixture weight sets: the first set of weights is spatially variant and incorporates an MRF edge-preserving smoothing prior; the second set of weights is governed by a Dirichlet prior in order to prune unnecessary mixture components. The model is trained using variational inference and the Majorization-Minimization (MM) algorithm, resulting in closed-form parameter updates. The algorithm was successfully evaluated in terms of various segmentation indices using the Berkeley image data base.

Accurate image registration plays a preponderant role in image super-resolution methods and in the related literature landmark based registration methods have gained increasing acceptance in this framework. However, their solution relies on point correspondences and on least squares estimation of the registration parameters necessitating further improvement. In this work, a maximum a posteriori scheme for image super-re solution is presented where the image registration part is accomplished in two steps. At first, the low resolution images are registered by establishing correspondences between robust SIFT features. In the second step, the estimation of the registration parameters is fine-tuned along with the estimation of the high resolution image, in an iterative procedure, using the maximization of the mutual information criterion. Numerical results showed that the reconstructed image is consistently of higher quality than in standard MAP-based methods employing only landmarks.

In this paper, we present a fully automated method for cell nuclei detection in Pap smear images. The locations of the candidate nuclei centroids in the image are detected with morphological analysis and they are refined in a second step, which incorporates a priori knowledge about the circumference of each nucleus. The elimination of the undesirable artifacts is achieved in two steps: the application of a distance-dependent rule on the resulted centroids; and the application of classification algorithms. In our method, we have examined the performance of an unsupervised (fuzzy C-means) and a supervised (support vector machines) classification technique. In both classification techniques, the effect of the refinement step improves the performance of the clustering algorithm. The proposed method was evaluated using 38 cytological images of conventional Pap smears containing 5617 recognized squamous epithelial cells. The results are very promising, even in the case of images with high degree of cell overlapping.

In medical imaging, constructing an atlas and bringing an image set in a single common reference frame may easily lead the analysis to erroneous conclusions, especially when the population under study is heterogeneous. In this paper, we propose a framework based on spectral clustering that is capable of partitioning an image population into sets that require a separate atlas, and identifying the most suitable templates to be used as coordinate reference frames. The spectral analysis step relies on pairwise distances that express anatomical differences between subjects as a function of the diffeomorphic warp required to match the one subject onto the other, plus residual information. The methodology is validated numerically on artificial and medical imaging data.

A new Bayesian model is proposed for image segmentation based upon Gaussian mixture models (GMM) with spatial smoothness constraints. This model exploits the Dirichlet compound multinomial (DCM) probability density to model the mixing proportions (i.e., the probabilities of class labels) and a Gauss-Markov random field (MRF) on the Dirichlet parameters to impose smoothness. The main advantages of this model are two. First, it explicitly models the mixing proportions as probability vectors and simultaneously imposes spatial smoothness. Second, it results in closed form parameter updates using a maximum a posteriori (MAP) expectation-maximization (EM) algorithm. Previous efforts on this problem used models that did not model the mixing proportions explicitly as probability vectors or could not be solved exactly requiring either time consuming Markov Chain Monte Carlo (MCMC) or inexact variational approximation methods. Numerical experiments are presented that demonstrate the superiority of the proposed model for image segmentation compared to other GMM-based approaches. The model is also successfully compared to state of the art image segmentation methods in clustering both natural images and images degraded by noise.

We propose a new approach for the restoration of polarimetric Stokes images, capable of simultaneously segmenting and restoring the images. In order to easily handle the admissibility constraints inherent to Stokes images, a proper transformation of the images is introduced. This transformation exploits the correspondence between any Stokes vector and the covariance matrix of the two components of the electric vector of the light wave. A Bayesian model based on a mixture of Gaussian kernels is used for the transformed images. Inference is achieved using the EM framework. To quantify the performances of this approach, the algorithm is tested with both synthetic and real data. We note that the pixels of the restored Stokes images issued from our approach are always physically admissible which is not the case for the nai¿ve pseudo-inverse approach.

In this study, heartbeat time series are classified using support vector machines (SVMs). Statistical methods and signal analysis techniques are used to extract features from the signals. The SVM classifier is favorably compared to other neural network-based classification approaches by performing leave-one-out cross validation. The performance of the SVM with respect to other state-of-the-art classifiers is also confirmed by the classification of signals presenting very low signal-to-noise ratio. Finally, the influence of the number of features to the classification rate was also investigated for two real datasets. The first dataset consists of long-term ECG recordings of young and elderly healthy subjects. The second dataset consists of long-term ECG recordings of normal subjects and subjects suffering from coronary artery disease.

A new hierarchical Bayesian model is proposed for image segmentation based on Gaussian mixture models (GMM) with a prior enforcing spatial smoothness. According to this prior, the local differences of the contextual mixing proportions (i.e. the probabilities of class labels) are Studentpsilas t-distributed. The generative properties of the Student's t-pdf allow this prior to impose smoothness and simultaneously model the edges between the segments of the image. A maximum a posteriori (MAP) expectation-maximization (EM) based algorithm is used for Bayesian inference. An important feature of this algorithm is that all the parameters are automatically estimated from the data in closed form. Numerical experiments are presented that demonstrate the superiority of the proposed model for image segmentation as compared to standard GMM-based approaches and to GMM segmentation techniques with ldquostandardrdquo spatial smoothness constraints.

We propose a pixel similarity-based algorithm enabling accurate rigid registration between single and multimodal images presenting gross dissimilarities due to noise, missing data or outlying measures. The method relies on the partitioning of a reference image by a Student's t-mixture model (SMM). This partition is then projected onto the image to be registered. The main idea is that a t-component in the reference image corresponds to a t-component in the image to be registered. If the images are correctly registered the weighted sum of distances between the corresponding components is minimized. The use of SMM components is justified by the property that they have heavier tails than standard Gaussians, thus providing robustness to outliers. Experimental results indicate that, even in the case of images presenting low SNR or important amount of dissimilarities due to temporal changes, the proposed algorithm compares favorably to the histogram-based mutual information method that is widely used in a variety of applications.

Gaussian mixture models have been widely used in image segmentation. However, such models are sensitive to outliers. In this paper, we consider a robust model for image segmentation based on mixtures of Student'st-distributions which have heavier tails than Gaussian and thus are not sensitive to outliers. Thet-distribution is one of the few heavy tailed probability density functions (pdf) closely related to the Gaussian, that gives tractable maximum likelihood inference via the Expectation-Maximization (EM) algorithm. Numerical experiments that demonstrate the properties of the proposed model for image segmentation are presented.

In this paper we present a new approach for curve clustering designed for analysis of spatiotemporal data. Such kind of data contains both spatial and temporal patterns that we desire to capture. The proposed methodology is based on regression and Gaussian mixture modeling and the novelty of the herein work is the incorporation of spatial smoothness constraints in the form of a prior for the data labels. This enables the proposed model to take into account the underlying property of spatiotemporal data that spatially adjacent data points most likely should belong to the same cluster. A maximum a posteriori Expectation Maximization (MAP-EM) algorithm is used for learning this model. We present numerical experiments with simulated data where the ground truth is known in order to assess the value of the introduced smoothness constraint, and also with real cardiac perfusion MRI data. The results are very promising and demonstrate the value of the proposed constraint for analysis of such data .

We propose a new approach for image segmentation based on a hierarchical and spatially variant mixture model. According to this model, the pixel labels are random variables and a smoothness prior is imposed on them. The main novelty of this work is a new family of smoothness priors for the label probabilities in spatially variant mixture models. These Gauss-Markov random field-based priors allow all their parameters to be estimated in closed form via the maximum a posteriori (MAP) estimation using the expectation-maximization methodology. Thus, it is possible to introduce priors with multiple parameters that adapt to different aspects of the data. Numerical experiments are presented where the proposed MAP algorithms were tested in various image segmentation scenarios. These experiments demonstrate that the proposed segmentation scheme compares favorably to both standard and previous spatially constrained mixture model-based segmentation.

New methods for detecting shot boundaries in video sequences and for extracting key frames using metrics based on information theory are proposed. The method for shot boundary detection relies on the mutual information (MI) and the joint entropy (JE) between the frames. It can detect cuts, fade-ins and fade-outs. The detection technique was tested on the TRECVID2003 video test set having different types of shots and containing significant object and camera motion inside the shots. It is demonstrated that the method detects both fades and abrupt cuts with high accuracy. The information theory measure provides us with better results because it exploits the inter-frame information in a more compact way than frame subtraction. It was also successfully compared to other methods published in literature. The method for key frame extraction uses MI as well. We show that it captures satisfactorily the visual content of the shot.

A joint probabilistic face detection and tracking algorithm, combining likelihood estimation and a prior probability, is proposed. The likelihood estimation scheme is based on the statistical training of sets of automatically generated feature points and a mutual information tracking cue, while the prior probability estimation is based on a Gaussian temporal model. The likelihood estimation process is the core of a multiple face detection scheme used to initialize the tracking process. The resulting system has been tested on real image sequences and is robust to significant partial occlusion and illumination changes.

An accurate, computationally efficient, and fully automated algorithm for the alignment of two-dimensional (2-D) serially acquired sections forming a three-dimensional (3-D) volume is presented. The approach relies on the optimization of a global energy function, based on the object shape, measuring the similarity between a slice and its neighborhood in the 3-D volume. Slice similarity is computed using the distance transform measure in both directions. No particular direction is privileged in the method avoiding global offsets, biases in the estimation and error propagation. The method was evaluated on real images [medical, biological, and other computerized tomography (CT) scanned 3-D data] and the experimental results demonstrated its accuracy as reconstuction errors are less than one degree in rotation and less than one pixel in translation.

A new method for detecting shot boundaries in video sequences using metrics based on information theory is proposed. The method relies on the mutual information and the joint entropy between frames and can detect cuts, fade-ins and fade-outs. The detection technique was tested on TV video sequences having different types of shots and significant object and camera motion inside the shots. It was favorably compared to other recently proposed shot cut detection techniques. The method is proven to detect both fades and abrupt cuts very effectively.

A joint probabilistic face detection and tracking algorithm for combining a likelihood estimation and a prior probability is proposed. Face tracking is achieved by a Bayesian framework. The likelihood estimation scheme is based on statistical training of sets of automatically generated feature points, while the prior probability estimation is based on the fusion of an information theoretic tracking cue and a Gaussian temporal model. The likelihood estimation process is the cone of a multiple face detection scheme used to initialize the tracking process. The resulting system was tested on real image sequences and is robust to significant partial occlusion and illumination changes.

Metrics based on mutual information and not resorting to ground truth data are proposed in this paper in order to measure tracking reliability under occlusion., The variations of the proposed metrics can be used as a quantitative estimate of changes in the tracking region, caused by occlusion, sudden movement or the deformation of the tracked object. The proposed metric was tested on an object tracking scheme using multiple feature point correspondences. Experimental results have shown that mutual information can effectively characterize object appearance and reappearance in many computer vision applications.

This paper presents an accurate, computationally efficient, fast and fully-automated algorithm for the alignment of 2D serially acquired sections forming a 3D volume. The method accounts for the main shortcomings of 3D image alignment: corrupted data (cuts and tears), dissimilarities or discontinuities between slices and missing slices. The approach relies on the determination of inter-slice correspondences. The features used for correspondence are extracted by a 2D physics-based deformable model parameterizing the object shape. Correspondence affinities and global constraints render the method efficient and reliable. The method has been evaluated on real images and the experimental results demonstrate its accuracy, as reconstruction errors are smaller than 1 degree in rotation and smaller than 1 pixel in translation.

A probabilistic deformable model for the representation of multiple brain structures is described. The statistically learned deformable model represents the relative location of different anatomical surfaces in brain magnetic resonance images (MRIs) and accommodates their significant variability across different individuals. The surfaces of each anatomical structure are parameterized by the amplitudes of the vibration modes of a deformable spherical mesh. For a given MRI in the training set, a vector containing the largest vibration modes describing the different deformable surfaces is created. This random vector is statistically constrained by retaining the most significant variation modes of its Karhunen-Loève expansion on the training population. By these means, the conjunction of surfaces are deformed according to the anatomical variability observed in the training set. Two applications of the joint probabilistic deformable model are presented: isolation of the brain from MRI using the probabilistic constraints embedded in the model and deformable model-based registration of three-dimensional multimodal (magnetic resonance/single photon emission computed tomography) brain images without removing nonbrain structures. The multi-object deformable model may be considered as a first step toward the development of a general purpose probabilistic anatomical atlas of the brain.

This paper addresses the problem of describing the significant intra- and inter-variability of 3D deformable structures within 3D image data sets. In pursuing it, a 3D probabilistic physically based deformable model is defined. The statistically learned deformable model captures the spatial relationships between the different objects surfaces, together with their shape variations. The structures of interest in each volume are parameterized by the amplitudes of the vibration modes of a deformable spherical mesh. For a given 3D image in the training set, a vector containing the largest vibration modes describing the desired object is created. This random vector is statistically constrained by retaining the most significant variation modes of its Karhunen-Loeve (KL) expansion on the considered population. The surfaces of the modeled structures thus deform according to the variability observed in the training set. A preliminary application of a 3D multi-object model for the segmentation of 3D brain structures from MR images is presented.

The registration of multimodal images remains an intricate issue, especially when the multimodal image pair shows non overlapping structures, missing data, noise or outliers. In this paper, we present a deformable model-based technique for the rigid registration of 2D and 3D multimodal images. The deformable model embeds a priori knowledge of the spatial correspondence and statistical variability of the different (eventually non overlapping) image features which are used in the registration procedure. The method is applied to the intrasubject registration of medical MR/SPECT images of the brain by constructing a deformable model incorporating information on both MR (head) and SPECT (brain) contours. In an off-line training procedure, the spatial relations between head and brain as well as the anatomical variations of these two structures are learned. The registration is performed by minimizing an objective function involving the contours of the MR and SPECT images.