In image-guided cardiac interventions, X-ray imaging and intravascular ultrasound (IVUS) imaging are two often used modalities. Interventional X-ray images, including angiography and fluoroscopy, are used to assess the lumen of the coronary arteries and to monitor devices in real time. IVUS provides rich intravascular information, such as vessel wall composition, plaque, and stent expansions, but lacks spatial orientations. Since the two imaging modalities are complementary to each other, it is highly desirable to co-register the two modalities to provide a comprehensive picture of the coronaries for interventional cardiologists. In this paper, we present a solution for co-registering 2-D angiography and IVUS through image-based device tracking. The presented framework includes learning-based vessel detection and device detections, model-based tracking, and geodesic distance-based registration. The system first interactively detects the coronary branch under investigation in a reference angiography image. During the pullback of the IVUS transducers, the system acquires both ECG-triggered fluoroscopy and IVUS images, and automatically tracks the position of the medical devices in fluoroscopy. The localization of tracked IVUS transducers and guiding catheter tips is used to associate an IVUS imaging plane to a corresponding location on the vessel branch under investigation. The presented image-based solution can be conveniently integrated into existing cardiology workflow. The system is validated with a set of clinical cases, and achieves good accuracy and robustness.

Catheter tracking has become more and more important in recent interventional applications. It provides real time navigation for the physicians and can be used to control a motion compensated fluoro overlay reference image for other means of guidance, e.g. involving a 3D anatomical model. Tracking the coronary sinus (CS) catheter is effective to compensate respiratory and cardiac motion for 3D overlay navigation to assist positioning the ablation catheter in Atrial Fibrillation (Afib) treatments. During interventions, the CS catheter performs rapid motion and non-rigid deformation due to the beating heart and respiration. In this paper, we model the CS catheter as a set of electrodes. Novelly designed hypotheses generated by a number of learning-based detectors are fused. Robust hypothesis matching through a Bayesian framework is then used to select the best hypothesis for each frame. As a result, our tracking method achieves very high robustness against challenging scenarios such as low SNR, occlusion, foreshortening, non-rigid deformation, as well as the catheter moving in and out of ROI. Quantitative evaluation has been conducted on a database of 13221 frames from 1073 sequences. Our approach obtains 0.50mm median error and 0.76mm mean error. 97.8% of evaluated data have errors less than 2.00mm. The speed of our tracking algorithm reaches 5 frames-per-second on most data sets. Our approach is not limited to the catheters inside the CS but can be extended to track other types of catheters, such as ablation catheters or circumferential mapping catheters.

A guidewire is a medical device inserted into vessels during image guided interventions for balloon inflation. During interventions, the guidewire undergoes non-rigid deformation due to patients' breathing and cardiac motions, and such 3D motions are complicated when being projected onto the 2D fluoroscopy. Furthermore, in fluoroscopy there exist severe image artifacts and other wire-like structures. All these make robust guidewire tracking challenging. To address these challenges, this paper presents a probabilistic framework for robust guidewire tracking. We first introduce a semantic guidewire model that contains three parts, including a catheter tip, a guidewire tip and a guidewire body. Measurements of different parts are integrated into a Bayesian framework as measurements of a whole guidewire for robust guidewire tracking. Moreover, for each part, two types of measurements, one from learning-based detectors and the other from online appearance models, are applied and combined. A hierarchical and multi-resolution tracking scheme is then developed based on kernel-based measurement smoothing to track guidewires effectively and efficiently in a coarse-to-fine manner. The presented framework has been validated on a test set of 47 sequences, and achieves a mean tracking error of less than 2 pixels. This demonstrates the great potential of our method for clinical applications.

Significant appearance changes of objects under different orientations could cause loss of tracking, "drifting." In this paper, we present a collaborative tracking framework to robustly track faces under large pose and expression changes and to learn their appearance models online. The collaborative tracking framework probabilistically combines measurements from an offline-trained generic face model with measurements from online-learned specific face appearance models in a dynamic Bayesian network. In this framework, generic face models provide the knowledge of the whole face class, while specific face models provide information on individual faces being tracked. Their combination, therefore, provides robust measurements for multiview face tracking. We introduce a mixture of probabilistic principal component analysis (MPPCA) model to represent the appearance of a specific face under multiple views, and we also present an online EM algorithm to incrementally update the MPPCA model using tracking results. Experimental results demonstrate that the collaborative tracking and online learning methods can handle large pose changes and are robust to distractions from the background.

Most of current computer-based facial expression analysis methods focus on the recognition of perfectly posed expressions, and hence are incapable of handling the individuals with expression impairments. In particular, patients with schizophrenia usually have impaired expressions in the form of "flat" or "inappropriate" affects, which make the quantification of their facial expressions a challenging problem. This paper presents methods to quantify the group differences between patients with schizophrenia and healthy controls, by extracting specialized features and analyzing group differences on a feature manifold. The features include 2D and 3D geometric features, and the moment invariants combining both 3D geometry and 2D textures. Facial expression recognition experiments on actors demonstrate that our combined features can better characterize facial expressions than either 2D geometric or texture features. The features are then embedded into an ISOMAP manifold to quantify the group differences between controls and patients. Experiments show that our results are strongly supported by the human rating results and clinical findings, thus providing a framework that is able to quantify the abnormality in patients with schizophrenia.

Although it is usually assumed in many pattern recognition problems that different patterns are distinguishable, some patterns may have inseparable overlap. For example, some facial expressions involve subtle muscle movements, and are difficult to separate from other expressions or neutral faces. In this paper, we consider such overlapped patterns as "clusters", and present a novel method to quantify cluster overlap based on the Bayes error estimation on manifolds. Our method first applies a manifold learning method, ISOMAP, to discover the intrinsic structure of data, and then measures the overlap of different clusters using the k-NN Bayes error estimation on the learned manifolds. Due to the ISOMAP"s capability of preserving geodesic distances and k-NN's localized estimation, the method can provide an accurate measure of the overlap between clusters, as demonstrated by our simulation experiments. The method is further applied for an analysis of a specific type of facial expression impairment in schizophrenia, i.e.,"flat effect", which refers to a severe reduction in emotional expressiveness. In this study, we capture facial expressions of individuals, and quantify their expression flatness by estimating overlap between different facial expressions. The experimental results show that the patient group has much larger facial expression overlap than the control group, and demonstrate that the flat affect is an important symptom in diagnosing schizophrenia patients.

We propose a novel framework for tissue abnormality characterization in normal appearing brain tissue (NABT) that is progressively deteriorating, using affinity propagation applied to multi-parametric data created using a combination of magnetic resonance (MR) protocols. While traditional tissue segmentation and clustering can reveal clusters pertaining to healthy and diseased tissue easily, a complete characterization of the effect of pathology requires the study of heterogeneity of NABT. The problem is rendered challenging by the fact that there are no training samples available for such tissue and hence classification based techniques cannot be used and neither can traditional clustering techniques since the number of clusters are not known a priori. Our framework for the automated clustering of tissue types employs a combination of a) manifold learning, that determines the underlying non-linear structure and embeds it into a lower dimensional space and b) affinity propagation (AP), which is a novel clustering technique that combines model- and similarity- based clustering, to automatically obtain exemplar-based clustering. We also define a novel probabilistic clustering technique. The number of clusters associated with a tissue type is indicative of its heterogeneity. By computing the overlap of these clusters in each of the MR protocols, we obtain a measure of the degree of abnormality in a tissue type and the protocol most sensitive in providing that classification. This general framework is applied towards the characterization of NABT in patients with multiple sclerosis. Results demonstrate a greater heterogeneity in NABT surrounding the lesions along with a greater overlap between the NABT and lesion tissue.

This paper presents methods of modeling and predicting face recognition (FR) system performance based on analysis of similarity scores. We define the performance of an FR system as its recognition accuracy, and consider the intrinsic and extrinsic factors affecting its performance. The intrinsic factors of an FR system include the gallery images, the FR algorithm, and the tuning parameters. The extrinsic factors include mainly query image conditions. For performance modeling, we propose the concept of "perfect recognition," based on which a performance metric is extracted from perfect recognition similarity scores (PRSS) to relate the performance of an FR system to its intrinsic factors. The PRSS performance metric allows tuning FR algorithm parameters offline for near optimal performance. In addition, the performance metric extracted from query images is used to adjust face alignment parameters online for improved performance. For online prediction of the performance of an FR system on query images, features are extracted from the actual recognition similarity scores and their corresponding PRSS. Using such features, we can predict online if an individual query image can be correctly matched by the FR system, based on which we can reduce the incorrect match rates. Experimental results demonstrate that the performance of an FR system can be significantly improved using the presented methods.

It is a challenging task to accurately model the performance of a face recognition system, and to predict its individual recognition results under various environments. This paper presents generic methods to model and predict the face recognition performance based on analysis of similarity measurement. We first introduce a concept of "perfect recognition", which only depends on the intrinsic structure of a recognition system. A metric extracted from perfect recognition similarity scores (PRSS) allows modeling the face recognition performance without empirical testing. This paper also presents an EM algorithm to predict the recognition rate of a query set. Furthermore, features are extracted from similarity scores to predict recognition results of individual queries. The presented methods can select algorithm parameters offline, predict recognition performance online, and adjust face alignment online for better recognition. Experimental results show that the performance of recognition systems can be greatly improved using presented methods.

In current face detection, mostly often used features are selected from a large set (e.g. Haar wavelets). Generally Haar wavelets only represent the local geometric feature. When applying those features to profile faces and eyes with irregular geometric patterns, the classifier accuracy is low in the later training stages, only near 50%. In this paper, instead of brute-force searching the large feature set, we propose to statistically learn the discriminant features for object detection. Besides applying Fisher discriminant analysis(FDA) in AdaBoost, we further propose the recursive nonparametric discriminant analysis (RNDA) to handle more general cases. Those discriminant analysis features are not constrained with geometric shape and can provide better accuracy. The compact size of feature set allows to select a near optimal subset of features and construct the probabilistic classifiers by greedy searching. The proposed methods are applied to multi-view face and eye detection and achieve good accuracy.