Body part regression is a promising new technique that enables content navigation through self-supervised learning. Using this technique, the global quantitative spatial location for each axial view slice is obtained from computed tomography (CT). However, it is challenging to define a unified global coordinate system for body CT scans due to the large variabilities in image resolution, contrasts, sequences, and patient anatomy. Therefore, the widely used supervised learning approach cannot be easily deployed. To address these concerns, we propose an annotation-free method named blind-unsupervised-supervision network (BUSN). The contributions of the work are in four folds: (1) 1030 multi-center CT scans are used in developing BUSN without any manual annotation. (2) the proposed BUSN corrects the predictions from unsupervised learning and uses the corrected results as the new supervision; (3) to improve the consistency of predictions, we propose a novel neighbor message passing (NMP) scheme that is integrated with BUSN as a statistical learning based correction; and (4) we introduce a new pre-processing pipeline with inclusion of the BUSN, which is validated on 3D multi-organ segmentation. The proposed method is trained on 1,030 whole body CT scans (230,650 slices) from five datasets, as well as an independent external validation cohort with 100 scans. From the body part regression results, the proposed BUSN achieved significantly higher median R-squared score (=0.9089) than the state-of-the-art unsupervised method (=0.7153). When introducing BUSN as a preprocessing stage in volumetric segmentation, the proposed pre-processing pipeline using BUSN approach increases the total mean Dice score of the 3D abdominal multi-organ segmentation from 0.7991 to 0.8145.

Multi-exposure X-ray imaging can see through objects and separate different material into transparent layers. However, layer motion makes the separation task under-determined. Instead of aligning the non-rigid motion, we address the layer separation problem in gradient domain and propose an energy optimization framework to regularize it by explicitly enforcing independence constraint. It is shown that gradient domain allows more accurate and robust independence analysis between non-stationary signal using mutual information (MI) and hence achieves better separation. Furthermore, gradient fields contain sufficient information for full reconstruction of separated layers by solving the Poisson Equation. For efficient regularization of the gradient separation, energy terms based on the Taylor expansion of MI is further derived. Evaluation on both synthesized and real datasets proves the effectiveness of our algorithm and its robustness to complex tissue motion.

High-dynamic-range medical images have intensity values which cannot be visualized on current low-dynamic-range displays. In this paper, we introduce a fast range compression method which avoids common artifacts such as loss of contrast, haloes and gradient inversions. The proposed method first compresses the intensity range using a global transfer function. It then extracts and enhances weak structures using a multiscale decomposition and artifact correction. We show that artifact correction can be formulated as a linear-programming problem, for which we propose an efficient approximate solution. Experiments on real data demonstrate the effectiveness and speed of the proposed algorithm.

Dual energy X-ray imaging can separate materials of different properties into different layers for more detailed inspection. However, motion during the acquisition of dual energy images causes significant challenges to layer separation. We propose a novel separation algorithm based on the sparse features in gradient domain. Instead of decomposing the image intensity values, we separate sparse gradient features to different layers and then reconstruct the layers based on the decomposed gradient fields by solving the Poisson Equation. The structure tensor, based on directional quadrature filters, is used to classify and separate the gradient field. Evaluation on both synthesized data and real datasets proves its effectiveness and robustness to complex non-rigid motion.

Traditional Bayesian restoration methods depend heavily on the accuracy of underlying generative models. For the challenging streak noise generated in the procedure of reconstruction from projections, Bayesian methods do not generalize well because accurate signal/noise models are not readily available. In this paper, we reformulate the reconstruction problem into a multi-image based restoration task and demonstrate that multiple images and mutual independence analysis can be utilized to significantly improve the generalization capability of traditional Bayesian frameworks in challenging scenarios. An efficient mutual independence analysis term is designed based on the properties of independent random variables to enforce the independent noise constraint between multiple images in an energy optimization framework, which can effectively detect and correct restoration error due to inaccurate generative models. Quantitative comparisons on phantom image and experiments on clinical scans both show significant improvements in accuracy and robustness of the proposed method.

Image registration for X-ray dual energy imaging is challenging due to the overlaid transparent layers (i.e., the bone and soft tissue) and the different appearances between the dual images acquired with X-rays at different energy spectra. Moreover, subpixel accuracy is necessary for good reconstruction of the bone and soft-tissue layers. This paper addresses these problems with a novel coupled Bayesian framework, in which the registration and reconstruction can effectively reinforce each other. With the reconstruction results, we can design accurate matching criteria for aligning the dual images, instead of treating them as multi-modality registration. Furthermore, prior knowledge of the bone and soft tissue can be exploited to detect poor reconstruction due to inaccurate registration; and hence correct registration errors in the coupled framework. A multiscale freeform registration algorithm is implemented to achieve subpixel registration accuracy. Promising results are obtained in the experiments.

Traditional iterative tomographic reconstruction methods resort to gradient decent methods and require significant computation due to slow convergence. We divide the iterative reconstruction into two stages in the image and Radon spaces, respectively. First, we refine the reconstruction result by image space adaptive filtering. This finds a feasible update direction based on signal modeling. Second, we minimize the discrepancy between the sinograms along the update direction in Radon space and guarantee convergence. Reconstruction from clinical data using the proposed algorithm converges extremely fast and provides satisfactory reconstruction results in far fewer iterations than traditional methods.

Various methods have been proposed for image enhancement and restoration. The main difficulty is how to enhance the structures uniformly while suppressing the noise without artifacts. In this paper, we tackle this problem in the gradient domain instead of the traditional intensity domain. By enhancing the gradient field, we can enhance the structure uniformly without overshooting at the boundary. Because the gradient field is very sensitive to noise, we apply an orientation-isotropy adaptive filter to the gradient field, suppressing the gradients in the noise regions while enhancing along the object boundaries. Thus we obtain a modulated gradient field, which is usually not integrable. We reconstruct the enhanced image from the modulated gradient field with least square errors by solving a Poisson equation. This method can enhance the object contrast uniformly, suppress the noise with no artifacts, and avoid setting stopping time as in PDE methods. Experiments on noisy images show the efficacy of our method.

We propose an HMM model for contour detection based on multiple visual cues in spatial domain and improve it by joint probabilistic matching to reduce background clutter. It is further integrated with unscented Kalman filter to exploit object dynamics in nonlinear systems for robust contour tracking.

Image restoration has been extensively studied in the past. But multi-image based restoration/compounding is still surprisingly primitive. It usually starts with weighted averaging of the multiple images followed by single-image based restoration methods, which discards the abundant information hinted in the multiple images that can help the restoration process. In this paper, we utilize the fact that the images are corrupted by independent noise and design a new independence measurement based on the properties of independent random variables. The new independence measurement can be efficiently evaluated and imposed as an energy term into the traditional maximum a posteriori (MAP) framework, compensating to the generative models of signal and noise. It can effectively prevent the signal from being smoothed out as noise and hence dramatically improve the restoration quality and robustness, especially when accurate noise/signal models are difficult to obtain. Experiments on real medical images show very promising results.

Bayesian methods have been extensively used in various applications. However, there are two intrinsic issues rarely addressed, namely generalization and validity, in the context of multiple image restoration, we show that traditional Bayesian methods are sensitive to model errors and cannot guarantee valid results satisfying the underlying prior knowledge, e.g. independent noise property. To improve the Bayesian framework's generalization, we propose to explicitly enforce the validity of the result. Independent noise prior is very important but largely under-utilized in previous literature. In this paper, we use mutual information (MI) to explicitly enforce the independence. Efficient approximations based on Taylor expansion are proposed to adapt MI into standard energy forms to regularize the Bayesian methods. The new regularized Bayesian framework effectively utilizes the traditional generative signal/noise models but is much more robust to various model errors, as demonstrated in experiments on some demanding imaging applications.

Sensor fusion for object tracking has become an active research direction during the past few years. But how to do it in a robust and principled way is still an open problem. In this paper, we propose a new fusion framework that combines both the bottom-up and top-down approaches to probabilistically fuse multiple sensing modalities. At the lower level, individual vision and audio trackers are designed to generate effective proposals for the fuser. At the higher level, the fuser performs reliable tracking by verifying hypotheses over multiple likelihood models from multiple cues. Unlike traditional fusion algorithms, the proposed framework is a closed-loop system where the fuser and trackers coordinate their tracking information. Furthermore, to handle nonstationary situations, the proposed framework evaluates the performance of the individual trackers and dynamically updates their object states. We present a real-time speaker tracking system based on the proposed framework by fusing object contour, color and sound source location. We report robust tracking results.

We have recently developed a real time automatic access control system suing both face and voice identification. We show that combining these two modalities achieves high recognition accuracy, especially in difficult recognition scenarios. Our system also supports online training of the user models, i.e., a person who has not been registered in the database can be added on the spot in real time. This system is composed of a face and eye corners detection module, a face recognition module, a speaker identification module, and a user interface module. We present the algorithm of each module with more emphasis on our information maximization based face and eye corners detection and user interface design. We discuss some issues related to the integration of components into the total system. We report the experimental results of the system performance together with comparison with those of some other systems.

This paper presents an efficient method to integrate various spatial-temporal constraints to regularize the contour tracking. Specifically, the global shape prior, contour smoothness and object dynamics are addressed. First, the contour is represented as a parametric shape, based on which a causal smoothness constraint can be developed to exploit the local spatial constraint. The causality nature of the constraint allows us to do efficient probabilistic contour detection using the powerful hidden Markov model (HMM). Finally, a unscented Kalman filter (UKF) is applied to estimate object parameters based on the nonlinear observation model (i.e. the relationship between the detected contour points and the contour parameters) and the object dynamics. Better than other variants of the recursive least mean square estimators (e.g., extended Kalman filter), the UKF approximates nonlinear systems up to the second order (third for Gaussian prior) with similar computational cost. This novel tracking algorithm is running in real-time and robust to severe distractions due to the comprehensive spatial-temporal constraints. It is applied to track people in bad illumination and cluttered environments. Promising results are reported.

In this paper an improved face detection method based on our previous information-based maximum discrimination approach is presented that maximizes the discrimination between face and non-face examples in a training set without using color or motion information. A short review of our previous method is given together with a description of a recent improvement of its detection speed. A person identification system has been developed that performs multi-modal person identification in real-time video based on this newly improved face detection method together with speaker identification.

Tracking objects using multiple cues yields more robust results. The well-known hidden Markov model (HMM) provides a powerful framework to incorporate multiple cues by expanding its observation. However, a plain HMM does not capture the inter-correlation between measurements of neighboring states when computing the transition probabilities. This can seriously damage the tracking performance. To overcome this difficulty, we propose a novel HMM framework targeted at contour-based object tracking. A joint probability data association filter (JPDAF) is used to compute the HMM's transition probabilities, taking into account the intercorrelated neighboring measurements. To ensure real-time performance, we have further developed an efficient method to calculate the data association probability via dynamic programming, which allows the proposed JPDAF-HMM to run comfortably at 30 frames/sec. This new tracking framework can easily incorporate various image cues (e.g., edge intensity, foreground region color and background region color), and also offers an online learning process to adapt to changes in the scene. To evaluate its tracking performance, we have applied the proposed JPDAF-HMM in various real-world video sequences. We report promising tracking results in complex environments.

Tracking objects involves the modeling of non-linear non-Gaussian systems. On one hand, variants of Kalman filters are limited by their Gaussian assumptions. On the other hand, conventional particle filter, e.g., CONDENSATION, uses transition prior as the proposal distribution. The transition prior does not take into account current observation data, and many particles can therefore be wasted in low likelihood area. To overcome these difficulties, unscented particle filter (UPF) has recently been proposed in the field of filtering theory. In this paper, we introduce the UPF framework into audio and visual tracking. The UPF uses the unscented Kalman filter to generate sophisticated proposal distributions that seamlessly integrate the current observation, thus greatly improving the tracking performance. To evaluate the efficacy of the UPF framework, we apply it in two real-world tracking applications. One is the audio-based speaker localization, and the other is the vision-based human tracking. The experimental results are compared against those of the widely used CONDENSATION approach and have demonstrated superior tracking performance.

To track multiple objects, top-down (model-based methods) and bottom-up (multi-layer analysis) methods have been proposed separately. A hierarchical MRF model is proposed to integrate these two trends into a MAP framework for tracking non-rigid objects such as human hands or faces. Parametric models of color, shape and frame difference for both foreground and background are given. Dynamic constraints are used to update the observation models and present an initial segmentation for the new frame. A novel hierarchical MRF model is proposed to efficiently refine the segmentation based on local smoothness constraints. The algorithm does not need to initialize and can detect new moving objects and track them. It can also handle the stopped objects because of the utilization of spatial-temporal constraints. Promising results are reported.

A common problem in most active contour methods is that the recursive searching scheme can only return a local optimal solution. Furthermore, the internal energy of the snake is not strong enough to control the shape of the contour. To overcome these difficulties, we develop a causal internal energy term based on a radial contour representation to encode the smooth constraint of the contour, and develop a global shape priori to control contour's shape and position based on object's dynamics. The causality nature of the representation allows us to efficiently find global optimal solution using dynamic programming. To validate the efficacy and robustness of the proposed approach, we apply this approach to track people in bad illumination and cluttered environments. We report promising results in the paper.

Relevance feedback schemes using linear/quadratic estimators have been applied in content-based image retrieval to improve retrieval performance significantly. One major difficulty in relevance feedback is to estimate the support of target images in high dimensional feature space with a relatively small number of training samples. We develop a novel scheme based on one-class SVM, which fits a tight hyper-sphere in the nonlinearly transformed feature space to include most of the target images based on positive examples. The use of a kernel provides us an elegant way to deal with nonlinearity in the distribution of the target images, while the regularization term in SVM provides good generalization ability. To validate the efficacy of the proposed approach, we test it on both synthesized data and real-world images. Promising results are achieved in both cases.

This paper introduces a novel adaptive texture feature selection algorithm for tracking. Specifically, we provide a statistical wavelet basis paradigm to maximally separate statistical characteristics of the object-in-interest and its background. The algorithm is based upon nonlinearly selecting basis elements out of dual dictionaries in an iterative fashion to continually improve a cost function that is suitable for tracking. We demonstrate that such a selection is effective with several difficult sequences that are affected by lighting changes, occlusion and background motion.