The automatic extraction and labeling of the rib centerlines is a useful yet challenging task in many clinical applications. In this paper, we propose a new approach integrating rib seed point detection and template matching to detect and identify each rib in chest CT scans. The bottom-up learning based detection exploits local image cues and top-down deformable template matching imposes global shape constraints. To adapt to the shape deformation of different rib cages whereas maintain high computational efficiency, we employ a Markov Random Field (MRF) based articulated rigid transformation method followed by Active Contour Model (ACM) deformation. Compared with traditional methods that each rib is individually detected, traced and labeled, the new approach is not only much more robust due to prior shape constraints of the whole rib cage, but removes tedious post-processing such as rib pairing and ordering steps because each rib is automatically labeled during the template matching. For experimental validation, we create an annotated database of 112 challenging volumes with ribs of various sizes, shapes, and pathologies such as metastases and fractures. The proposed approach shows orders of magnitude higher detection and labeling accuracy than state-of-the-art solutions and runs about 40 seconds for a complete rib cage on the average.

Lymph nodes are assessed routinely in clinical practice and their size is followed throughout radiation or chemotherapy to monitor the effectiveness of cancer treatment. This paper presents a robust learning-based method for automatic detection and segmentation of solid lymph nodes from CT data, with the following contributions. First, it presents a learning based approach to solid lymph node detection that relies on marginal space learning to achieve great speedup with virtually no loss in accuracy. Second, it presents a computationally efficient segmentation method for solid lymph nodes (LN). Third, it introduces two new sets of features that are effective for LN detection, one that self-aligns to high gradients and another set obtained from the segmentation result. The method is evaluated for axillary LN detection on 131 volumes containing 371 LN, yielding a 83.0% detection rate with 1.0 false positive per volume. It is further evaluated for pelvic and abdominal LN detection on 54 volumes containing 569 LN, yielding a 80.0% detection rate with 3.2 false positives per volume. The running time is 5-20 s per volume for axillary areas and 15-40 s for pelvic. An added benefit of the method is the capability to detect and segment conglomerated lymph nodes.

In this paper, we present an approach to automate the planning of an endovascular stent graft for abdominal aortic aneurysms (AAAs), which are treated with bifurcated prosthesis (Y-stents) when located close to the iliac bifurcation. During the intervention, the folded Y-stent graft — consisting of several parts — is inserted via the iliac region and expanded inside the patient's body. The first step of the proposed approach is to detect different body landmarks with a statistical method. In the next step, these landmarks are used to calculate two vascular centerlines, which provide multiplanar reformatting (MPR) slices that are used for an automatic segmentation of the artery walls. The segmented artery walls provide the manufacturer specific measures to choose an adequate bifurcated prosthesis. In a final step, the expansion of the stent is simulated in the patient's data. Results for 50 abdominal aortic aneurysm cases provided by computed tomography angiography (CTA) acquisitions are successfully verified by a virtual stenting expert.

A fundamental issue in multiple landmark detection is the reduction of computational cost. This problem has previously been addressed mainly by reducing the complexity of each individual landmark detector. We address the problem by optimizing the search strategy of multiple landmarks. When the relative positions of landmarks are constrained, the search space can be reduced, thereby reducing the computation. The proposed method leverages the theory of submodular functions to provide a constant factor approximation guarantee of the optimal speed. Although the theory of submodular functions is well known, to the best of our knowledge, this is the first time it is applied to the landmark detection problem. We demonstrate our method by fast and accurate detection of human body landmarks including bones, organs, and vessels in 3D CT images from a diverse dataset of around 2000 volumes with pathological patients. We further provide different search space criteria and variations.

We propose a novel method for removing irrelevant frames from a video given user-provided frame-level labeling for a very small number of frames. We first hypothesize a number of windows which possibly contain the object of interest, and then determine which window(s) truly contain the object of interest. Our method enjoys several favorable properties. First, compared to approaches where a single descriptor is used to describe a whole frame, each window's feature descriptor has the chance of genuinely describing the object of interest; hence it is less affected by background clutter. Second, by considering the temporal continuity of a video instead of treating frames as independent, we can hypothesize the location of the windows more accurately. Third, by infusing prior knowledge into the patch-level model, we can precisely follow the trajectory of the object of interest. This allows us to largely reduce the number of windows and hence reduce the chance of overfitting the data during learning. We demonstrate the effectiveness of the method by comparing it to several other semi-supervised learning approaches on challenging video clips.

In computer vision, the bag-of-visual words image representation has been shown to yield good results. Recent work has shown that modeling the spatial relationship between visual words further improves performance. Previous work extracts higher-order spatial features exhaustively. However, these spatial features are expensive to compute. We propose a novel method that simultaneously performs feature selection and feature extraction. Higher-order spatial features are progressively extracted based on selected lower order ones, thereby avoiding exhaustive computation. The method can be based on any additive feature selection algorithm such as boosting. Experimental results show that the method is computationally much more efficient than previous approaches, without sacrificing accuracy.

We propose an iterative method for discovering objects in images. In each iteration, the current estimate of the layout is processed by a sequence of perceptual grouping rules. Perceptual grouping appears to be the basis of visual organization of human. It is concerned with the problem of the formation of wholes from parts. The method does not rely on the mixture of Gaussian model and hence avoids the model selection problem. We use synthetic and real images to demonstrate that the obtained result is better than that obtained by other methods.

This paper presents a probabilistic framework for discovering objects in video. The video can switch between different shots, the unknown objects can leave or enter the scene at multiple times, and the background can be cluttered. The framework consists of an appearance model and a motion model. The appearance model exploits the consistency of object parts in appearance across frames. We use maximally stable extremal regions as observations in the model and hence provide robustness to object variations in scale, lighting and viewpoint. The appearance model provides location and scale estimates of the unknown objects through a compact probabilistic representation. The compact representation contains knowledge of the scene at the object level, thus allowing us to augment it with motion information using a motion model. This framework can be applied to a wide range of different videos and object types, and provides a basis for higher level video content analysis tasks. We present applications of video object discovery to video content analysis problems such as video segmentation and threading, and demonstrate superior performance to methods that exploit global image statistics and frequent itemset data mining techniques.

The bag-of-words representation has attracted a lot of attention recently in the field of object recognition. Based on the bag-of-words representation, topic models such as probabilistic latent semantic analysis (PLSA) have been applied to unsupervised object discovery in still images. In this paper, we extend topic models from still images to motion videos with the integration of a temporal model. We propose a novel spatial-temporal framework that uses topic models for appearance modeling, and the probabilistic data association (PDA) filter for motion modeling. The spatial and temporal models are tightly integrated so that motion ambiguities can be resolved by appearance, and appearance ambiguities can be resolved by motion. We show promising results that cannot be achieved by appearance or motion modeling alone.

Topic models from the text understanding literature have shown promising results in unsupervised image categorization and object localization. Categories are treated as topics, and words are formed by vector quantizing local descriptors of image patches. Limitations of topic models include their weakness in localizing objects, and the requirement of a fairly large proportion of words coming from the object. We present a new approach that employs correspondences between images to provide information about object configuration, which in turn enhances the reliability of object localization and categorization. This approach is efficient, as it requires only a small number of correspondences. We show improved categorization and localization performance on real and synthetic data. Moreover, we can push the limits of topic models when the proportion of words coming from the object is very low.

We present a novel approach to background cutout for image editing. We show how background cutout can be achieved without any user labeling. This is in contrast to current methods, where the user needs to label each image separately. Our method uses automatic object discovery methods to provide location and scale estimates of the object of interest; these estimates then provide seeds for initializing color distributions of a segmentation algorithm. We show that our approach can achieve similar performance as traditional methods that require users to specify for each image a bounding box of the target object.

In this paper, we propose a general object detection framework which combines the Hidden Markov Model with the Discriminative Random Fields. Recent object detection alrithms have achieved impressive results by using graphical models, such as Markov Random Field. These models, however, have only been applied to two dimensional images. In many scenarios, video is the directly available source rather than images, hence an important information for detecting objects has been omitted — the temporal information. To demonstrate the importance of temporal information, we apply graphical models to the task of text detection in video and compare the result of with and without temporal information. We also show the superiority of the proposed models over simple heuristics such as median filter over time.

We propose a novel unsupervised learning algorithm to extract the layout of an image by learning latent object-related aspects. Unlike traditional image segmentation algorithms that segment an image using feature similarity, our method is able to learn high-level object characteristics (aspects) from a large number of unlabeled images containing similar objects to facilitate image segmentation. Our method does not require human to annotate the training set and works without supervision. We use a graphical model to address the learning of aspects and layout extraction together. In particular, aspect-feature dependency from multiple images is learned via the expectation-maximization algorithm. We demonstrate that, by associating latent aspects to spatial structure, the proposed method achieves much better layout extraction results than using probabilistic latent semantic analysis.

Content-free image retrieval uses accumulated user feedback records to retrieve images without analyzing image pixels. We present a Bayesian-based algorithm to analyze user feedback and show that it outperforms a recent maximum entropy content-free algorithm, according to extensive experiments on trademark logo and 3D model datasets. The proposed algorithm also has the advantage of being applicable to both content-free and traditional content-based image retrieval, thus providing a common framework for these two paradigms

For information retrieval, relevance feedback is an important technique. This paper proposes a relevance feedback technique which is based on a probabilistic framework. The binary feature vectors in our experiment are high-level semantic features of trademark logo images, each feature representing the presence or absence of a certain shape or object. The images were labeled by human experts of the trademark office. We compared our probabilistic method with several existing methods such as MARS, MindReader, and one-class SVM. Our method outperformed the others.

This paper introduces a shape descriptor, the soft shape context, motivated by the shape context method. Unlike the original shape context method, where each image point was hard assigned into a single histogram bin, we instead allow each image point to contribute to multiple bins, hence more robust to distortions. The soft shape context can easily be integrated into the iterative closest point (ICP) method as an auxiliary feature vector, enriching the representation of an image point from spatial information only, to spatial and shape information. This yields a registration method more robust than the original ICP method. The method is general for 2D shapes. It does not calculate derivatives, hence being able to handle shapes with junctions and discontinuities. We present experimental results to demonstrate the robustness compared with the standard ICP method.