Utilizing the idea of long-term cumulative return, reinforcement learning (RL) has shown remarkable performance in various fields. We follow the formulation of landmark localization in 3D medical images as an RL problem. Whereas value-based methods have been widely used to solve RL-based localization problems, we adopt an actor-critic based direct policy search method framed in a temporal difference learning approach. In RL problems with large state and/or action spaces, learning the optimal behavior is challenging and requires many trials. To improve the learning, we introduce a partial policy-based reinforcement learning to enable solving the large problem of localization by learning the optimal policy on smaller partial domains. Independent actors efficiently learn the corresponding partial policies, each utilizing their own independent critic. The proposed policy reconstruction from the partial policies ensures a robust and efficient localization, where the sub-agents uniformly contribute to the state-transitions based on their simple partial policies mapping to binary actions. Experiments with three different localization problems in 3D CT and MR images showed that the proposed reinforcement learning requires a significantly smaller number of trials to learn the optimal behavior compared to the original behavior learning scheme in RL. It also ensures a satisfactory performance when trained on fewer images.

We propose a framework for localization and classification of masses in breast ultrasound images. We have experimentally found that training convolutional neural network-based mass detectors with large, weakly annotated datasets presents a non-trivial problem, while overfitting may occur with those trained with small, strongly annotated datasets. To overcome these problems, we use a weakly annotated dataset together with a smaller strongly annotated dataset in a hybrid manner. We propose a systematic weakly and semi-supervised training scenario with appropriate training loss selection. Experimental results show that the proposed method can successfully localize and classify masses with less annotation effort. The results trained with only 10 strongly annotated images along with weakly annotated images were comparable to results trained from 800 strongly annotated images, with the 95% confidence interval (CI) of difference -3%-5%, in terms of the correct localization (CorLoc) measure, which is the ratio of images with intersection over union with ground truth higher than 0.5. With the same number of strongly annotated images, additional weakly annotated images can be incorporated to give a 4.5% point increase in CorLoc, from 80% to 84.50% (with 95% CIs 76%-83.75% and 81%-88%). The effects of different algorithmic details and varied amount of data are presented through ablative analysis.

We present a method to extract coronary vessels from fluoroscopic x-ray sequences. Given the vessel structure for the source frame, vessel correspondence candidates in the subsequent frame are generated by a novel hierarchical search scheme to overcome the aperture problem. Optimal correspondences are determined within a Markov random field optimization framework. Post-processing is performed to extract vessel branches newly visible due to the inflow of contrast agent. Quantitative and qualitative evaluation conducted on a dataset of 18 sequences demonstrate the effectiveness of the proposed method.

The availability of accurate depth cameras have made real-time human pose estimation possible; however, there are still demands for faster algorithms on low power processors. This paper introduces 1000 frames per second pose estimation method on a single core CPU. A large computation gain is achieved by random walk sub-sampling. Instead of training trees for pixel-wise classification, a regression tree is trained to estimate the probability distribution to the direction toward the particular joint, relative to the current position. At test time, the direction for the random walk is randomly chosen from a set of representative directions. The new position is found by a constant step toward the direction, and the distribution for next direction is found at the new position. The continual random walk through 3D space will eventually produce an expectation of step positions, which we estimate as the joint position. A regression tree is built separately for each joint. The number of random walk steps can be assigned for each joint so that the computation time is consistent regardless of the size of body segmentation. The experiments show that even with large computation gain, the accuracy is higher or comparable to the state-of-the-art pose estimation methods.

We present a new energy model for optical flow estimation on discrete MRF framework. The proposed model yields discrete analog to the prevailing model with diffusion tensor-based regularizer, which has been optimized by variational approach. Inspired from the fact that the regularization process works as a convolution kernel filtering, we formulate the difference between original flow and filtered flow as a smoothness prior. Then the discrete framework enables us to employ a robust penalizer less concerning convexity and differentiability of the energy function. In addition, we provide a new kernel design based on the bilateral filter, adaptively controlling intensity variance according to the local statistics. The proposed kernel simultaneously addresses over-segmentation and over-smoothing problems, which is hard to achieve by tuning parameters. Involving a complex graph structure with large label sets, this work also presents a strategy to efficiently reduce memory requirement and computational time to a tolerable state. Experimental result shows the proposed method yields plausible results on the various data sets including large displacement and textured region.

This paper proposes a new rolled fingerprint construction approach incorporating a state-of-the-art nonrigid image registration method based upon a Markov random field (MRF) energy model. The proposed method finds dense correspondences between images from a rolled fingerprint sequence and warps the entire fingerprint area to synthesize a rolled fingerprint. This method can generate conceptually more accurate rolled fingerprints by preserving the geometric properties of the finger surface as opposed to ink-based rolled impressions and other existing rolled fingerprint construction methods. To verify the accuracy of the proposed method, various comparative experiments were designed to reveal differences among the rolled construction methods. The results show that the proposed method is significantly superior in various aspects compared to previous approaches.

In this paper, we propose a fully automatic method to segment bone compartments in magnetic resonance (MR) images of knee joints gathered from a public database for research on knee osteoarthritis (OA), the osteoarthritis initiative (OAI). Considering the fixed scanning parameters which include position and flexion of the knee joint, the proposed method efficiently utilizes both shape and intensity priors obtained from pre-segmented data, and iteratively applies branch-and-mincut to a local subset of configurations of shape templates. More specifically, at each iteration, the optimal among a subset of the whole range in translation, rotation, and scale parameters are decomposed and separately computed, and motion is greedily selected by the lowest energy. Experimental results demonstrate the increased accuracy and efficiency compared to when only shape priors are applied and when branch-and-mincut is applied to the whole range of parameters at once, respectively.

This paper presents an efficient feature-based nonrigid registration method for multiphase liver CT volumes. While radiologists routinely examine multiphase liver CT to detect hepatic diseases, they usually search corresponding points between 3D CT volumes by visual inspections using 2D slice images. As the liver is a deformable organ, there exist complex nonrigid transformations between liver CT volumes obtained at difference time points (phases). We introduce a fully automatic registration application for multiphase liver CT volumes. For two given liver CT volumes, we extract 3D features with their descriptors, and estimate correspondences by finding nearest neighbor in descriptor space. An energy function is constructed using the correspondence information and the smoothness measure of free-form deformation model based on B-splines. We integrate an approximated smoothness energy function and a robust correspondence energy estimator controlled by the confidence radius of the matching distance in this energy model. The energy function is optimized by sequentially reducing the confidence radius, and outlier correspondences are discarded systematically during convergence. We propose a highly efficient optimization procedure using the preconditioned nonlinear conjugate gradient method. In the experiments, we will provide quantitative and qualitative results on synthetic and clinical data sets. 1

An efficient registration algorithm working on non-rigid 3D objects is presented. We formulate the registration as a discrete labeling problem on MRFs model whose energy can be minimized by optimization techniques in the literature. Due to the huge search range in three-dimensional space, previous approaches produces a vast amount of labels for a node in the MRFs graph. To reduce the number of labels, we decompose a node into three nodes so that the labels in each node represent just one-dimensional displacement. This procedure introduces a factor node with a clique potential of size three, defining ternary interaction between the decomposed nodes. We convert the factor node into pairwise interactions and adopt the tree-reweighted message passing technique, which guarantees the convergence of lower bound of the energy function. In experiments we use clinical and synthetically deformed 3D medical images. Result shows the proposed method enhances computational efficiency without loss of accuracy. 1

This paper presents a robust warping method for minutiae based fingerprint matching approaches. In this method, a deformable fingerprint surface is described using a triangular mesh model. For given two extracted minutiae sets and their correspondences, the proposed method constructs an energy function using a robust correspondence energy estimator and smoothness measuring of the mesh model. We obtain a convergent deformation pattern using an efficient gradient based energy optimization method. This energy optimization approach deals successfully with deformation errors caused by outliers, which are more difficult problems for the thin-plate spline (TPS) model. The proposed method is fast and the run-time performance is comparable with the method based on the TPS model. In the experiments, we provide a visual inspection of warping results on given correspondences and quantitative results using database.

In virtual colonography, precise segmentation is essential for accurate diagnosis. For the segmentation of colon wall, we propose a novel multi-phase deformable model using a level set method. By defining an asymmetric energy functional, the proposed model can simultaneously segment regions with different characteristics. Compared with the conventional multi-phase models, it shows better convergence without ambiguity. Experimental results with real CT images demonstrate that the proposed algorithm outperforms other methods.

In this paper, to represent 2-D shape as a relational structure, i.e. graph, we propose a new shape decomposition scheme composed of two stages: first, a given shape is decomposed into meaningful parts by using the constrained morphological decomposition (CMD) in a recursive manner. More specifically, the CMD adopts the use of the opening operation with the ball-shaped structuring element and the weighted convexity to select the optimal decomposition. Second, the iterative merging stage provides a compact graph-based representation based on the weighted convexity difference. From the experimental results for various and modified 2-D shapes, it is believed that the graph-based representation for 2-D shape coincides with that based on human insight, and also provides robustness to scaling, rotation, noise, shape deformation and occlusion.

Nowadays, the demand on user-friendly querying interface such as query-by-sketch and query-by-editing is an important issue in the content-based retrieval system for 3-D object database. Especially in MPEG-7, P3DS (Perceptual 3-D Shape) descriptor has been developed in order to provide the user-friendly querying, which can not be covered by an existing international standard for description and browsing of 3-D object database. Since the P3DS descriptor is based on the part-based representation of 3-D object, it is a kind of attributed relational gra (ARG) so that the ARG matching algorithm naturally follows as the core procedure for the similarity matching of the P3DS descriptor. In this paper, given a P3DS database from the corresponding 3-D object database, we bring focus into investigating the pros and cons of the target ARG matching algorithms. In order to demonstrate the objective evidence of our conclusion, we have conducted the experiments based on the database of 480 3-D objects with 33 categories in terms of the bull's eye performance, average normalized modified retrieval rate, and precision/recall curve.

Recently, the 3-D object retrieval system attempts to integrate 2-D visual descriptors since some applications including e-shopping cannot employ 3-D object as a query. For an example, the multiple views have been developed in MPEG-7, but it cannot consider the spatial distribution of 3-D object and has heavy computational complexity because of the matching procedure in a combinatorial manner. In this paper, a novel descriptor for the 3-D object retrieval system, the regular polyhedral descriptor or regular polyhedron, is proposed. The proposed descriptor, which consists of the selection of equally distributed view positions and matching strategy using the regular polyhedron, is based on the multiple view, but it can improve the multiple view by using the property of the regular polyhedron. Experimental results show that the proposed descriptor has the retrieval performance comparable to the multiple views, in terms of the bull's eye performance (BEP) and average normalized modified retrieval rate (ANMRR).

We propose an algorithm to construct a 3-D surface model from a set of range data, based on non-uniform rational B-splines (NURBS) surface-fitting technique. It is assumed that the range data is initially unorganized and scattered 3-D points, while their connectivity is also unknown. The proposed algorithm consists of three stages: initial model approximation employing K-means clustering, hierarchical decomposition of the initial model, and construction of the NURBS surface patch network. The initial model is approximated by both a polyhedral and triangular model. Then, the initial model is represented by a hierarchical graph, which is efficiently used to construct the G/sup 1/ continuous NURBS patch network of the whole object. Experiments are carried out on synthetic and real range data to evaluate the performance of the proposed algorithm. It is shown that the initial model as well as the NURBS patch network are constructed automatically with tolerable computation. The modeling error of the NURBS model is reduced to 10%, compared with the initial mesh model.

In this paper, we consider the problem of recognizing 3D object from a single 2D intensity image obtained from unknown position and orientation. We propose the feature set based correspondence algorithm between 3-D model features and 2-D image features, in contrast to the conventional approaches which use a single local feature or fixed number of local features. As a measure of the similarity between feature sets, the Hausdorff distance with explicit paring (HDEP) is proposed and extended to the partial HDEP, using the notion of the partial distance to cope with the problems which occur when there are backgrounds and some of image features are missing or severely deviated from each original value. The 3D object recognition system using this algorithm with hypothesis-verification scheme is implemented and tested on real images with backgrounds and occlusion.

A 3-D shape recovery algorithm, which is based on the hybrid reflectance model and M ring shaped lights for specular surfaces, is presented. We derive a radiance L/sub r/ as a function of slant angle /spl theta//sub n/ of a surface point in polar coordinate. Then, by using the input intensity images and the radiance function, the M slant angle at each point are obtained. The final slant angle is determined as a weighted sum of the M slant angles. Finally, the surface gradient, which is obtained from the slant angle and its contours, is integrated to the depth map. Examples are provided using both synthetic and real data.>

A fixed-point-error analysis for several 1D fast DCT algorithms is presented. For comparison, a direct-form approach is also included in the investigation. A statistical model is used as the basis for predicting the fixed-point error in implementing the algorithms, and a suitable scaling scheme is selected to avoid overflow. Closed-form expressions for both the mean and variance for fixed-point error are derived and compared with experimental results. Simulation results show close agreement between theory and experiment, validating the analysis. The results show that one of the algorithms is better than others in terms of average SNR performance. Based on the 1D analysis, attempts are made to investigate the fixed-point-error analysis of the two-column approach for 2D DCT. It is found that the fixed-point-error characteristics of the row-column approach for 2D DCTs is very similar to that of their 1D counterparts.>

Recently, a novel fast algorithm for 2-D N*N DCT (discrete cosine transform), where N=2/sup m/, was proposed. Only half the number of multiplications required for the conventional row-column approach are needed. However, the relationship between the input-output indices for the postaddition stage in the algorithm is seemingly very irregular. In the present work, the authors derive general and systematic expressions for the relation of the postaddition stage in the 2-D DCT algorithm by representing it in matrix form and developing a method for partitioning the matrices. The results show that the signal flow graph from input to output has a recursive structure where the structure for smaller N appears recursively for larger N. Hence, one can obtain an organized and regular structure for the input-output relation in the postaddition stage.>