Robust and fast detection of anatomical structures is a prerequisite for both diagnostic and interventional medical image analysis. Current solutions for anatomy detection are typically based on machine learning techniques that exploit large annotated image databases in order to learn the appearance of the captured anatomy. These solutions are subject to several limitations, including the use of suboptimal feature engineering techniques and most importantly the use of computationally suboptimal search-schemes for anatomy detection. To address these issues, we propose a method that follows a new paradigm by reformulating the detection problem as a behavior learning task for an artificial agent. We couple the modeling of the anatomy appearance and the object search in a unified behavioral framework, using the capabilities of deep reinforcement learning and multi-scale image analysis. In other words, an artificial agent is trained not only to distinguish the target anatomical object from the rest of the body but also how to find the object by learning and following an optimal navigation path to the target object in the imaged volumetric space. We evaluated our approach on 1487 3D-CT volumes from 532 patients, totaling over 500,000 image slices and show that it significantly outperforms state-of-the-art solutions on detecting several anatomical structures with no failed cases from a clinical acceptance perspective, while also achieving a 20-30 percent higher detection accuracy. Most importantly, we improve the detection-speed of the reference methods by 2-3 orders of magnitude, achieving unmatched real-time performance on large 3D-CT scans.

In X-ray fluoroscopy, static overlays are used to visualize soft tissue. We propose a system for cardiac and respiratory motion compensation of these overlays. It consists of a 3-D motion model created from real-time magnetic resonance (MR) imaging. Multiple sagittal slices are acquired and retrospectively stacked to consistent 3-D volumes. Slice stacking considers cardiac information derived from the ECG and respiratory information extracted from the images. Additionally, temporal smoothness of the stacking is enhanced. Motion is estimated from the MR volumes using deformable 3-D/3-D registration. The motion model itself is a linear direct correspondence model using the same surrogate signals as slice stacking. In X-ray fluoroscopy, only the surrogate signals need to be extracted to apply the motion model and animate the overlay in real time. For evaluation, points are manually annotated in oblique MR slices and in contrast-enhanced X-ray images. The 2-D Euclidean distance of these points is reduced from 3.85 to 2.75 mm in MR and from 3.0 to 1.8 mm in X-ray compared with the static baseline. Furthermore, the motion-compensated overlays are shown qualitatively as images and videos.

This brief proposes a truncated distance (TL1) kernel, which results in a classifier that is nonlinear in the global region but is linear in each subregion. With this kernel, the subregion structure can be trained using all the training data and local linear classifiers can be established simultaneously. The TL1 kernel has good adaptiveness to nonlinearity and is suitable for problems which require different nonlinearities in different areas. Though the TL1 kernel is not positive semidefinite, some classical kernel learning methods are still applicable which means that the TL1 kernel can be directly used in standard toolboxes by replacing the kernel evaluation. In numerical experiments, the TL1 kernel with a pregiven parameter achieves similar or better performance than the radial basis function kernel with the parameter tuned by cross validation, implying the TL1 kernel a promising nonlinear kernel for classification tasks.

Respiratory signals are required for image gating and motion compensation in minimally invasive interventions. In X-ray fluoroscopy, extraction of a respiratory signal can be challenging due to characteristics of interventional imaging, in particular injection of contrast agent and automatic exposure control. We present a novel method for respiratory signal extraction based on dimensionality reduction that can tolerate these events. Images are divided into patches of multiple sizes. Low-dimensional embeddings are generated for each patch using illumination-invariant kernel PCA. Patches with respiratory information are selected automatically by agglomerative clustering. The signals from this respiratory cluster are combined robustly to a single respiratory signal. In the experiments, we evaluate our method on a variety of scenarios. If the diaphragm is visible, we track its superior-inferior motion as ground truth. Our method has a correlation coefficient of more than 91% with the ground truth irrespective of whether or not contrast agent injection or automatic exposure control occur. Additionally, we show that very similar signals are estimated from biplane sequences and from sequences without visible diaphragm. Since all these cases are handled automatically, the method is robust enough to be considered for use in a clinical setting.

In image-guided interventional procedures, live 2-D X-ray images can be augmented with preoperative 3-D computed tomography or MRI images to provide planning landmarks and enhanced spatial perception. An accurate alignment between the 3-D and 2-D images is a prerequisite for fusion applications. This paper presents a dynamic rigid 2-D/3-D registration framework, which measures the local 3-D-to-2-D misalignment and efficiently constrains the update of both planar and non-planar 3-D rigid transformations using a novel point-to-plane correspondence model. In the simulation evaluation, the proposed method achieved a mean 3-D accuracy of 0.07 mm for the head phantom and 0.05 mm for the thorax phantom using single-view X-ray images. In the evaluation on dynamic motion compensation, our method significantly increases the accuracy comparing with the baseline method. The proposed method is also evaluated on a publicly-available clinical angiogram data set with "gold-standard" registrations. The proposed method achieved a mean 3-D accuracy below 0.8 mm and a mean 2-D accuracy below 0.3 mm using single-view X-ray images. It outperformed the state-of-the-art methods in both accuracy and robustness in single-view registration. The proposed method is intuitive, generic, and suitable for both initial and dynamic registration scenarios.

Obstacle fusion algorithms usually perform obstacle association and gating in order to improve the obstacle position if it was detected by multiple sensors. However, this strategy is not common in multi sensor occupancy grid fusion. Thus, the quality of the fused grid, in terms of obstacle position accuracy, largely depends on the sensor with the lowest accuracy. In this paper an efficient method to associate obstacles across sensor grids is proposed. Imprecise sensors are discounted locally in cells where a more accurate sensor, that detected the same obstacle, derived free space. Furthermore, fixed discount factors to optimize false negative and false positive rates are used. Because of its generic formulation with the covariance of each sensor grid, the method is scalable to any sensor setup. The quantitative evaluation with a highly precise navigation map shows an increased obstacle position accuracy compared to standard evidential occupancy grid fusion.

Fast and robust detection of anatomical structures or pathologies represents a fundamental task in medical image analysis. Most of the current solutions are however suboptimal and unconstrained by learning an appearance model and exhaustively scanning the space of parameters to detect a specific anatomical structure. In addition, typical feature computation or estimation of meta-parameters related to the appearance model or the search strategy, is based on local criteria or predefined approximation schemes. We propose a new learning method following a fundamentally different paradigm by simultaneously modeling both the object appearance and the parameter search strategy as a unified behavioral task for an artificial agent. The method combines the advantages of behavior learning achieved through reinforcement learning with effective hierarchical feature extraction achieved through deep learning. We show that given only a sequence of annotated images, the agent can automatically and strategically learn optimal paths that converge to the sought anatomical landmark location as opposed to exhaustively scanning the entire solution space. The method significantly outperforms state-of-the-art machine learning and deep learning approaches both in terms of accuracy and speed on 2D magnetic resonance images, 2D ultrasound and 3D CT images, achieving average detection errors of 1-2 pixels, while also recognizing the absence of an object from the image. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In the diagnosis of myocardial infarction, magnetic resonance imaging can provide information about myocardial contractility and tissue characterization, including viability. In current clinical practice, separate scans are required for each aspect. A recently proposed method showed how the same information can be extracted from a single, short scan of \(4\,\text {s}\) , but made strong assumptions about the underlying cardiac motion. We propose a fixed-point iteration scheme that retains the benefits of their approach while lifting its limitations, making it robust to cardiac arrhythmia. We compare our method to the state of the art using phantom data as well as data from 11 patients and show a consistent improvement of all evaluation criteria, e. g. the end-diastolic Dice coefficient of an arrythmic case improves from \(86\,\%\) (state-of-the-art method) to \(94\,\%\) (proposed method). These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Robust and fast solutions for anatomical object detection and segmentation support the entire clinical workflow from diagnosis, patient stratification, therapy planning, intervention and follow-up. Current state-of-the-art techniques for parsing volumetric medical image data are typically based on machine learning methods that exploit large annotated image databases. Two main challenges need to be addressed, these are the efficiency in scanning high-dimensional parametric spaces and the need for representative image features which require significant efforts of manual engineering. We propose a pipeline for object detection and segmentation in the context of volumetric image parsing, solving a two-step learning problem: anatomical pose estimation and boundary delineation. For this task we introduce Marginal Space Deep Learning (MSDL), a novel framework exploiting both the strengths of efficient object parametrization in hierarchical marginal spaces and the automated feature design of Deep Learning (DL) network architectures. In the 3D context, the application of deep learning systems is limited by the very high complexity of the parametrization. More specifically 9 parameters are necessary to describe a restricted affine transformation in 3D, resulting in a prohibitive amount of billions of scanning hypotheses. The mechanism of marginal space learning provides excellent run-time performance by learning classifiers in clustered, high-probability regions in spaces of gradually increasing dimensionality. To further increase computational efficiency and robustness, in our system we learn sparse adaptive data sampling patterns that automatically capture the structure of the input. Given the object localization, we propose a DL-based active shape model to estimate the non-rigid object boundary. Experimental results are presented on the aortic valve in ultrasound using an extensive dataset of 2891 volumes from 869 patients, showing significant improvements of up to 45.2% over the state-of-the-art. To our knowledge, this is the first successful demonstration of the DL potential to detection and segmentation in full 3D data with parametrized representations.

Electrophysiology (EP) studies and catheter ablation have become important treatment options for several types of cardiac arrhythmias. We present a novel image-based approach for automatic detection and 3-D reconstruction of EP catheters where the physician marks the catheter to be reconstructed by a single click in each image. The result can be used to provide 3-D information for enhanced navigation throughout EP procedures. Our approach involves two X-ray projections acquired from different angles, and it is based on two steps: First, we detect the catheter in each view after manual initialization using a graph-search method. Then, the detection results are used to reconstruct a full 3-D model of the catheter based on automatically determined point pairs for triangulation. An evaluation on 176 different clinical fluoroscopic images yielded a detection rate of 83.4%. For measuring the error, we used the coupling distance which is a more accurate quality measure than the average point-wise distance to a reference. For successful outcomes, the 2-D detection error was 1.7 mm ±1.2 mm. Using successfully detected catheters for reconstruction, we obtained a reconstruction error of 1.8 mm ±1.1 mm on phantom data. On clinical data, our method yielded a reconstruction error of 2.2 mm ±2.2 mm.

Geometry and physics of absorption imaging impose certain constraints on X-ray projections. Recently, the Epipolar Consistency Conditions (ECC) have been introduced and applied to motion correction in flat-detector computed tomography (CT). They are based on redundant information in transmission images along epipolar lines. Unlike other consistency conditions for CT scans, they act directly on an arbitrary pair of X-ray images. This paper proposes an application of ECC to 3D patient tracking in interventional radiology. We evaluate the proposed method against 2D-3D registration with a previously acquired CT. Our experiments on synthetic data based on a patient CT and phantom data from an interventional C-arm demonstrate that our method is able to compensate online for rotations of up to ±10° and translations of ±25 mm between consecutive frames. We successfully track rotations of as much as 45° over 45 images. The outstanding property of the approach is that no 3D scan is required for tracking a 3D object in space. We show, that small rotations of about 3° in space and translations of about 50 mm can be tracked based on just two reference X-ray images. Since the proposed approach works directly on X-ray images, it exceeds regular 2D-3D registration with a CT in an order of magnitude in computational speed. We conclude that ECC are a simple and effective new tool for pre-aligment and online patient tracking for fluoroscopic sequences.

This paper presents the derivation of the Epipolar Consistency Conditions (ECC) between two X-ray images from the Beer-Lambert law of X-ray attenuation and the Epipolar Geometry of two pinhole cameras, using Grangeat's theorem. We motivate the use of Oriented Projective Geometry to express redundant line integrals in projection images and define a consistency metric, which can be used, for instance, to estimate patient motion directly from a set of X-ray images. We describe in detail the mathematical tools to implement an algorithm to compute the Epipolar Consistency Metric and investigate its properties with detailed random studies on both artificial and real FD-CT data. A set of six reference projections of the CT scan of a fish were used to evaluate accuracy and precision of compensating for random disturbances of the ground truth projection matrix using an optimization of the consistency metric. In addition, we use three X-ray images of a pumpkin to prove applicability to real data. We conclude, that the metric might have potential in applications related to the estimation of projection geometry. By expression of redundancy between two arbitrary projection views, we in fact support any device or acquisition trajectory which uses a cone-beam geometry. We discuss certain geometric situations, where the ECC provide the ability to correct 3D motion, without the need for 3D reconstruction.

Three-dimensional (3-D) reconstruction of histological slice sequences offers great benefits in the investigation of different morphologies. It features very high-resolution which is still unmatched by in vivo 3-D imaging modalities, and tissue staining further enhances visibility and contrast. One important step during reconstruction is the reversal of slice deformations introduced during histological slice preparation, a process also called image unwarping. Most methods use an external reference, or rely on conservative stopping criteria during the unwarping optimization to prevent straightening of naturally curved morphology. Our approach shows that the problem of unwarping is based on the superposition of low-frequency anatomy and high-frequency errors. We present an iterative scheme that transfers the ideas of the Gauss-Seidel method to image stacks to separate the anatomy from the deformation. In particular, the scheme is universally applicable without restriction to a specific unwarping method, and uses no external reference. The deformation artifacts are effectively reduced in the resulting histology volumes, while the natural curvature of the anatomy is preserved. The validity of our method is shown on synthetic data, simulated histology data using a CT data set and real histology data. In the case of the simulated histology where the ground truth was known, the mean Target Registration Error (TRE) between the unwarped and original volume could be reduced to less than 1 pixel on average after six iterations of our proposed method.

4-D time-resolved velocity-encoded phase-contrast MRI (4-D PCI) is a fully non-invasive technique to assess hemodynamics in vivo with a broad range of potential applications in multiple cardiovascular diseases. It is capable of providing quantitative flow values and anatomical information simultaneously. The long acquisition time, however, still inhibits its wider clinical use. Acceleration is achieved at present using parallel MRI (pMRI) techniques which can lead to substantial loss of image quality for higher acceleration factors. Both the high-dimensionality and the significant degree of spatio-temporal correlation in 4-D PCI render it ideally suited for recently proposed compressed sensing (CS) techniques. We propose the Multi-Dimensional Flow-preserving Compressed Sensing (MuFloCoS) method to exploit these properties. A multi-dimensional iterative reconstruction is combined with an interleaved sampling pattern (I-VT), an adaptive masked and weighted temporal regularization (TMW) and fully automatically obtained vessel-masks. The performance of the novel method was analyzed concerning image quality, feasibility of acceleration factors up to 15, quantitative flow values and diagnostic accuracy in phantom experiments and an in vivo carotid study with 18 volunteers. Comparison with iterative state-of-the-art methods revealed significant improvements using the new method, the temporal normalized root mean square error of the peak velocity was reduced by 45.32% for the novel MuFloCoS method with acceleration factor 9. The method was furthermore applied to two patient cases with diagnosed high-grade stenosis of the ICA, which confirmed the performance of MuFloCoS to produce valuable results in the presence of pathological findings in 56 s instead of over 8 min (full sampling).

Classical surgery is being overtaken by minimally invasive and transcatheter procedures. As there is no direct view or access to the affected anatomy, advanced imaging techniques such as 3D C-arm computed tomography (CT) and C-arm fluoroscopy are routinely used in clinical practice for intraoperative guidance. However, due to constraints regarding acquisition time and device configuration, intraoperative modalities have limited soft tissue image quality and reliable assessment of the cardiac anatomy typically requires contrast agent, which is harmful to the patient and requires complex acquisition protocols. We propose a probabilistic sparse matching approach to fuse high-quality preoperative CT images and nongated, noncontrast intraoperative C-arm CT images by utilizing robust machine learning and numerical optimization techniques. Thus, high-quality patient-specific models can be extracted from the preoperative CT and mapped to the intraoperative imaging environment to guide minimally invasive procedures. Extensive quantitative experiments on 95 clinical datasets demonstrate that our model-based fusion approach has an average execution time of 1.56 s, while the accuracy of 5.48 mm between the anchor anatomy in both images lies within expert user confidence intervals. In direct comparison with image-to-image registration based on an open-source state-of-the-art medical imaging library and a recently proposed quasi-global, knowledge-driven multi-modal fusion approach for thoracic-abdominal images, our model-based method exhibits superior performance in terms of registration accuracy and robustness with respect to both target anatomy and anchor anatomy alignment errors.

C-arm computed tomography (CT) is an innovative technique that enables a C-arm system to generate 3-D images from a set of 2-D X-ray projections. This technique can reduce treatment-related complications and may improve interventional efficacy and safety. However, state-of-the-art C-arm systems rely on a circular short scan for data acquisition, which limits coverage in the axial direction. This limitation was reported as a problem in hepatic vascular interventions. To solve this problem, as well as to further extend the value of C-arm CT, axially extended-volume C-arm CT is needed. For example, such an extension would enable imaging the full aorta, the peripheral arteries or the spine in the interventional room, which is currently not feasible. In this paper, we demonstrate that performing long object imaging using a reverse helix is feasible in the interventional room. This demonstration involved developing a novel calibration method, assessing geometric repeatability, implementing a reconstruction method that applies to real reverse helical data, and quantitatively evaluating image quality. Our results show that: 1) the reverse helical trajectory can be implemented and reliably repeated on a multiaxis C-arm system; and 2) a long volume can be reconstructed with satisfactory image quality using reverse helical data.

In interventional radiology, preoperative 3-D volumes can be fused with intra-operative 2-D fluoroscopic images. Since the accuracy is crucial to the clinical usability of image fusion, patient motion resulting in misalignments has to be corrected during the procedure. In this paper, a novel gradient based differential approach is proposed to estimate the 3-D rigid motion from the 2-D tracking of contour points. The mathematical relationship between the 3-D differential motion and the 2-D motion is derived using the 3-D gradient, based on which a tracking-based motion compensation pipeline is introduced. Given the initial registration, the contour points are extracted and tracked along 2-D frames. The 3-D rigid motion is estimated using the iteratively re-weighted least square minimization to enhance the robustness. Our novel approach is evaluated on 10 datasets consisting of 1010 monoplane fluoroscopic images of a thorax phantom with 3-D rigid motion. Over all datasets, the maximum structure shift in the 2-D projection caused by the 3-D motion varies from 17.3 mm to 33.2 mm. Our approach reduces the 2-D structure shift to the range of 1.93 mm to 6.52 mm. For the most challenging longitudinal off-plane rotation, our approach achieves an average coverage of 79.9% regarding to the ground truth.

Intra-operative imaging techniques for obtaining the shape and morphology of soft-tissue surfaces in vivo are a key enabling technology for advanced surgical systems. Different optical techniques for 3-D surface reconstruction in laparoscopy have been proposed, however, so far no quantitative and comparative validation has been performed. Furthermore, robustness of the methods to clinically important factors like smoke or bleeding has not yet been assessed. To address these issues, we have formed a joint international initiative with the aim of validating different state-of-the-art passive and active reconstruction methods in a comparative manner. In this comprehensive in vitro study, we investigated reconstruction accuracy using different organs with various shape and texture and also tested reconstruction robustness with respect to a number of factors like the pose of the endoscope as well as the amount of blood or smoke present in the scene. The study suggests complementary advantages of the different techniques with respect to accuracy, robustness, point density, hardware complexity and computation time. While reconstruction accuracy under ideal conditions was generally high, robustness is a remaining issue to be addressed. Future work should include sensor fusion and in vivo validation studies in a specific clinical context. To trigger further research in surface reconstruction, stereoscopic data of the study will be made publically available at www.open-CAS.com upon publication of the paper.

It is known that a reduction of the field-of-view in 3-D X-ray imaging is proportional to a reduction in radiation dose. The resulting truncation, however, is incompatible with conventional reconstruction algorithms. Recently, a novel method for region of interest reconstruction that uses neither prior knowledge nor extrapolation has been published, named approximated truncation robust algorithm for computed tomography (ATRACT). It is based on a decomposition of the standard ramp filter into a 2-D Laplace filtering and a 2-D Radon-based residual filtering step. In this paper, we present two variants of the original ATRACT. One is based on expressing the residual filter as an efficient 2-D convolution with an analytically derived kernel. The second variant is to apply ATRACT in 1-D to further reduce computational complexity. The proposed algorithms were evaluated by using a reconstruction benchmark, as well as two clinical data sets. The results are encouraging since the proposed algorithms achieve a speed-up factor of up to 245 compared to the 2-D Radon-based ATRACT. Reconstructions of high accuracy are obtained, e.g., even real-data reconstruction in the presence of severe truncation achieve a relative root mean square error of as little as 0.92% with respect to nontruncated data.

3D displays enable immersive visual impressions but the impact on the human perception still is not fully understood. Viewing conditions like the convergence-accommodation (C-A) conflict have an unnatural influence on the visual system and might even lead to visual discomfort. As visual perception is individual we assumed the impact of simulated 3D content on the visual system to be as well. In this study we aimed to analyze the stereoscopic visual performance of 17 subjects for disparities inside and outside the in literature defined zone of comfortable viewing to provide an individual evaluation of the impact of increased disparities on the performance of the visual system. Stereoscopic stimuli were presented in a four-alternative forced choice (4AFC) setup in different disparities. The response times as well as the correct decision rates indicated the performance of stereoscopic vision. The results showed that increased disparities lead to a decline in performance. Further, the impact of the presented disparities is dependent on the difficulty of the task. The decline of performance as well as the deciding disparities for the decline were subject dependent.

Fundus imaging is one of the most frequently used modalities for screening, diagnosis of eye diseases and some vascular abnormalities. Due to its wide availability, automatic evaluation of fundus images offers great potential benefits to current clinical practice. The basis of many automatic evaluations or diagnosis is the segmentation of the eye background, most notably, the detection of the optic disk and the segmentation of the retinal vessel tree. In this work we propose a variant of the fast radial symmetry transform (FRST), adapted to its application in the detection of the optic disk in fundus images. We evaluated and compared the performance of our method to the standard FRST and the similar, gradient based circular Hough transform using 45 images of a high resolution database with gold standard information available. We demonstrated in our experiments that the proposed method outperforms the state-of-the-art algorithms with 0.051 ± 0.073 optic disk diameter localization error in average.

Fundus imaging is the most commonly used modality to collect information about the human eye background. Objective and quantitative assessment of quality for the acquired images is essential for manual, computer-aided and fully automatic diagnosis. In this paper, we present a no-reference quality metric to quantify image noise and blur and its application to fundus image quality assessment. The proposed metric takes the vessel tree visible on the retina as guidance to determine an image quality score. In our experiments, the performance of this approach is demonstrated by correlation analysis with the established full-reference metrics peak-signal-to-noise ratio (PSNR) and structural similarity (SSIM). We found a Spearman rank correlation for PSNR and SSIM of 0.89 and 0.91. For real data, our metric correlates reasonable to a human observer, indicating high agreement to human visual perception.

Minimally invasive procedures are of importance in modern surgery due to reduced operative trauma and recovery time. To enable robot assisted interventions, automatic tracking of endoscopie tools is an essential task. State-of-the-art techniques rely on 2-D color information only which is error prone for varying illumination and unpredictable color distribution within the human body. In this paper, we use a novel 3-D Time-of-Flight/RGB endoscope that allows to use both color and range information to locate laparoscopic instruments in 3-D. Regarding color and range information the proposed technique calculates a score to indicate which information is more reliable and adopts the next steps of the localization procedure based on this reliability. In experiments on real data the tool tip is located with an average 3-D distance error of less than 4 mm compared to manually labeled ground truth data with a frame-rate of 10 fps.

Automatic assessment of retinal vessels plays an important role in the diagnosis of various eye, as well as systemic diseases. A public screening is highly desirable for prompt and effective treatment, since such diseases need to be diagnosed at an early stage. Automated and accurate segmentation of the retinal blood vessel tree is one of the challenging tasks in the computer‐aided analysis of fundus images today. We improve the concept of matched filtering, and propose a novel and accurate method for segmenting retinal vessels. Our goal is to be able to segment blood vessels with varying vessel diameters in high‐resolution colour fundus images. All recent authors compare their vessel segmentation results to each other using only low‐resolution retinal image databases. Consequently, we provide a new publicly available high‐resolution fundus image database of healthy and pathological retinas. Our performance evaluation shows that the proposed blood vessel segmentation approach is at least comparable with recent state‐of‐the‐art methods. It outperforms most of them with an accuracy of 95% evaluated on the new database.

Tissue perfusion measurement using C-arm angiography systems capable of CT-like imaging (C-arm CT) is a novel technique with potentially high benefit for catheter guided treatment of stroke in the interventional suite. However, perfusion C-arm CT (PCCT) is challenging: the slow C-arm rotation speed only allows measuring samples of contrast time attenuation curves (TACs) every 5-6 s if reconstruction algorithms for static data are used. Furthermore, the peak values of the TACs in brain tissue typically lie in a range of 5-30 HU, thus perfusion imaging is very sensitive to noise. We present a dynamic, iterative reconstruction (DIR) approach to reconstruct TACs described by a weighted sum of basis functions. To reduce noise, a regularization technique based on joint bilateral filtering (JBF) is introduced. We evaluated the algorithm with a digital dynamic brain phantom and with data from six canine stroke models. With our dynamic approach, we achieve an average Pearson correlation (PC) of the PCCT canine blood flow maps to co-registered perfusion CT maps of 0.73. This PC is just as high as the PC achieved in a recent PCCT study, which required repeated injections and acquisitions.

We present a novel machine learning-based system for unstained cell detection in bright-field microscope images. The system is fully automatic since it requires no manual parameter tuning. It is also highly invariant with respect to illumination conditions and to the size and orientation of cells. Images from two adherent cell lines and one suspension cell line were used in the evaluation for a total number of more than 3500 cells. Besides real images, simulated images were also used in the evaluation. The detection error was between approximately zero and 15.5% which is a significantly superior performance compared to baseline approaches.

The patient dose in computed tomography (CT) imaging is linked to measurement noise. Various noise-reduction techniques have been developed that adapt structure preserving filters like anisotropic diffusion or bilateral filters to CT noise properties. We introduce a structure adaptive sinogram (SAS) filter that incorporates the specific properties of the CT measurement process. It uses a point-based forward projector to generate a local structure representation called ray contribution mask (RCM). The similarities between neighboring RCMs are used in an enhanced variant of the bilateral filtering concept, where the photometric similarity is replaced with the structural similarity. We evaluate the performance in four different scenarios: The robustness against reconstruction artifacts is demonstrated by a scan of a high-resolution-phantom. Without changing the modulation transfer function (MTF) nor introducing artifacts, the SAS filter reduces the noise level by 13.6%. The image sharpness and noise reduction capabilities are visually assessed on in vivo patient scans and quantitatively evaluated on a simulated phantom. Unlike a standard bilateral filter, the SAS filter preserves edge information and high-frequency components of organ textures well. It shows a homogeneous noise reduction behavior throughout the whole frequency range. The last scenario uses a simulated edge phantom to estimate the filter MTF for various contrasts: the noise reduction for the simple edge phantom exceeds 80%. For low contrasts at 55 Hounsfield units (HU), the mid-frequency range is slightly attenuated, at higher contrasts of approximately 100 HU and above, the MTF is fully preserved.

Fluoroscopic overlay images rendered from preoperative volumetric data can provide additional anatomical details to guide physicians during catheter ablation procedures for treatment of atrial fibrillation (AFib). As these overlay images are often compromised by cardiac and respiratory motion, motion compensation methods are needed to keep the overlay images in sync with the fluoroscopic images. So far, these approaches have either required simultaneous biplane imaging for 3-D motion compensation, or in case of monoplane X-ray imaging, provided only a limited 2-D functionality. To overcome the downsides of the previously suggested methods, we propose an approach that facilitates a full 3-D motion compensation even if only monoplane X-ray images are available. To this end, we use a training phase that employs a biplane sequence to establish a patient specific motion model. Afterwards, a constrained model-based 2-D/3-D registration method is used to track a circumferential mapping catheter. This device is commonly used for AFib catheter ablation procedures. Based on the experiments on real patient data, we found that our constrained monoplane 2-D/3-D registration outperformed the unconstrained counterpart and yielded an average 2-D tracking error of 0.6 mm and an average 3-D tracking error of 1.6 mm. The unconstrained 2-D/3-D registration technique yielded a similar 2-D performance, but the 3-D tracking error increased to 3.2 mm mostly due to wrongly estimated 3-D motion components in X-ray view direction. Compared to the conventional 2-D monoplane method, the proposed method provides a more seamless workflow by removing the need for catheter model re-initialization otherwise required when the C-arm view orientation changes. In addition, the proposed method can be straightforwardly combined with the previously introduced biplane motion compensation technique to obtain a good trade-off between accuracy and radiation dose reduction.

Tissue perfusion measurement during catheter-guided stroke treatment in the interventional suite is currently not possible. In this work, we present a novel approach that uses a C-arm angiography system capable of computed tomography (CT)-like imaging (C-arm CT) for this purpose. With C-arm CT one reconstructed volume can be obtained every 4-6 s which makes it challenging to measure the flow of an injected contrast bolus. We have developed an interleaved scanning (IS) protocol that uses several scan sequences to increase temporal sampling. Using a dedicated 4-D reconstruction approach based on partial reconstruction interpolation (PRI) we can optimally process our data. We evaluated our combined approach (IS-PRI) with simulations and a study in five healthy pigs. In our simulations, the cerebral blood flow values (unit: ml/100 g/min) were 60 (healthy tissue) and 20 (pathological tissue). For one scan sequence the values were estimated with standard deviations of 14.3 and 2.9, respectively. For two interleaved sequences the standard deviations decreased to 3.6 and 1.5, respectively. We used perfusion CT to validate the in vivo results. With two interleaved sequences we achieved promising correlations ranging from r=0.63 to r=0.94. The results suggest that C-arm CT tissue perfusion imaging is feasible with two interleaved scan sequences.

Atrial fibrillation is a major cause of stroke. Its treatment is performed under fluoroscopic image guidance. Augmented fluoroscopy has become a useful tool during the ablation procedure for navigation under X-ray. Unfortunately, current navigation systems do not provide tools to localize and visualize a cryo-balloon catheters in 3-D. This is why we present a new approach to reconstruct the cryo-ballon catheter, modeled as a sphere, from two views. The reconstruction result can then be overlayed onto live fluoroscopic images during the procedure. In simulation studies, we compared our technique to a reference method. While both methods worked equally well on noise-free data, we found our method more reliable if the input data was affected by noise. For example, in the presence of noise with a standard deviation of 4 mm, our maximum 3-D reconstruction error was less than 1 mm.

Methods for 2D/3D face recognition typically combine results obtained independently from the 2D and 3D data, respectively. There has not been much emphasis on data fusion at an early stage, even though it is at least potentially more powerful to exploit possible synergies between the two modalities. In this paper, we propose photogeometric features that interpret both the photometric texture and geometric shape information of 2D manifolds in a consistent manner. The 4D features encode the spatial distribution of gradients that are derived generically for any scalar field on arbitrary organized surface meshes. We apply the descriptor for biometric face recognition with a time-of-flight sensor. The method consists of three stages: (i) facial landmark localization with a HOG/SVM sliding window framework, (ii) extraction of photogeometric feature descriptors from time-of-flight data, using the inherent grayscale intensity information of the sensor as the 2D manifold's scalar field, (iii) probe matching against the gallery. Recognition based on the photogeometric features achieved 97.5% rank-1 identification rate on a comprehensive time-of-flight dataset (26 subjects, 364 facial images).

Being able to segment the esophagus without user interaction from 3-D CT data is of high value to radiologists during oncological examinations of the mediastinum. The segmentation can serve as a guideline and prevent confusion with pathological tissue. However, limited contrast to surrounding structures and versatile shape and appearance make segmentation a challenging problem. This paper presents a multistep method. First, a detector that is trained to learn a discriminative model of the appearance is combined with an explicit model of the distribution of respiratory and esophageal air. In the next step, prior shape knowledge is incorporated using a Markov chain model. We follow a "detect and connect" approach to obtain the maximum a posteriori estimate of the approximate esophagus shape from hypothesis about the esophagus contour in axial image slices. Finally, the surface of this approximation is nonrigidly deformed to better fit the boundary of the organ. The method is compared to an alternative approach that uses a particle filter instead of a Markov chain to infer the approximate esophagus shape, to the performance of a human observer and also to state of the art methods, which are all semiautomatic. Cross-validation on 144 CT scans showed that the Markov chain based approach clearly outperforms the particle filter. It segments the esophagus with a mean error of 1.80 mm in less than 16 s on a standard PC. This is only 1 mm above the interobserver variability and can compete with the results of previously published semiautomatic methods.

Genetic programming is the usage of the paradigm of survival of the fittest in scientific computing. It is applied to evolve solutions to problems where dependencies between multiple input factors are unknown. In this paper we propose and evaluate the application of a specifically adapted genetic programming framework to optimize the rule base of an expert system. The expert system controls a computer-aided-design software and targets the automation of a manufacturing process. The used steady state genetic programming framework introduces some variations on the selection and evolution operators normally used in genetic programming. In particular: size enforcing mutation, dynamic fitness calculation and size constraint ranking. The genetic programming system is evaluated with real data and led to an improved expert system performance of about 22 percent.

Lymph nodes have high clinical relevance but detection is challenging as they are hard to see due to low contrast and irregular shape. In this paper, a method for fully automatic mediastinal lymph node detection in 3-D computed tomography (CT) images of the chest area is proposed. Discriminative learning is used to detect lymph nodes based on their appearance. Because lymph nodes can easily be confused with other structures, it is vital to incorporate as much anatomical knowledge as possible to achieve good detection rates. Here, a learned prior of the spatial distribution is proposed to model this knowledge. As atlas matching is generally inaccurate in the chest area because of anatomical variations, this prior is not learned in the space of a single atlas, but in the space of multiple ones that are attached to anatomical structures. During test, the priors are weighted and merged according to spatial distances. Cross-validation on 54 CT datasets showed that the prior based detector yields a true positive rate of 52.3% for seven false positives per volume image, which is about two times better than without a spatial prior.

In this paper we propose and validate a PCA-based respiratory motion model for motion compensation during image-guided cardiac interventions. In a preparatory training phase, a preoperative 3-D segmentation of the coronary arteries is automatically registered with a cardiac gated biplane cineangiogram, and used to build a respiratory motion model. This motion model is subsequently used as a prior within the intraoperative registration process for motion compensation to restrict the search space. Our hypothesis is that the use of this model-constrained registration increases the robustness and registration accuracy, especially for weak data constraints such as low signal-to-noise ratio, the lack of contrast information, or an intraoperative monoplane setting. This allows for reducing radiation exposure without compromising on registration accuracy. Synthetic data as well as phantom and clinical datasets have been used to validate the model-based registration in terms of registration accuracy, robustness and speed. We were able to significantly accelerate the intraoperative registration with a 3-D TRE of less than 2 mm for both monoplane images and intraprocedure settings with missing contrast information based on 2-D guidewire tracking, which makes it feasible for motion correction in clinical procedures.

In this paper a novel system for automatic detection and segmentation of focal liver lesions in CT images is presented. It utilizes a probabilistic boosting tree to classify points in the liver as either lesion or parenchyma, thus providing both detection and segmentation of the lesions at the same time and fully automatically. To make the segmentation more robust, an iterative classification scheme is integrated, that incorporates knowledge gained from earlier iterations into later decisions. Finally, a comprehensive evaluation of both the segmentation and the detection performance for the most common hypo dense lesions is given. Detection rates of 77% could be achieved with a sensitivity of 0.95 and a specificity of 0.93 for lesion segmentation at the same settings.

Over the past ten years similarity measures based on intensity distributions have become state-of-the-art in automatic multimodal image registration. An implementation for clinical usage has to support a plurality of images. However, a generally applicable parameter configuration for the number and sizes of histogram bins, optimal Parzen-window kernel widths or background thresholds cannot be found. This explains why various research groups present partly contradictory empirical proposals for these parameters. This paper proposes a set of data-driven estimation schemes for a parameter-free implementation that eliminates major caveats of heuristic trial and error. We present the following novel approaches: a new coincidence weighting scheme to reduce the influence of background noise on the similarity measure in combination with Max-Lloyd requantization, and a tradeoff for the automatic estimation of the number of histogram bins. These methods have been integrated into a state-of-the-art rigid registration that is based on normalized mutual information and applied to CT-MR, PET-MR, and MR-MR image pairs of the RIRE 2.0 database. We compare combinations of the proposed techniques to a standard implementation using default parameters, which can be found in the literature, and to a manual registration by a medical expert. Additionally, we analyze the effects of various histogram sizes, sampling rates, and error thresholds for the number of histogram bins. The comparison of the parameter selection techniques yields 25 approaches in total, with 114 registrations each. The number of bins has no significant influence on the proposed implementation that performs better than both the manual and the standard method in terms of acceptance rates and target registration error (TRE). The overall mean TRE is 2.34 mm compared to 2.54 mm for the manual registration and 6.48 mm for a standard implementation. Our results show a significant TRE reduction for distortion-corrected magnetic resonance images.

This paper presents a comparison study between 10 automatic and six interactive methods for liver segmentation from contrast-enhanced CT images. It is based on results from the "MICCAI 2007 Grand Challenge" workshop, where 16 teams evaluated their algorithms on a common database. A collection of 20 clinical images with reference segmentations was provided to train and tune algorithms in advance. Participants were also allowed to use additional proprietary training data for that purpose. All teams then had to apply their methods to 10 test datasets and submit the obtained results. Employed algorithms include statistical shape models, atlas registration, level-sets, graph-cuts and rule-based systems. All results were compared to reference segmentations five error measures that highlight different aspects of segmentation accuracy. All measures were combined according to a specific scoring system relating the obtained values to human expert variability. In general, interactive methods reached higher average scores than automatic approaches and featured a better consistency of segmentation quality. However, the best automatic methods (mainly based on statistical shape models with some additional free deformation) could compete well on the majority of test images. The study provides an insight in performance of different segmentation approaches under real-world conditions and highlights achievements and limitations of current image analysis techniques.

A major disadvantage of magnetic resonance imaging (MRI) compared to other imaging modalities like computed tomography is the fact that its intensities are not standardized. Our contribution is a novel method for MRI signal intensity standardization of arbitrary MRI scans, so as to create a pulse sequence dependent standard intensity scale. The proposed method is the first approach that uses the properties of all acquired images jointly (e.g., T1- and T2-weighted images). The image properties are stored in multidimensional joint histograms. In order to normalize the probability density function (pdf) of a newly acquired data set, a nonrigid image registration is performed between a reference and the joint histogram of the acquired images. From this matching a nonparametric transformation is obtained, which describes a mapping between the corresponding intensity spaces and subsequently adapts the image properties of the newly acquired series to a given standard. As the proposed intensity standardization is based on the probability density functions of the data sets only, it is independent of spatial coherence or prior segmentations of the reference and current images. Furthermore, it is not designed for a particular application, body region or acquisition protocol. The evaluation was done using two different settings. First, MRI head images were used, hence the approach can be compared to state-of-the-art methods. Second, whole body MRI scans were used. For this modality no other normalization algorithm is known in literature. The Jeffrey divergence of the pdfs of the whole body scans was reduced by 45%. All used data sets were acquired during clinical routine and thus included pathologies.

Generating 3-D images of the heart during interventional procedures is a significant challenge. In addition to real time fluoroscopy, angiographic C-arm systems can also now be used to generate 3-D/4-D CT images on the same system. One protocol for cardiac CT uses ECG triggered multisweep scans. A 3-D volume of the heart at a particular cardiac phase is then reconstructed by applying Feldkamp (FDK) reconstruction to the projection images with retrospective ECG gating. In this work we introduce a unified framework for heart motion estimation and dynamic cone-beam reconstruction using motion corrections. The benefits of motion correction are 1) increased temporal and spatial resolution by removing cardiac motion which may still exist in the ECG gated data sets and 2) increased signal-to-noise ratio (SNR) by using more projection data than is used in standard ECG gated methods. Three signal-enhanced reconstruction methods are introduced that make use of all of the acquired projection data to generate a 3-D reconstruction of the desired cardiac phase. The first averages all motion corrected back-projections; the second and third perform a weighted averaging according to 1) intensity variations and 2) temporal distance relative to a time resolved and motion corrected reference FDK reconstruction. In a comparison study seven methods are compared: nongated FDK, ECG-gated FDK, ECG-gated, and motion corrected FDK, the three signal-enhanced approaches, and temporally aligned and averaged ECG-gated FDK reconstructions. The quality measures used for comparison are spatial resolution and SNR. Evaluation is performed using phantom data and animal models. We show that data driven and subject-specific motion estimation combined with motion correction can decrease motion-related blurring substantially. Furthermore, SNR can be increased by up to 70% while maintaining spatial resolution at the same level as is provided by the ECG-gated FDK. The presented framework provides excellent image quality for cardiac C-arm CT.

Contactless human-machine-interfaces (HMIs) are an important issue in various applications where a haptic interaction with an input device is not possible or not appropriate. Newly developed Time-of-Flight cameras provide 3D information of the observed scene in real-time at constant lateral resolutions of thousands of pixels. Additionally, a gray-value image of the observed scene is available. Our work compromises three major contributions: First, the robust and real-time capable segmentation of the hand by incorporating 3D and gray-value information; Second, the reliable classification of the performed static gesture using robust features; Third, the design of an HMI which uses the classified gesture as well as the 3D position of the hand to enable complex and convenient user interactions. The benefit of using a ToF camera is that the 3D information is just not available from classical 2D camera systems and thus only with ToF cameras the three dimensions of freedom which are given for non-haptic interactions can be fully used. Currently, classification rates of 98.2% are achieved user-dependent and 94.3% user-independent for 6 gestures. Tests with untrained persons yielded a good to very good acceptance of the HMI.

We present an algorithm for the segmentation of the liver in 2-D computed tomography slice images. The basis for our algorithm is an implicit active shape model. In order to detect the liver boundary and guide the shape model deformation, a boundary classifier has been integrated into the implicit framework in a novel manner. The accuracy of the algorithm has been evaluated for 20 test cases including both normal and abnormal livers.

The design and implementation of the reconstruction system in medical X-ray imaging is a challenging issue due to its immense computational demands. In order to ensure an efficient clinical workflow it is inevitable to meet high performance requirements. Hence, the usage of hardware acceleration is mandatory. The software architecture of the reconstruction system is required to be modular in a sense that different accelerator hardware platforms are supported and it must be possible to implement different parts of the algorithm using different acceleration architectures and techniques.

The projection data measured in computed tomography (CT) and, consequently, the slices reconstructed from these data are noisy. We present a new wavelet based structure-preserving method for noise reduction in CT-images that can be used in combination with different reconstruction methods. The approach is based on the assumption that data can be decomposed into information and temporally uncorrelated noise. In CT two spatially identical images can be generated by reconstructions from disjoint subsets of projections: using the latest generation dual source CT-scanners one image can be reconstructed from the projections acquired at the first, the other image from the projections acquired at the second detector. For standard CT-scanners the two images can be generated by splitting up the set of projections into even and odd numbered projections. The resulting images show the same information but differ with respect to image noise. The analysis of correlations between the wavelet representations of the input images allows separating information from noise down to a certain signal-to-noise level. Wavelet coefficients with small correlation are suppressed, while those with high correlations are assumed to represent structures and are preserved. The final noise-suppressed image is reconstructed from the averaged and weighted wavelet coefficients of the input images. The proposed method is robust, of low complexity and adapts itself to the noise in the images. The quantitative and qualitative evaluation based on phantom as well as real clinical data showed, that high noise reduction rates of around 40% can be achieved without noticeable loss of image resolution.

In this paper, we introduce a new algorithm for 3-D image reconstruction from cone-beam (CB) projections acquired along a partial circular scan. Our algorithm is based on a novel, exact factorization of the initial 3-D reconstruction problem into a set of independent 2-D inversion problems, each of which corresponds to finding the object density on one, single plane. Any such 2-D inversion problem is solved numerically using a projected steepest descent iteration scheme. We present a numerical evaluation of our factorization algorithm using computer-simulated CB data, without and with noise, of the FORBILD head phantom and of a disk phantom. First, we study quantitatively the impact of the reconstruction parameters on the algorithm performance. Next, we present reconstruction results for visual assessment of the achievable image quality and provide, for comparison, results obtained with two other state-of-the-art reconstruction algorithms for the circular short-scan.

Ultrasound (US) is among the most popular diagnostic techniques today. It is non-invasive, fast, comparably cheap, and does not require ionizing radiation. US is commonly used to examine the size, and structure of the thyroid gland. In clinical routine, thyroid imaging is usually performed by means of 2-D US. Conventional approaches for measuring the volume of the thyroid gland or its nodules may therefore be inaccurate due to the lack of 3-D information. This work reports a semi-automatic segmentation approach for the classification, and analysis of the thyroid gland based on 3-D US data. The images are scanned in 3-D, pre-processed, and segmented. Several pre-processing methods, and an extension of a commonly used geodesic active contour level set formulation are discussed in detail. The results obtained by this approach are compared to manual interactive segmentations by a medical expert in five representative patients. Our work proposes a novel framework for the volumetric quantification of thyroid gland lobes, which may also be expanded to other parenchymatous organs.

Precise localization of moving targets is essential to increase local control of the cancer via dose escalation while reducing the severity of normal tissue complication. Localization of targets in real time with radio-opaque marker is less favorable considering the excess radiation dose to the patient and potential complications of implantation. Various external surrogates could provide indications of the targets' positions during the breathing process. However, there is a great deal of uncertainty in the correlation between external surrogates and internal target positions/trajectory during respiratory cycles. In order to address this problem, we have developed an algorithm that automatically establishes correspondences between the fluoroscopic sequence frames taken from the patient on the day of treatment and the various phases of a 4DCT planning data set. Image based mapping/synchronization procedure is performed using an underlying Markov model established for the breathing process. The mapping procedure is formulated as an optimization process and is solved efficiently using a dynamic programming technique. Results on the phantom, synthetic, and real patient data demonstrate the effectiveness of the proposed method in coping with respiratory correlation variations. The approach could primarily be used for automatic gating interval adaptation in the gated radiotherapy.

This paper presents a new algorithm based on the Mumford-Shah model for simultaneously detecting the edge features of two images and jointly estimating a consistent set of transformations to match them. Compared to the current asymmetric methods in the literature, this fully symmetric method allows one to determine one-to-one correspondences between the edge features of two images. The entire variational model is realized in a multiscale framework of the finite element approximation. The optimization process is guided by an estimation minimization-type algorithm and an adaptive generalized gradient flow to guarantee a fast and smooth relaxation. The algorithm is tested on T1 and T2 magnetic resonance image data to study the parameter setting. We also present promising results of four applications of the proposed algorithm: interobject monomodal registration, retinal image registration, matching digital photographs of neurosurgery with its volume data, and motion estimation for frame interpolation.

In this paper we consider the problem of object recognition and localization in a probabilistic framework. An object is represented by a parametric probability density, and the computation of pose parameters is implemented as a nonlinear parameter estimation problem. The presence of a probabilistic model allows for recognition according to Bayes rule. The introduced probabilistic model requires no prior segmentation but characterizes the statistical properties of observed intensity values in the image plane. A detailed discussion of the applied theoretical framework is followed by a concise experimental evaluation which demonstrates the benefit of the proposed approach.

The computation of camera motion from image measurements is a parameter estimation problem. We show that for the analysis of the problem's sensitivity, the parameterization must enjoy the property of fairness, which makes sensitivity results invariant to changes of coordinates. We prove that Cartesian unit norm vectors and quaternions are fair parameterizations of rotations and translations, respectively, and that spherical coordinates and Euler angles are not. We extend the Gauss-Markov theorem to implicit formulations with constrained parameters, a necessary step in order to take advantage of fair parameterizations. We show how maximum likelihood (ML) estimation problems whose sensitivity depends on a large number of parameters, such as coordinates of points in the scene, can be partitioned into equivalence classes, with problems in the same class exhibiting the same sensitivity.

A new Bayesian framework for 3-D object classification and localization is introduced. Objects are represented as probability density functions, and observed features are treated as random variables. These probability density functions turn out a non geometric nature of models and characterize the statistical behavior of local object features like points or lines. The parameterization of model densities covers several terms of object recognition: locations and instabilities of features, rotation and translation, projection, the assignment of image and model features, as well as relations. This paper treats especially the probabilistic modeling of relational dependencies between single features. The mathematical framework, the training algorithms, as well as the localization and classification modules are discussed in detail. The experimental evaluation shows the usefulness of the introduced concepts on real image data.

Linear discriminant or Karhunen-Loeve transforms are established techniques for mapping features into a lower dimensional subspace. This paper introduces a uniform statistical framework, where the computation of the optimal feature reduction is formalized as a maximum-likelihood estimation problem. The experimental evaluation of this suggested extension of linear selection methods shows a slight improvement of the recognition accuracy.

This work describes a statistical approach to deal with learning and recognition problems in the field of computer vision. An abstract theoretical framework is provided, which is suitable for automatic model generation from examples, identification, and localization of objects. Both, the learning and localization stage are formalized as parameter estimation tasks. The statistical learning phase is unsupervised with respect to the matching of model and scene features. The general mathematical description yields algorithms which can even treat parameter estimation problems from projected data. The experiments show that this probabilistic approach is suitable for solving 2D and 3D object recognition problems using grey-level images. The method can also be applied to 3D image processing issues using range images, i.e. 3D input data.>

This contribution treats the problem of learning and recognizing 3D objects using 2D views. We present a new Bayesian approach to 3D computer vision based on the expectation-maximization algorithm, where learning and classification of objects correspond to parameter estimation algorithms. We give a formal description of different learning and recognition stages and conclude the associated statistical optimization problems for each Bayesian decision. The training stage is supposed to be unsupervised in the sense that no explicit feature matching among different views is necessary. Finally, the experimental part of the paper considers the special case, where observable point features are assumed to be normally distributed and the object and its projections are modeled by mixture density functions.

In this contribution we describe steps towards the implementation of an active robot vision system. In a sequence of images taken by a camera mounted on the hand of a robot, we detect, track, and estimate the position and orientation (pose) of a three-dimensional moving object. The extraction of the region of interest is done automatically by a motion tracking step. For learning 3-D objects using two-dimensional views and estimating the object's pose, a uniform statistical method is presented which is based on the expectation-maximization-algorithm (EM-algorithm). An explicit matching between features of several views is not necessary. The acquisition of the training sequence required for the statistical learning process needs the correlation between the image of an object and its pose; this is performed automatically by the robot. The robot's camera parameters are determined by a hand/eye-calibration and a subsequent computation of the camera position using the robot position. During the motion estimation stage the moving object is computed using active, elastic contours (snakes). We introduce a new approach for online initializing the snake on the first images of the given sequence, and show that the method of snakes is suited for real time motion tracking.>