We introduce PLIKS (Pseudo-Linear Inverse Kinematic Solver) for reconstruction of a 3D mesh of the human body from a single 2D image. Current techniques directly regress the shape, pose, and translation of a parametric model from an input image through a non-linear mapping with minimal flexibility to any external influences. We approach the task as a model-in-the-loop optimization problem. PLIKS is built on a linearized formulation of the parametric SMPL model. Using PLIKS, we can analytically reconstruct the human model via 2D pixel-aligned vertices. This enables us with the flexibility to use accurate camera calibration information when available. PLIKS offers an easy way to introduce additional constraints such as shape and translation. We present quantitative evaluations which confirm that PLIKS achieves more accurate reconstruction with greater than 10% improvement compared to other state-of-the-art methods with respect to the standard 3D human pose and shape benchmarks while also obtaining a reconstruction error improvement of 12.9 mm on the newer AGORA dataset.

Denoising is an indispensable step in processing low-dose X-ray fluoroscopic images that requires development of specialized high-quality algorithms able to operate in near real-time. We address this problem with an efficient deep learning approach based on the process-centric view of traditional iterative thresholding methods. We develop a novel trainable patch-based multiscale framework for sparse image representation. In a computationally efficient way, it allows us to accurately reconstruct important image features on multiple levels of decomposition with patch dictionaries of reduced size and complexity. The flexibility of the chosen machine learning approach allows us to tailor the learned basis for preserving important structural information in the image and noticeably minimize the amount of artifacts. Our denoising results obtained with real clinical data demonstrate significant quality improvement and are computed much faster in comparison with the BM3D algorithm.

Electrophysiology (EP) studies and catheter ablation have become important treatment options for several types of cardiac arrhythmias. We present a novel image-based approach for automatic detection and 3-D reconstruction of EP catheters where the physician marks the catheter to be reconstructed by a single click in each image. The result can be used to provide 3-D information for enhanced navigation throughout EP procedures. Our approach involves two X-ray projections acquired from different angles, and it is based on two steps: First, we detect the catheter in each view after manual initialization using a graph-search method. Then, the detection results are used to reconstruct a full 3-D model of the catheter based on automatically determined point pairs for triangulation. An evaluation on 176 different clinical fluoroscopic images yielded a detection rate of 83.4%. For measuring the error, we used the coupling distance which is a more accurate quality measure than the average point-wise distance to a reference. For successful outcomes, the 2-D detection error was 1.7 mm ±1.2 mm. Using successfully detected catheters for reconstruction, we obtained a reconstruction error of 1.8 mm ±1.1 mm on phantom data. On clinical data, our method yielded a reconstruction error of 2.2 mm ±2.2 mm.

Cryo-balloon catheters have attracted an increasing amount of interest in the medical community as they can reduce patient risk during left atrial pulmonary vein ablation procedures. As cryo-balloon catheters are not equipped with electrodes, they cannot be localized automatically by electro-anatomical mapping systems. As a consequence, X-ray fluoroscopy has remained an important means for guidance during the procedure. Most recently, image guidance methods for fluoroscopy-based procedures have been proposed, but they provide only limited support for cryo-balloon catheters and require significant user interaction. To improve this situation, we propose a novel method for automatic cryo-balloon catheter detection in fluoroscopic images by detecting the cryo-balloon catheter's built-in X-ray marker. Our approach is based on a blob detection algorithm to find possible X-ray marker candidates. Several of these candidates are then excluded using prior knowledge. For the remaining candidates, several catheter specific features are introduced. They are processed using a machine learning approach to arrive at the final X-ray marker position. Our method was evaluated on 75 biplane fluoroscopy images from 40 patients, from two sites, acquired with a biplane angiography system. The method yielded a success rate of 99.0% in plane A and 90.6% in plane B, respectively. The detection achieved an accuracy of 1.00 mm±0.82 mm in plane A and 1.13 mm±0.24 mm in plane B. The localization in 3-D was associated with an average error of 0.36 mm±0.86 mm.

Fluoroscopic overlay images rendered from preoperative volumetric data can provide additional anatomical details to guide physicians during catheter ablation procedures for treatment of atrial fibrillation (AFib). As these overlay images are often compromised by cardiac and respiratory motion, motion compensation methods are needed to keep the overlay images in sync with the fluoroscopic images. So far, these approaches have either required simultaneous biplane imaging for 3-D motion compensation, or in case of monoplane X-ray imaging, provided only a limited 2-D functionality. To overcome the downsides of the previously suggested methods, we propose an approach that facilitates a full 3-D motion compensation even if only monoplane X-ray images are available. To this end, we use a training phase that employs a biplane sequence to establish a patient specific motion model. Afterwards, a constrained model-based 2-D/3-D registration method is used to track a circumferential mapping catheter. This device is commonly used for AFib catheter ablation procedures. Based on the experiments on real patient data, we found that our constrained monoplane 2-D/3-D registration outperformed the unconstrained counterpart and yielded an average 2-D tracking error of 0.6 mm and an average 3-D tracking error of 1.6 mm. The unconstrained 2-D/3-D registration technique yielded a similar 2-D performance, but the 3-D tracking error increased to 3.2 mm mostly due to wrongly estimated 3-D motion components in X-ray view direction. Compared to the conventional 2-D monoplane method, the proposed method provides a more seamless workflow by removing the need for catheter model re-initialization otherwise required when the C-arm view orientation changes. In addition, the proposed method can be straightforwardly combined with the previously introduced biplane motion compensation technique to obtain a good trade-off between accuracy and radiation dose reduction.

Catheter tracking has become more and more important in recent interventional applications. It provides real time navigation for the physicians and can be used to control a motion compensated fluoro overlay reference image for other means of guidance, e.g. involving a 3D anatomical model. Tracking the coronary sinus (CS) catheter is effective to compensate respiratory and cardiac motion for 3D overlay navigation to assist positioning the ablation catheter in Atrial Fibrillation (Afib) treatments. During interventions, the CS catheter performs rapid motion and non-rigid deformation due to the beating heart and respiration. In this paper, we model the CS catheter as a set of electrodes. Novelly designed hypotheses generated by a number of learning-based detectors are fused. Robust hypothesis matching through a Bayesian framework is then used to select the best hypothesis for each frame. As a result, our tracking method achieves very high robustness against challenging scenarios such as low SNR, occlusion, foreshortening, non-rigid deformation, as well as the catheter moving in and out of ROI. Quantitative evaluation has been conducted on a database of 13221 frames from 1073 sequences. Our approach obtains 0.50mm median error and 0.76mm mean error. 97.8% of evaluated data have errors less than 2.00mm. The speed of our tracking algorithm reaches 5 frames-per-second on most data sets. Our approach is not limited to the catheters inside the CS but can be extended to track other types of catheters, such as ablation catheters or circumferential mapping catheters.

Atrial fibrillation is a major cause of stroke. Its treatment is performed under fluoroscopic image guidance. Augmented fluoroscopy has become a useful tool during the ablation procedure for navigation under X-ray. Unfortunately, current navigation systems do not provide tools to localize and visualize a cryo-balloon catheters in 3-D. This is why we present a new approach to reconstruct the cryo-ballon catheter, modeled as a sphere, from two views. The reconstruction result can then be overlayed onto live fluoroscopic images during the procedure. In simulation studies, we compared our technique to a reference method. While both methods worked equally well on noise-free data, we found our method more reliable if the input data was affected by noise. For example, in the presence of noise with a standard deviation of 4 mm, our maximum 3-D reconstruction error was less than 1 mm.

Cardiac ablation involves the risk of serious complications when thermal injury to the esophagus occurs. This paper proposes to reduce the risk of such injuries by a proactive visualization technique, improving physician awareness of the esophagus location in the absence of or in addition to a reactive monitoring device such as a thermal probe. This is achieved by combining a graphical representation of the esophagus with live fluoroscopy. Toward this goal, we present an automated method to reconstruct and visualize a 3-D esophagus model from fluoroscopy image sequences acquired using different C-arm viewing directions. In order to visualize the esophagus under fluoroscopy, it is first biomarked by swallowing a contrast agent such as barium. Images obtained in this procedure are then used to automatically extract the 2-D esophagus silhouette and reconstruct a 3-D surface of the esophagus internal wall. Once the 3-D representation has been computed, it can be visualized using fluoroscopy overlay techniques. Compared to 3-D esophagus imaging using CT or C-arm CT, our proposed fluoroscopy method requires low radiation dose and enables a simpler workflow on geometry-calibrated standard C-arm systems.

This paper presents an object localization and tracking algorithm integrating audio and video-based object localization results. A face tracking algorithm and a microphone array are used to compute two single-modality speaker position estimates. These position estimates are then combined into a global position estimate using a decentralized Kalman filter. Experiments with a model railway show that such an approach yields more robust results for audio-visual object tracking than either modality by itself.

Object localization based on audio and video information is important for the analysis of dynamic scenes, such as video conferences or traffic situations. In this paper, we view the the dynamic audio-video object localization problem as a joint recursive estimation problem. It is solved using a decentralized Kalman filter fusing both audio and video position estimates. To better take into account different object maneuvers, multiple state-space equations are also incorporated. The result is a recursive multi-state multi-sensor estimator. Experiments show that it yields significantly improved joint position estimates compared to results achieved by using either an audio or a video system only.

This paper proposes a solution to the problem of robust speaker localization under adverse acoustic conditions. The approach is based on the classification of time delay estimates. Two classification techniques are investigated in detail: maximum likelihood (ML) classification and classification based on histogram comparison. Their performance under adverse acoustic conditions is compared to outcomes obtained with the traditional approach which uses time delay estimates directly to infer speaker positions. Experiments indicate that the ML classification method provides little improvement over the traditional method. On the other hand, using histogram classification, we can improve the probability of correct speaker localization by more than 60% compared to either the traditional approach or the ML classification technique.

Block-based feature extraction and clustering algorithms usually have to trade off between resolution and accuracy, as larger image block tends to generate more representative features at the expense of clustering resolution. We propose a new postprocessing technique for optimally combining the labeling results from overlapping image regions. This technique, the modified maximum a posteriori (MMAP) method, utilizes both local and global information from the neighborhood of the image region under consideration. Consequently, the resolution of the clustering becomes independent of the accuracy of the feature. Experimental results show dramatic improvement of the classification accuracy over methods that do not postprocess the clustering labels with the MMAP algorithm.

Image decomposition based on the discrete wavelet transform (DWT) has been proposed for efficient storage and progressive transmission of images for visual browsing in digital image libraries. Although the compression aspects of the DWT have been carefully researched, reconstruction errors due to corrupted wavelet coefficients have received less attention. In this paper we consider the problem of bit errors affecting uniformly quantized wavelet coefficients. The proposed method, which is based on a local image model, simultaneously detects and masks corrupted wavelet coefficients.