Pancreatic diseases are often only diagnosed at a late stage, and pancreatic cancer is the most feared due to a very high mortality. Abnormalities of the main pancreatic duct, such as blockages and dilatation, are often (early) signs of such pancreatic diseases, but are difficult to detect in standard Computed Tomography image series. Photon-Counting Computed Tomography with its higher resolution improves the detectability of this duct, allowing diagnostic assessment. A comprehensive visualization in a single view requires a centerline-based unfolding of the duct and pancreas. However, manual centerline annotation is tedious. To automate this process, we introduce a fully automated pipeline for pancreatic duct unfolding by robustly extracting the centerline using Dijkstra’s algorithm on a cost map derived from a segmentation probability map. The core contribution of this work lies in the processing of the data-driven cost map leading to a consistent centerline for generating CPR visualizations of the pancreas. To improve individual steps within the pipeline, we investigate further enhancements such as segmentation filtering and the topology-preserving skeleton recall loss. In the evaluation, we assess performance of our method on both ultra-high-resolution and regular PCCT images. We find that the centerline can be consistently extracted from both scan types, where the centerlines from the ultra-high resolution images exhibit a slightly lower median error of 0.58 mm compared to the 0.73 mm using the regular resolution.

In tumor therapy, estimating tumor growth is crucial to get an early information regarding tumor therapy response and, if neccessary, adapt therapy. We propose a novel deep learning based algorithm using deep convolutional sparse autoencoders to find a minimal representation of tumor shape and texture for colorectal liver metastases. Furthermore, we provide a prediction of future lesion growth based on single slice CT tumor images which prospectively can be used as a prognosis for physicians. The state of the art in tumor treatment assessment for solid tumors mainly uses tumor diameter in single CT slices as the treatment response criterion (RECIST). However, whereas the correlation between RECIST and final treatment outcome was shown to be significant, its effect size is still limited. With our approach we achieve a Matthews correlation coefficient of 52.0% in predicting tumor treatment response compared to 28.2% with radiologic assessment, as well as an AUC of 0.814 opposed to 0.698.

Dedicated visualization methods are among the most important tools of modern computer-aided medical applications. Reformation methods such as Multiplanar Reformation or Curved Planar Reformation have evolved as useful tools that facilitate diagnostic and therapeutic work. In this paper, we present a novel approach that can be seen as a generalization of Multiplanar Reformation to curved surfaces. The main concept is to generate reformatted medical volumes driven by the individual anatomical geometry of a specific patient. This process generates flat views of anatomical structures that facilitate many tasks such as diagnosis, navigation and annotation. Our reformation framework is based on a non-linear as-rigid-as-possible volumetric deformation scheme that uses generic triangular surface meshes as input. To manage inevitable distortions during reformation, we introduce importance maps which allow controlling the error distribution and improving the overall visual quality in areas of elevated interest. Our method seamlessly integrates with well-established concepts such as the slice-based inspection of medical datasets and we believe it can improve the overall efficiency of many medical workflows. To demonstrate this, we additionally present an integrated visualization system and discuss several use cases that substantiate its benefits.

Lymph nodes are assessed routinely in clinical practice and their size is followed throughout radiation or chemotherapy to monitor the effectiveness of cancer treatment. This paper presents a robust learning-based method for automatic detection and segmentation of solid lymph nodes from CT data, with the following contributions. First, it presents a learning based approach to solid lymph node detection that relies on marginal space learning to achieve great speedup with virtually no loss in accuracy. Second, it presents a computationally efficient segmentation method for solid lymph nodes (LN). Third, it introduces two new sets of features that are effective for LN detection, one that self-aligns to high gradients and another set obtained from the segmentation result. The method is evaluated for axillary LN detection on 131 volumes containing 371 LN, yielding a 83.0% detection rate with 1.0 false positive per volume. It is further evaluated for pelvic and abdominal LN detection on 54 volumes containing 569 LN, yielding a 80.0% detection rate with 3.2 false positives per volume. The running time is 5-20 s per volume for axillary areas and 15-40 s for pelvic. An added benefit of the method is the capability to detect and segment conglomerated lymph nodes.

In this paper we present a hierarchical, learning-based approach for automatic and accurate liver segmentation from 3D CT volumes. We target CT volumes that come from largely diverse sources (e.g., diseased in six different organs) and are generated by different scanning protocols (e.g., contrast and non-contrast, various resolution and position). Three key ingredients are combined to solve the segmentation problem. First, a hierarchical framework is used to efficiently and effectively monitor the accuracy propagation in a coarse-to-fine fashion. Second, two new learning techniques, marginal space learning and steerable features, are applied for robust boundary inference. This enables handling of highly heterogeneous texture pattern. Third, a novel shape space initialization is proposed to improve traditional methods that are limited to similarity transformation. The proposed approach is tested on a challenging dataset containing 174 volumes. Our approach not only produces excellent segmentation accuracy, but also runs about fifty times faster than state-of-the-art solutions [7, 9].

We present a new computational method for reconstructing a vector velocity field from scattered, pulsed-wave ultrasound Doppler data. The main difficulty is that the Doppler measurements are incomplete, for they do only capture the velocity component along the beam direction. We thus propose to combine measurements from different beam directions. However, this is not yet sufficient to make the problem well posed because 1) the angle between the directions is typically small and 2) the data is noisy and nonuniformly sampled. We propose to solve this reconstruction problem in the continuous domain using regularization. The reconstruction is formulated as the minimizer of a cost that is a weighted sum of two terms: 1) the sum of squared difference between the Doppler data and the projected velocities 2) a quadratic regularization functional that imposes some smoothness on the velocity field. We express our solution for this minimization problem in a B-spline basis, obtaining a sparse system of equations that can be solved efficiently. Using synthetic phantom data, we demonstrate the significance of tuning the regularization according to the a priori knowledge about the physical property of the motion. Next, we validate our method using real phantom data for which the ground truth is known. We then present reconstruction results obtained from clinical data that originate from 1) blood flow in carotid bifurcation and 2) cardiac wall motion.

We propose a novel method for image reconstruction from nonuniform samples with no constraints on their locations. We adopt a variational approach where the reconstruction is formulated as the minimizer of a cost that is a weighted sum of two terms: (1) the sum of squared errors at the specified points and (2) a quadratic functional that penalizes the lack of smoothness. We search for a solution that is a uniform spline and show how it can be determined by solving a large, sparse system of linear equations. We interpret the solution of our approach as an approximation of the analytical solution that involves radial basis functions and demonstrate the computational advantages of our approach. Using the two-scale relation for B-splines, we derive an algebraic relation that links together the linear systems of equations specifying reconstructions at different levels of resolution. We use this relation to develop a fast multigrid algorithm. We demonstrate the effectiveness of our approach on some image reconstruction examples.

The quantitative assessment of cardiac motion is a fundamental concept to evaluate ventricular malfunction. We present a new optical-flow-based method for estimating heart motion from two-dimensional echocardiographic sequences. To account for typical heart motions, such as contraction/expansion and shear, we analyze the images locally by using a local-affine model for the velocity in space and a linear model in time. The regional motion parameters are estimated in the least-squares sense inside a sliding spatiotemporal B-spline window. Robustness and spatial adaptability is achieved by estimating the model parameters at multiple scales within a coarse-to-fine multiresoluion framework. We use a wavelet-like algorithm for computing B-spline-weighted inner products and moments at dyadic scales to increase computational efficiency. In order to characterize myocardial contractility and to simplify the detection of myocardial dysfunction, the radial component of the velocity with respect to a reference point is color coded and visualized inside a time-varying region of interest. The algorithm was first validated on synthetic data sets that simulate a beating heart with a speckle-like appearance of echocardiograms. The ability to estimate motion from real ultrasound sequences was demonstrated by a rotating phantom experiment. The method was also applied to a set of in vivo echocardiograms from an animal study. Motion estimation results were in good agreement with the expert echocardiographic reading.

We propose a new spatio-temporal elastic registration algorithm for motion reconstruction from a series of images. The specific application is to estimate displacement fields from two-dimensional ultrasound sequences of the heart. The basic idea is to find a spatio-temporal deformation field that effectively compensates for the motion by minimizing a difference with respect to a reference frame. The key feature of our method is the use of a semi-local spatio-temporal parametric model for the deformation using splines, and the reformulation of the registration task as a global optimization problem. The scale of the spline model controls the smoothness of the displacement field. Our algorithm uses a multiresolution optimization strategy to obtain a higher speed and robustness. We evaluated the accuracy of our algorithm using a synthetic sequence generated with an ultrasound simulation package, together with a realistic cardiac motion model. We compared our new global multiframe approach with a previous method based on pairwise registration of consecutive frames to demonstrate the benefits of introducing temporal consistency. Finally, we applied the algorithm to the regional analysis of the left ventricle. Displacement and strain parameters were evaluated showing significant differences between the normal and pathological segments, thereby illustrating the clinical applicability of our method.

We introduce local weighted geometric moments that are computed from an image within a sliding window at multiple scales. When the window function satisfies a two-scale relation, we prove that lower order moments can be computed efficiently at dyadic scales by using a multiresolution wavelet-like algorithm. We show that B-splines are well-suited window functions because, in addition to being refinable, they are positive, symmetric, separable, and very nearly isotropic (Gaussian shape). We present three applications of these multiscale local moments. The first is a feature-extraction method for detecting and characterizing elongated structures in images. The second is a noise-reduction method which can be viewed as a multiscale extension of Savitzky-Golay filtering. The third is a multiscale optical-flow algorithm that uses a local affine model for the motion field, extending the Lucas-Kanade optical-flow method. The results obtained in all cases are promising.

We propose a novel multiresolution-multigrid based signal reconstruction method from arbitrarily spaced samples. The signal is reconstructed on a uniform grid using B-splines basis functions. The computation of spline weights is formulated as a variational problem. Specifically, we minimize a cost that is a weighted sum of two terms: (i) the sum of squared errors at the specified points; (ii) a quadratic functional that penalizes the lack of smoothness. The problem is equivalent to solving a very large system of linear equations, with the dimension equal to the number of grid points. We develop a computationally efficient multiresolution-multigrid scheme for solving the system. We demonstrate the method with image reconstruction from contour points.

We define multiscale moments that are estimated locally by analyzing an image through a sliding window at multiple scales. When the analysis window satisfies a two-scale relation, we prove that these moments can be computed very efficiently using a multiresolution wavelet-like algorithm. We also show that B-spline windows are best suited for this kind of analysis because, in addition to being refinable, they are positive, symmetric and very nearly isotropic. We present two applications of our method. The first is a feature extraction method for detecting strands of DNA in noisy cryoelectron-micrographs. The second is an extension of the Lucas-Kanade optical flow algorithm (see Lucas, B. and Kanade, T., Proc. DARPA IU Workshop, p.121-30, 1981) that assumes a local affine model of the motion field. The results obtained in both cases are very promising.