Abstract Active learning aims to reduce the high labeling cost involved in training machine learning models on large datasets by efficiently labeling only the most informative samples. Recently, deep active learning has shown success on various tasks. However, the conventional evaluation schemes are either incomplete or below par. This study critically assesses various active learning approaches, identifying key factors essential for choosing the most effective active learning method. It includes a comprehensive guide to obtain the best performance for each case, in image classification and semantic segmentation. For image classification, the AL methods improve by a large-margin when integrated with data augmentation and semi-supervised learning, but barely perform better than the random baseline. In this work, we evaluate them under more realistic settings and propose a more suitable evaluation protocol. For semantic segmentation, previous academic studies focused on diverse datasets with substantial annotation resources. In contrast, data collected in many driving scenarios is highly redundant, and most medical applications are subject to very constrained annotation budgets. The study evaluates active learning techniques under various conditions including data redundancy, the use of semi-supervised learning, and differing annotation budgets. As an outcome of our study, we provide a comprehensive usage guide to obtain the best performance for each case.

Data privacy is typically particularly difficult to achieve in medical applications of machine learning, despite its importance in this area. The datasets are often small, which is why machine learning models such as neural networks tend to memorize information about the training data. This allows confidential and sensitive information about patients to be extracted from the model. Further challenging achieving data privacy is that the best possible utility must be ensured. In this work, we aim to detect tics based on video data of patients with Gilles de la Tourette syndrome. Facial landmarks were used as a lower-dimensional representation of the video data. Through membership inference attacks, we show that training a simple neural network directly on sensitive training data leaks information about the training data, and that this can be prevented by suitable pretraining on a large amount of unlabeled public data of healthy subjects. The proposed approach can not only reduce the attack accuracy to 52.15 %, but also achieves a high tic detection accuracy of 86.53 %.

We present a novel algorithm for the registration of pulmonary CT scans. Our method is designed for large respiratory motion by integrating sparse keypoint correspondences into a dense continuous optimization framework. The detection of keypoint correspondences enables robustness against large deformations by jointly optimizing over a large number of potential discrete displacements, whereas the dense continuous registration achieves subvoxel alignment with smooth transformations. Both steps are driven by the same normalized gradient fields data term. We employ curvature regularization and a volume change control mechanism to prevent foldings of the deformation grid and restrict the determinant of the Jacobian to physiologically meaningful values. Keypoint correspondences are integrated into the dense registration by a quadratic penalty with adaptively determined weight. Using a parallel matrix-free derivative calculation scheme, a runtime of about 5 min was realized on a standard PC. The proposed algorithm ranks first in the EMPIRE10 challenge on pulmonary image registration. Moreover, it achieves an average landmark distance of 0.82 mm on the DIR-Lab COPD database, thereby improving upon the state of the art in accuracy by 15%. Our algorithm is the first to reach the inter-observer variability in landmark annotation on this dataset.

In this paper a novel Large Deformation Diffeomorphic Metric Mapping (LDDMM) scheme is presented which has significantly lower computational and memory demands than standard LDDMM but achieves the same accuracy. We exploit the smoothness of velocities and transformations by using a coarser discretization compared to the image resolution. This reduces required memory and accelerates numerical optimization as well as solution of transport equations. Accuracy is essentially unchanged as the mismatch of transformed moving and fixed image is incorporated into the model at high resolution. Reductions in memory consumption and runtime are demonstrated for registration of lung CT images. State-of-the-art accuracy is shown for the challenging DIR-Lab chronic obstructive pulmonary disease (COPD) lung CT data sets obtaining a mean landmark distance after registration of 1.03 mm and the best average results so far. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Intra-interventional respiratory motion estimation has become vital for image-guided interventions, especially radiation therapy. While real-time tracking of highly discriminative landmarks like tumours and markers is possible with classic approaches (e.g. template matching), their robustness decreases when used with non-ionising imaging (4D MRI or US). Furthermore, they ignore the motion of neighbouring structures. We address these challenges by dividing the computation of dense deformable registration in two phases: First, a low-parametric full domain patient-specific motion model is learnt. Second, a sparse subset of feature locations is used to track motion locally, while the global motion patterns are constrained by the learnt model. In contrast to previous work, we optimise both objectives (local similarity and globally smooth motion) jointly using a coupled convex energy minimisation. This improves the tracking robustness and leads to a more accurate global motion estimation. The algorithm is computationally efficient and significantly outperforms classic template matching-based dense field estimation in 12 of 14 challenging 4D MRI and 4D ultrasound sequences. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This study presents a new visuo-haptic virtual reality (VR) training and planning system for percutaneous transhepatic cholangio-drainage (PTCD) based on partially segmented virtual patient models. We only use partially segmented image data instead of a full segmentation and circumvent the necessity of surface or volume mesh models. Haptic interaction with the virtual patient during virtual palpation, ultrasound probing and needle insertion is provided. Furthermore, the VR simulator includes X-ray and ultrasound simulation for image-guided training. The visualization techniques are GPU-accelerated by implementation in Cuda and include real-time volume deformations computed on the grid of the image data. Computation on the image grid enables straightforward integration of the deformed image data into the visualization components. To provide shorter rendering times, the performance of the volume deformation algorithm is improved by a multigrid approach. To evaluate the VR training system, a user evaluation has been performed and deformation algorithms are analyzed in terms of convergence speed with respect to a fully converged solution. The user evaluation shows positive results with increased user confidence after a training session. It is shown that using partially segmented patient data and direct volume rendering is suitable for the simulation of needle insertion procedures such as PTCD.

Statistical shape and appearance models are often based on the accurate identification of one-to-one correspondences in a training data set. At the same time, the determination of these corresponding landmarks is the most challenging part of such methods. Hufnagel etal developed an alternative method using correspondence probabilities for a statistical shape model. We propose the use of probabilistic correspondences for statistical appearance models by incorporating appearance information into the framework. A point-based representation is employed representing the image by a set of vectors assembling position and appearances. Using probabilistic correspondences between these multi-dimensional feature vectors eliminates the need for extensive preprocessing to find corresponding landmarks and reduces the dependence of the generated model on the landmark positions. Then, a maximum a-posteriori approach is used to derive a single global optimization criterion with respect to model parameters and observation dependent parameters, that directly affects shape and appearance information of the considered structures. Model generation and fitting can be expressed by optimizing the same criterion. The developed framework describes the modeling process in a concise and flexible mathematical way and allows for additional constraints as topological regularity in the modeling process. Furthermore, it eliminates the demand for costly correspondence determination. We apply the model for segmentation and landmark identification in hand X-ray images, where segmentation information is modeled as further features in the vectorial image representation. The results demonstrate the feasibility of the model to reconstruct contours and landmarks for unseen test images. Furthermore, we apply the model for tissue classification, where a model is generated for healthy brain tissue using 2D MRI slices. Applying the model to images of stroke patients the probabilistic correspondences are used to classify between healthy and pathological structures. The results demonstrate the ability of the probabilistic model to recognize healthy and pathological tissue automatically.

Modeling of respiratory motion has become increasingly important in various applications of medical imaging (e.g., radiation therapy of lung cancer). Current modeling approaches are usually confined to intra-patient registration of 3D image data representing the individual patient's anatomy at different breathing phases. We propose an approach to generate a mean motion model of the lung based on thoracic 4D computed tomography (CT) data of different patients to extend the motion modeling capabilities. Our modeling process consists of three steps: an intra-subject registration to generate subject-specific motion models, the generation of an average shape and intensity atlas of the lung as anatomical reference frame, and the registration of the subject-specific motion models to the atlas in order to build a statistical 4D mean motion model (4D-MMM). Furthermore, we present methods to adapt the 4D mean motion model to a patient-specific lung geometry. In all steps, a symmetric diffeomorphic nonlinear intensity-based registration method was employed. The Log-Euclidean framework was used to compute statistics on the diffeomorphic transformations. The presented methods are then used to build a mean motion model of respiratory lung motion using thoracic 4D CT data sets of 17 patients. We evaluate the model by applying it for estimating respiratory motion of ten lung cancer patients. The prediction is evaluated with respect to landmark and tumor motion, and the quantitative analysis results in a mean target registration error (TRE) of 3.3 ±1.6 mm if lung dynamics are not impaired by large lung tumors or other lung disorders (e.g., emphysema). With regard to lung tumor motion, we show that prediction accuracy is independent of tumor size and tumor motion amplitude in the considered data set. However, tumors adhering to non-lung structures degrade local lung dynamics significantly and the model-based prediction accuracy is lower in these cases. The statistical respiratory motion model is capable of providing valuable prior knowledge in many fields of applications. We present two examples of possible applications in radiation therapy and image guided diagnosis.

In this paper a variational approach for the combined segmentation and registration of temporal image sequences is presented. The purpose of the proposed method is to estimate respiratory-induced organ motion in temporal CT image sequences and to segment a structure of interest simultaneously. In this model the segmentation of all images in the sequences is obtained by finding a non-linear registration to an initial segmentation in a reference image. A dense non-linear displacement field is estimated using image intensities and segmentation information in the images. Both problems (registration and segmentation) are formulated in a joint variational approach and solved simultaneously. A validation of the combined registration and segmentation approach is presented and demonstrates that the simultaneous solution of both problems improves the segmentation performance over a sequential application of the registration and segmentation steps.