Active learning (AL) seeks to reduce annotation costs by selecting the most informative samples for labeling, making it particularly valuable in resource-constrained settings. However, traditional evaluation methods, which focus solely on final accuracy, fail to capture the full dynamics of the learning process. To address this gap, we propose PALM (Performance Analysis of Active Learning Models), a unified and interpretable mathematical model that characterizes AL trajectories through four key parameters: achievable accuracy, coverage efficiency, early-stage performance, and scalability. PALM provides a predictive description of AL behavior from partial observations, enabling the estimation of future performance and facilitating principled comparisons across different strategies. We validate PALM through extensive experiments on CIFAR-10/100 and ImageNet-50/100/200, covering a wide range of AL methods and self-supervised embeddings. Our results demonstrate that PALM generalizes effectively across datasets, budgets, and strategies, accurately predicting full learning curves from limited labeled data. Importantly, PALM reveals crucial insights into learning efficiency, data space coverage, and the scalability of AL methods. By enabling the selection of cost-effective strategies and predicting performance under tight budget constraints, PALM lays the basis for more systematic, reproducible, and data-efficient evaluation of AL in both research and real-world applications. The code is available at: https://github.com/juliamachnio/PALM.

Learning temporal patterns from multivariate longitudinal data is challenging especially in cases when data is sporadic, as often seen in, e.g., healthcare applications where the data can suffer from irregularity and asynchronicity as the time between consecutive data points can vary across features and samples, hindering the application of existing deep learning models that are constructed for complete, evenly spaced data with fixed sequence lengths. In this article, a novel deep learning-based model is developed for modeling multiple temporal features in sporadic data using an integrated deep learning architecture based on a recurrent neural network (RNN) unit and a continuous-time autoregressive (CAR) model. The proposed model, called CARRNN, uses a generalized discrete-time autoregressive (AR) model that is trainable end-to-end using neural networks modulated by time lags to describe the changes caused by the irregularity and asynchronicity. It is applied to time-series regression and classification tasks for Alzheimer's disease progression modeling, intensive care unit (ICU) mortality rate prediction, human activity recognition, and event-based digit recognition, where the proposed model based on a gated recurrent unit (GRU) in all cases achieves significantly better predictive performance than the state-of-the-art methods using RNNs, GRUs, and long short-term memory (LSTM) networks.

Optimal surface methods are a class of graph cut methods posing surface estimation as an n-ary ordered labeling problem. They are used in medical imaging to find interacting and layered surfaces optimally and in low order polynomial time. Representing continuous surfaces with discrete sets of labels, however, leads to discretization errors and, if graph representations are made dense, excessive memory usage. Limiting memory usage and computation time of graph cut methods are important and graphs that locally adapt to the problem has been proposed as a solution. Min-marginal energies computed using dynamic graph cuts offer a way to estimate solution uncertainty and these uncertainties have been used to decide where graphs should be adapted. Adaptive graphs, however, introduce extra parameters, complexity, and heuristics. We propose a way to use min-marginal energies to estimate continuous solution labels that does not introduce extra parameters and show empirically on synthetic and medical imaging datasets that it leads to improved accuracy. The increase in accuracy was consistent and in many cases comparable with accuracy otherwise obtained with graphs up to eight times denser, but with proportionally less memory usage and improvements in computation time.

Mammographic risk scoring has commonly been automated by extracting a set of handcrafted features from mammograms, and relating the responses directly or indirectly to breast cancer risk. We present a method that learns a feature hierarchy from unlabeled data. When the learned features are used as the input to a simple classifier, two different tasks can be addressed: i) breast density segmentation, and ii) scoring of mammographic texture. The proposed model learns features at multiple scales. To control the models capacity a novel sparsity regularizer is introduced that incorporates both lifetime and population sparsity. We evaluated our method on three different clinical datasets. Our state-of-the-art results show that the learned breast density scores have a very strong positive relationship with manual ones, and that the learned texture scores are predictive of breast cancer. The model is easy to apply and generalizes to many other segmentation and scoring problems.

In this paper, we propose a multi-scale, multi-kernel shape, compactly supported kernel bundle framework for stationary velocity field-based image registration (Wendland kernel bundle stationary velocity field, wKB-SVF). We exploit the possibility of directly choosing kernels to construct a reproducing kernel Hilbert space (RKHS) instead of imposing it from a differential operator. The proposed framework allows us to minimize computational cost without sacrificing the theoretical foundations of SVF-based diffeomorphic registration. In order to recover deformations occurring at different scales, we use compactly supported Wendland kernels at multiple scales and orders to parameterize the velocity fields, and the framework allows simultaneous optimization over all scales. The performance of wKB-SVF is extensively compared to the 14 non-rigid registration algorithms presented in a recent comparison paper. On both MGH10 and CUMC12 datasets, the accuracy of wKB-SVF is improved when compared to other registration algorithms. In a disease-specific application for intra-subject registration, atrophy scores estimated using the proposed registration scheme separates the diagnostic groups of Alzheimer's and normal controls better than the state-of-the-art segmentation technique. Experimental results show that wKB-SVF is a robust, flexible registration framework that allows theoretically well-founded and computationally efficient multi-scale representation of deformations and is equally well-suited for both inter- and intra-subject image registration.

To develop statistical methods for shapes with a tree-structure, we construct a shape space framework for tree-shapes and study metrics on the shape space. This shape space has singularities which correspond to topological transitions in the represented trees. We study two closely related metrics on the shape space, TED and QED. QED is a quotient euclidean distance arising naturally from the shape space formulation, while TED is the classical tree edit distance. Using Gromov's metric geometry, we gain new insight into the geometries defined by TED and QED. We show that the new metric QED has nice geometric properties that are needed for statistical analysis: Geodesics always exist and are generically locally unique. Following this, we can also show the existence and generic local uniqueness of average trees for QED. TED, while having some algorithmic advantages, does not share these advantages. Along with the theoretical framework we provide experimental proof-of-concept results on synthetic data trees as well as small airway trees from pulmonary CT scans. This way, we illustrate that our framework has promising theoretical and qualitative properties necessary to build a theory of statistical tree-shape analysis.

To achieve sparse parametrizations that allow intuitive analysis, we aim to represent deformation with a basis containing interpretable elements, and we wish to use elements that have the description capacity to represent the deformation compactly. To accomplish this, we introduce in this paper higher-order momentum distributions in the large deformation diffeomorphic metric mapping (LDDMM) registration framework. While the zeroth-order moments previously used in LDDMM only describe local displacement, the first-order momenta that are proposed here represent a basis that allows local description of affine transformations and subsequent compact description of nontranslational movement in a globally nonrigid deformation. The resulting representation contains directly interpretable information from both mathematical and modeling perspectives. We develop the mathematical construction of the registration framework with higher-order momenta, we show the implications for sparse image registration and deformation description, and we provide examples of how the parametrization enables registration with a very low number of parameters. The capacity and interpretability of the parametrization using higher-order momenta lead to natural modeling of articulated movement, and the method promises to be useful for quantifying ventricle expansion and progressing atrophy during Alzheimer's disease.

We present a fully automated framework for scoring a patient's risk of cardiovascular disease (CVD) and mortality from a standard lateral radiograph of the lumbar aorta. The framework segments abdominal aortic calcifications for computing a CVD risk score and performs a survival analysis to validate the score. Since the aorta is invisible on X-ray images, its position is reasoned from 1) the shape and location of the lumbar vertebrae and 2) the location, shape, and orientation of potential calcifications. The proposed framework follows the principle of Bayesian inference, which has several advantages in the complex task of segmenting aortic calcifications. Bayesian modeling allows us to compute CVD risk scores conditioned on the seen calcifications by formulating distributions, dependencies, and constraints on the unknown parameters. We evaluate the framework on two datasets consisting of 351 and 462 standard lumbar radiographs, respectively. Promising results indicate that the framework has potential applications in diagnosis, treatment planning, and the study of drug effects related to CVD.

This study presents a fully automatic, data-driven approach for texture-based quantitative analysis of chronic obstructive pulmonary disease (COPD) in pulmonary computed tomography (CT) images. The approach uses supervised learning where the class labels are, in contrast to previous work, based on measured lung function instead of on manually annotated regions of interest (ROIs). A quantitative measure of COPD is obtained by fusing COPD probabilities computed in ROIs within the lung fields where the individual ROI probabilities are computed using a k nearest neighbor (kNN ) classifier. The distance between two ROIs in the kNN classifier is computed as the textural dissimilarity between the ROIs, where the ROI texture is described by histograms of filter responses from a multi-scale, rotation invariant Gaussian filter bank. The method was trained on 400 images from a lung cancer screening trial and subsequently applied to classify 200 independent images from the same screening trial. The texture-based measure was significantly better at discriminating between subjects with and without COPD than were the two most common quantitative measures of COPD in the literature, which are based on density. The proposed measure achieved an area under the receiver operating characteristic curve (AUC) of 0.713 whereas the best performing density measure achieved an AUC of 0.598. Further, the proposed measure is as reproducible as the density measures, and there were indications that it correlates better with lung function and is less influenced by inspiration level.

We present methods to quantify the medial tibio- femoral (MTF) joint contact area (CA) and congruity index (CI) from low-field magnetic resonance imaging (MRI). Firstly, based on the segmented MTF cartilage compartments, we computed the contact area using the Euclidian distance transformation. The CA was defined as the area of the tibial superior surface and the femoral inferior surface that are less than a voxel width apart. Furthermore, the CI is computed point-by-point by assessing the first- and second-order general surface features over the contact area. Mathematically, it is the inverse distance between the local normal vectors (first-order features) scaled by the local normal curvatures (second-order features) along the local direction of principal knee motion in a local reference coordinate system formed by the directions of principal curvature and the surface normal vector. The abilities of the CA and the CI for diagnosing osteoarthritis (OA) at different levels (disease severity was assessed using the Kellgren and Lawrence Index, KL) were cross-validated on 288 knees at baseline. Longitudinal analysis was performed on 245 knees. The precision quantified on 31 scan-rescan pairs (RMS CV) for CA was 13.7% and for CI 7.5%. The CA increased with onset of the disease and then decreased with OA progression. The CI was highest in healthy and decreased with the onset of OA and further with disease progression. The CI showed an AUC of 0.69 (p < 0.0001) for separating KL = 0 and KL > 0. For separating KL < 1 or KL = 1 and KL > 1 knees, the AUC for CI was 0.73 (p < 0.0001). The CA demonstrated longitudinal responsiveness (SRM) at all stages of OA, whereas the CI did for advanced OA only. Eventually, the quantified CA and the CI might be suitable to help explaining OA onset, diagnosis of (early) OA, and measuring the efficacy of DMOADs in clinical trials.

The mean is often the most important statistic of a dataset as it provides a single point that summarizes the entire set. While the mean is readily defined and computed in Euclidean spaces, no commonly accepted solutions are currently available in more complicated spaces, such as spaces of tree-structured data. In this paper we study the notion of means, both generally in Gromov's CAT(0)-spaces (metric spaces of non-positive curvature), but also specifically in the space of tree-like shapes. We prove local existence and uniqueness of means in such spaces and discuss three different algorithms for computing means. We make an experimental evaluation of the three algorithms through experiments on three different sets of data with tree-like structure: a synthetic dataset, a leaf morphology dataset from images, and a set of human airway subtrees from medical CT scans. This experimental study provides great insight into the behavior of the different methods and how they relate to each other. More importantly, it also provides mathematically well-founded, tractable and robust “average trees”. This statistic is of utmost importance due to the ever-presence of tree-like structures in human anatomy, e.g., airways and vascularization systems.

Transform Domain Wyner-Ziv (TDWZ) video coding is an efficient Distributed Video coding solution providing new features such as low complexity encoding, by mainly exploiting the source statistics at the decoder based on the availability of decoder side information. The accuracy of the decoder side information has a major impact on the performance of TDWZ. In this paper, a novel multi-hypothesis based TDWZ video coding is presented to exploit the redundancy between multiple side information and the source information. The decoder used optical flow for side information calculation. Compared with the best available single estimation mode TDWZ, the proposed multi-hypothesis based TDWZ achieves robustly better Rate-Distortion (RD) performance and the overall improvement is up to 0.6 dB at high bitrate and up to 2 dB compared with the DISCOVER TDWZ video codec.

In this paper, we propose an energy-based algorithm for motion-compensated video super-resolution (VSR) targeted on upscaling of standard definition (SD) video to high-definition (HD) video. Since the motion (flow field) of the image sequence is generally unknown, we introduce a formulation for the joint estimation of a super-resolution (SR) sequence and its flow field. Via the calculus of variations, this leads to a coupled system of partial differential equations for image sequence and motion estimation. We solve a simplified form of this system and, as a by-product, we indeed provide a motion field for super-resolved sequences. To the best of our knowledge, computing super-resolved flows has not been done before. Most advanced SR methods found in literature cannot be applied to general video with arbitrary scene content and/or arbitrary optical flows, as it is possible with our simultaneous VSR method. A series of experiments shows that our method outperforms other VSR methods when dealing with general video input and that it continues to provide good results even for large scaling factors up to 8 × 8.

We have developed a breast coordinate system that is based on breast anatomy to register female breasts into a common coordinate frame in 2-D mediolateral (ML) or mediolateral oblique (MLO) view mammograms. The breasts are registered according to the location of the pectoral muscle and the nipple and the shape of the breast boundary because these are the most robust features independent of the breast size and shape. On the basis of these landmarks, we have constructed a nonlinear mapping between the parameter frame and the breast region in the mammogram. This mapping makes it possible to identify the corresponding positions and orientations among all of the ML or MLO mammograms, which facilitates an implicit use of the registration, i.e., no explicit image warping is needed. We additionally show how the coordinate transform can be used to extract Gaussian derivative features so that the feature positions and orientations are registered and extracted without nonlinearly deforming the images. We use the proposed breast coordinate transform in a cross-sectional breast cancer risk assessment study of 490 women, in which we attempt to learn breast cancer risk factors from mammograms that were taken prior to when the breast cancer became visible to a radiologist. The coordinate system provides both the relative position and orientation information on the breast region from which the features are derived. In addition, the coordinate system can be used in temporal studies to pinpoint anatomically equivalent locations between the mammograms of each woman and among the mammograms of all of the women in the study. The results of the cross-sectional study show that the classification into cancer and control groups can be improved by using the new coordinate system, compared to other systems evaluated. Comparisons were performed using the area-under-the-receiver-operating-characteristic-curve score. In general, the new coordinate system makes an accurate anatomical registration of breasts possible, which suggests its wide applicability wherever 2-D mammogram registration is required.

The estimation of covariance matrices is a crucial step in several statistical tasks. Especially when using few samples of a high dimensional representation of shapes, the standard maximum likelihood estimation (ML) of the covariance matrix can be far from the truth, is often rank deficient, and may lead to unreliable results. In this paper, we discuss regularization by prior knowledge using maximum a posteriori (MAP) estimates. We compare ML to MAP using a number of priors and to Tikhonov regularization. We evaluate the covariance estimates on both synthetic and real data, and we analyze the estimates' influence on a missing-data reconstruction task, where high resolution vertebra and cartilage models are reconstructed from incomplete and lower dimensional representations. Our results demonstrate that our methods outperform the traditional ML method and Tikhonov regularization.

This paper presents two distribution independent weighting schemes for k-Nearest-Neighbors (kNN). Applying the first scheme in a Leave-One-Out (LOO) setting corresponds to performing complete b-fold cross validation (b-CCV), while applying the second scheme corresponds to performing bootstrapping in the limit of infinite iterations. We demonstrate that the soft kNN errors obtained through b-CCV can be obtained by applying the weighted kNN in a LOO setting, and that the proposed weighting schemes can decrease the variance and improve the generalization of kNN in a CV setting.

This paper presents a new geometric framework for analysis of planar treelike shapes for applications such as shape matching, recognition and morphology, using the geometry of the space of treelike shapes. Mathematically, the shape space is given the structure of a stratified set which is a quotient of a normed vector space with a metric inherited from the vector space norm. We give examples of geodesic paths in tree-space corresponding to fundamental deformations of small trees, and discuss how these deformations are key building blocks for understanding deformations between larger trees.

In the present work we discuss the potential of recently developed classification algorithm, learning vector quantization (LVQ), for the analysis of laser induced fluorescence (LIF) spectra, recorded from normal and malignant bladder tissue samples. The algorithm is prototype based and inherently regularizing, which is desirable, for the LIF spectra because of its high dimensionality and features being settled at widely spaced intervals (sparseness). We discuss the effect of different parameters influencing the performance of LVQ in LIF data classification. Further, we compare and cross validate the classification accuracy of LVQ with other classifiers (eg. SVM and multi layer perception) for the same data set. Good agreement has been obtained between LVQ based classification of spectroscopy data and histopathology results which demonstrate the use of LVQ classifier in bladder cancer diagnosis.

We present a variational framework for deinterlacing that was originally used for inpainting and subsequently redeveloped for deinterlacing. From the framework, we derive a motion adaptive (MA) deinterlacer and a motion compensated (MC) deinterlacer and test them together with a selection of known deinterlacers. To illustrate the need for MC deinterlacing, the problem of details in motion (DIM) is introduced. It cannot be solved by MA deinterlacers or any simpler deinterlacers but only by MC deinterlacers. The major problem in MC deinterlacing is computing reliable optical flow [motion estimation (ME)] in interlaced video. We discuss a number of strategies for computing optical flows on interlaced video hoping to shed some light on this problem. We produce results on challenging real world video data with our variational MC deinterlacer that in most cases are indistinguishable from the ground truth.

We investigate the possibility to develop methodologies for assessing effect specific structural changes of the breast tissue using a general statistical machine learning framework. We present an approach of obtaining objective mammographic pattern measures quantifying a specific biological effect, such as hormone replacement therapy (HRT). We compare results using this approach to using standard density measures. We show that the proposed method can quantify both age related effects and effects caused by HRT. Age effects are significantly detected by our method where standard methodologies fail. The separation of HRT subpopulations using our approach is comparable to the best methodology, which is interactive.

A novel variational algorithm is developed for video inpainting. Within a Bayesian framework, using standard maximum a posteriori to variational formulation rationale, we derive a minimum energy formulation for the estimation of a reconstructed sequence as well as motion recovery. From the Euler-Lagrange Equations, we propose a full multiresolution algorithm in order to compute a good local minimizer for our energy and discuss its numerical implementation. Experimental results for synthetic as well as real sequences are presented. 1

What if eye trackers could be downloaded and used immediately with standard cameras connected to a computer, without the need for an expert to setup the system? This has already the case for head trackers, so why not for eye trackers?Using components off-the-shelf (COTS) for camera-based eye tracking tasks has many advantages, but it certainly introduces several new problems as less assumptions on the system can be made. As a consequence of using COTS the price for eye tracking devices can be reduced while increasing the accessibility of these systems. Eye tracking based on COTS holds potential for a large number of possible applications such as in the games industry and eye typing [Majaranta and Räihä 2002]. Different cameras may be purchased depending on the need and the amount of money the user is willing to spend on the camera. In this framework it is not possible to use IR light sources and other novel engineered devices as they cannot be bought in a common hardware store. Very little control over the cameras and the geometry of the setup can be expected. The methods employed for eye tracking should therefore be able to handle changes in light conditions and image defocusing and scale changes [Hansen and Pece 2003]. On the same token pan-and-tilt cameras cannot be used, thus forcing such systems to be passive. Figure 1 shows a possible setup of a COTS-based eye tracker. When designing systems for the general public, it is unrealistic to assume that people are able to do camera calibration and make accurate setups of camera, monitor and user. Since little is known about the setup, would this then require a vast amount of calibration points needed for gaze estimation? That is, how many calibration points are really needed? Obviously the more calibration points are used the better the chances are to be able to infer the mapping from the image to gaze direction. It would even be possible to sample the entire function space provided sufficiently many calibration points are given. From the point of view of the users, a low number of calibration points is preferred as calibration may be considered as a tedious procedure. Systems that require many calibration points for every session are therefore not likely to succeed. It is also important to know the accuracy in gaze determination when using COTS to determine their applicability for various tasks.

We propose a non-intrusive eye tracking system intended for the use of everyday gaze typing using web cameras. We argue that high precision in gaze tracking is not needed for on-screen typing due to natural language redundancy. This facilitates the use of low-cost video components for advanced multi-modal interactions based on video tracking systems. Robust methods are needed to track the eyes using web cameras due to the poor image quality. A realtime tracking scheme using a mean-shift color tracker and an Active Appearance Model of the eye is proposed. It is possible from this model to infer the state of the eye such as eye corners and the pupil location under scale and rotational changes.

The isophote curvature times the gradient magnitude to some power has been studied in the literature as a measure of cornerness in images, and stability in terms of sampling noise has been proposed by selecting corners in these measures at high scale and locating them at fine scale. We will examine the problem of tracking extrema of these measures in the linear scale-space and conclude that annihilations and creations generically occurs, so that corners in general cannot be tracked to arbitrarily fine/coarse scale. However, there are quantitatively differences, and the analysis indicates that isophote curvature times the gradient magnitude is best suited for binary images.

In optic flow based velocity estimation the image brightness constraint equation is used. However, for measurements performed at a certain scale, the brightness constraint equation does not apply. We therefore use a recently developed approach which reconciles optic flow and scale space theory. It specifically incorporates the scale (aperture) of image measurements, leading to a scheme which is essentially different from existing approaches. To obtain a unique velocity field, the data-derived information has to be augmented with physical knowledge. By keeping a strict separation between data-derived and external information, we can locally adapt or modify the user-supplied information without affecting the image-derived information. The two free scale parameters in time and space can be used for attentive vision (selecting particular velocities or objects) and to improve the reliability of velocity estimates.

Reconstruction of noise-corrupted surfaces may be stated as a (in general nonconvex) functional minimization problem. For functionals with quadratic data term, this paper addresses the criteria for such functionals to be convex, and the variational approach for minimization. I present two automatic and general methods of approximation with convex functionals based on Gaussian convolution. They are compared to the Blake-Zisserman graduated nonconvexity (GNC) method (1987) and Bilbro et al. (1992) and Geiger and Girosi's (1991) mean field annealing (MFA) of a weak membrane.

The reconstruction of noise corrupted surfaces can be inferred by methodologies such as Bayesian estimation and minimum description length. Both of these imply a formulation where the reconstruction minimizes a functional. Often this functional is non convex and the minimum cannot be found by simple gradient methods. The paper concerns functionals with quadratic data term, criteria for such functionals to be convex, and the variational approach of minimizing non convex functionals. Initial convexity of the approximating functional is considered to be a critical point. Two fully automatic methods of generating convex functionals are presented. They are based on Gaussian convolution and are compared to the Blake-Zisserman graduated non convexity (GNC) (A. Blake, A. Zisserman, 1987) and G.L. Bilbro et al. (1992) and D. Geiger and F. Girosi's (1991) mean field annealing (MFA) of the weak membrane.>

Regularization of ill-posed problems has been applied in various ways to surface reconstruction problems. Typically the smoothness term has been found in an ad hoc manner. In this work the a priori knowledge of the surface is used to construct the smoothness terms. If all surface normals are considered equally probable, different smoothness terms can be derived for different projections using Bayesian estimation. These smoothness terms implement discontinuous regularization by the Lorentzian estimator under orthographic and perspective projection, and imply a convex solution space, if the standard deviation of the noise is smaller than some quantity. Under stereo projection, occluded areas are punished dependent on the distance from the cameras, as they are more probable to exist on nearby objects than on distant objects. A general scheme of developing smoothness terms under assumptions of isotropy or anisotropy is outlined. 1

Noise-corrupted signals and images can be reconstructed by regularization. If discontinuities must be preserved in the reconstruction, a non-convex solution space is implied. The solution of minimum energy can be approximated by the Graduated Non-Convexity (GNC) algorithm. The GNC approximates the non-convex solution space by a convex solution space, and varies the solution space slowly towards the non-convex solution space. This work provides a method of finding the convex approximation to the solution space, and the convergent series of solution spaces. The same methodology can be used on a wide range of regularization schemes. The approximation of the solution space is carried out by a scale space extension of the smoothness measure, in which a coarse-to-fine analysis can be performed. It is proven, that this scale space extension yields a convex solution space. GNC by smoothness focusing is tested against the Blake-Zisserman formulation and is shown to yield better results in most cases. Furthermore, is it pointed out that Mean Field Annealing (MFA) of the weak string does not necessarily imply GNC, but behaves in a predictable and inexpedient manner. 1