Self-supervised learning (SSL) methods have achieved remarkable success in learning image representations allowing invariances in them — but therefore discarding transformation information that some computer vision tasks actually require. While recent approaches attempt to address this limitation by learning equivariant features using linear operators in feature space, they impose restrictive assumptions that constrain flexibility and generalization. We introduce a weaker definition for the transformation relation between image and feature space denoted as equivariance-coherence. We propose a novel SSL auxillary task that learns equivariance-coherent representations through intermediate transformation reconstruction, which can be integrated with existing joint embedding SSL methods. Our key idea is to reconstruct images at intermediate points along transformation paths, e.g. when training on 30° rotations, we reconstruct the 10° and 20° rotation states. Reconstructing intermediate states requires the transformation information used in augmentations, rather than suppressing it, and therefore fosters features containing the augmented transformation information. Our method decomposes feature vectors into invariant and equivariant parts, training them with standard SSL losses and reconstruction losses, respectively. We demonstrate substantial improvements on synthetic equivariance benchmarks while maintaining competitive performance on downstream tasks requiring invariant representations. The approach seamlessly integrates with existing SSL methods (iBOT, DINOv2) and consistently enhances performance across diverse tasks, including segmentation, detection, depth estimation, and video dense prediction. Our framework provides a practical way for augmenting SSL methods with equivariant capabilities while preserving invariant performance.

Monocular depth estimation (MDE) is a challenging task in computer vision, often hindered by the cost and scarcity of high-quality labeled datasets. We tackle this challenge using auxiliary datasets from related vision tasks for an alternating training scheme with a shared decoder built on top of a pre-trained vision foundation model, while giving a higher weight to MDE. Through extensive experiments we demonstrate the benefits of incorporating various in-domain auxiliary datasets and tasks to improve MDE quality on average by ~ 11 %. Our experimental analysis shows that auxiliary tasks have different impacts, confirming the importance of task selection, highlighting that quality gains are not achieved by merely adding data. Remarkably, our study reveals that using semantic segmentation datasets as Multi-Label Dense Classification (MLDC) often results in additional quality gains. Lastly, our method significantly improves the data efficiency for the considered MDE datasets, enhancing their quality while reducing their size by at least 80%. This paves the way for using auxiliary data from related tasks to improve MDE quality despite limited availability of high-quality labeled data. Code is available at https://jugit.fz-juelich.de/ias-8/mdeaux.

To cope with the rapid growth in available data, the efficiency of data analysis and machine learning libraries has recently received increased attention. Although great advancements have been made in traditional array-based computations, most are limited by the resources available on a single computation node. Consequently, novel approaches must be made to exploit distributed resources, e.g. distributed memory architectures. To this end, we introduce HeAT, an array-based numerical programming framework for large-scale parallel processing with an easy-to-use NumPy-like API. HeAT utilizes PyTorch as a node-local eager execution engine and distributes the workload on arbitrarily large high-performance computing systems via MPI. It provides both low-level array computations, as well as assorted higher-level algorithms. With HeAT, it is possible for a NumPy user to take full advantage of their available resources, significantly lowering the barrier to distributed data analysis. When compared to similar frameworks, HeAT achieves speedups of up to two orders of magnitude.

High-angular resolution diffusion imaging (HARDI) is a magnetic resonance technique estimating the direction of self-diffusion of water molecules in biological tissue. HARDI encodes at each pixel (voxel) the orientation distribution function (ODF) of water diffusion molecules, i.e. the probability distribution function of finding a water molecule which moved in a certain direction during the observation time. As a consequence ODF images differ from usual gray scale images with respect to their underlying geometry as well as with respect to their error distribution. We present a Bayesian estimator for ODF images considering these differences. To this end, we derive a likelihood function based on the Rician distribution of the NMR signals and propose prior distributions considering ODFs as Riemanian manifolds. Utilizing properties of spherical harmonics and the square root representation of ODFs allows us to effectively reconstruct and regularize ODF images in one step within this Riemannian framework. Experiments demonstrate the merits of our approach on synthetic as well as on real data.

Relations between deterministic (e.g. variational or PDE based methods) and Bayesian inference have been known for a long time. However, a classification of deterministic approaches into those methods which can be handled within a Bayesian framework and those with no such statistical counterpart is still missing in literature. After providing such taxonomy, we present a Bayesian framework for embedding the former ones into a statistical context allowing to equip them with advantages of probabilistic estimation theory. A stochastic point of view allows us (1) to learn influence functions and derivative filter, (2) adapt diffusion and regularization approaches to changes in the image characteristics (e.g. varying noise levels), and (3) to estimate error bounds on the solution. For the latter ones we present alternative learning schemes also allowing their parameters to be related to the image statistics such that hand tuning becomes dispensable. We demonstrate that a statistical point of view on diffusion and regularization schemes leads to image denoising performances comparable with state of the art Markov random field approaches while being computationally much more effective.

Diffusion tensor magnetic resonance imaging (DT-MRI) is a non-invasive imaging technique allowing to estimate the molecular self-diffusion tensors of water within surrounding tissue. Due to the low signal-to-noise ratio of magnetic resonance images, reconstructed tensor images usually require some sort of regularization in a post-processing step. Previous approaches are either suboptimal with respect to the reconstructing or regularization step. This paper presents a Bayesian approach for simultaneous reconstructing and regularization of DT-MR images that allows to resolve the disadvantages of previous approaches. To this end, estimation theoretical concepts are generalized to tensor valued images that are considered as Riemannian manifolds. Doing so allows us to derive a maximum a posterior estimator of the tensor image that considers both the statistical characteristics of the Rician noise occurring in MR images as well as the nonlinear structure of tensor valued images. Experiments on synthetic data as well as real DT-MRI data validate the advantage of considering both statistical as well as geometrical characteristics of DT-MRI.

This paper presents a generalized theoretical framework for designing accurate steerable filters for orientation estimation. We derive the necessary properties of orientation estimation filters in their most general form. Based on our framework, we implemented a highly angular-specific filter. Numerical experiments show the enhanced accuracy of our proposed filter as compared with existing filters.

Global optical flow estimation methods based on variational calculus contain a regularization parameter which controls the tradeoff between the different constraints on the optical flow field. The counterpart to the regularization parameter are the hyper-parameters in the Bayesian framework. These hyper-parameters have distinct physical meanings and thus can be inferred from the observable data. We derive a combined marginal maximum likelihood/maximum a posteriori (MML/MAP) estimator for simultaneously estimating hyper-parameters and optical flow for all differential variational approaches directly from the observed signal without any prior knowledge of the optical flow. Experiments demonstrate the performance of this optimization technique and show that the choice of the regularization parameter is an essential key-point in order to obtain precise motion estimation.