Abstract. We consider the task of image reconstruction while simultaneously decomposing the reconstructed image into components with different features. A commonly used tool for this is a variational approach with an infimal convolution of appropriate functions as a regularizer. Especially for noise corrupted observations, incorporating these functionals into the classical method of Bregman iterations provides a robust method for obtaining an overall good approximation of the true image by stopping the iteration early according to a discrepancy principle. However, crucially, the quality of the separate components depends further on the proper choice of the regularization weights associated with the infimally convoluted functionals. Here, we propose the method of Nested Bregman iterations to improve a decomposition in a structured way. This allows for the transformation of the task of choosing the weights into the problem of stopping the iteration according to a meaningful criterion based on normalized cross-correlation. We discuss the well-definedness and the convergence behavior of the proposed method and illustrate its strength numerically with various image decomposition tasks employing infimal convolution functionals.

This paper introduces the first public large-scale, long-span dataset with sea turtle photographs captured in the wild -$SeaTurtleID2022$. The dataset contains 8729 photographs of 438 unique individuals collected within 13 years, making it the longest-spanned dataset for animal re-identification. Each photograph includes various annotations, e.g., identity, encounter timestamp, and body parts segmentation masks. Instead of a standard ’’random" split, the dataset allows for two realistic and ecologically motivated splits: (i) time-aware: a closed-set with training, validation, and test data from different days/years, and (ii) open-set: with new unknown individuals in test and validation sets. We show that time-aware splits are essential for benchmarking methods for re-identification, as random splits lead to performance overestimation. Furthermore, a baseline instance segmentation and re-identification performance over various body parts is provided. At last, an end-to-end system for sea turtle re-identification is proposed and evaluated. The proposed system based on Hybrid Task Cascade for head instance segmentation and ArcFace-trained feature-extractor achieved an accuracy of 86.8%.

In this paper, we present WildlifeDatasets – an open-source toolkit intended primarily for ecologists and computer-vision / machine-learning researchers. The WildlifeDatasets is written in Python, allows straightforward access to publicly available wildlife datasets, and provides a wide variety of methods for dataset pre-processing, performance analysis, and model fine-tuning. We show-case the toolkit in various scenarios and baseline experiments, including, to the best of our knowledge, the most comprehensive experimental comparison of datasets and methods for wildlife re-identification, including both local descriptors and deep learning approaches. Furthermore, we provide the first-ever foundation model for individual re-identification within a wide range of species – MegaDescriptor – that provides state-of-the-art performance on animal re-identification datasets and outperforms other pre-trained models such as CLIP and DINOv2 by a significant margin. To make the model available to the general public and to allow easy integration with any existing wildlife monitoring applications, we provide multiple MegaDescriptor flavors (i.e., Small, Medium, and Large) through the HuggingFace hub.

Abstract. We introduce a method for the fast estimation of data-adapted, spatially and temporally dependent regularization parameter-maps for variational image reconstruction, focusing on total variation (TV) minimization. The proposed approach is inspired by recent developments in algorithm unrolling using deep neural networks (NNs) and relies on two distinct subnetworks. The first subnetwork estimates the regularization parameter-map from the input data. The second subnetwork unrolls [Formula: see text] iterations of an iterative algorithm which approximately solves the corresponding TV-minimization problem incorporating the previously estimated regularization parameter-map. The overall network is then trained end-to-end in a supervised learning fashion using pairs of clean and corrupted data but crucially without the need for access to labels for the optimal regularization parameter-maps. We first prove consistency of the unrolled scheme by showing that the unrolled minimizing energy functional used for the supervised learning [Formula: see text]-converges, as [Formula: see text] tends to infinity, to the corresponding functional that incorporates the exact solution map of the TV-minimization problem. Then, we apply and evaluate the proposed method on a variety of large-scale and dynamic imaging problems with retrospectively simulated measurement data for which the automatic computation of such regularization parameters has been so far challenging using the state-of-the-art methods: a 2D dynamic cardiac magnetic resonance imaging (MRI) reconstruction problem, a quantitative brain MRI reconstruction problem, a low-dose computed tomography problem, and a dynamic image denoising problem. The proposed method consistently improves the TV reconstructions using scalar regularization parameters, and the obtained regularization parameter-maps adapt well to imaging problems and data by leading to the preservation of detailed features. Although the choice of the regularization parameter-maps is data-driven and based on NNs, the subsequent reconstruction algorithm is interpretable since it inherits the properties (e.g., convergence guarantees) of the iterative reconstruction method from which the network is implicitly defined.

In the context of image processing, we study a class of integral regularizers defined in terms of spatially inhomogeneous integrands that depend on general linear differential operators. Particularly, the spatial dependence is assumed to be only measurable. The setting is made rigorous by means of the theory of Radon measures and of suitable function spaces modeled on functions of bounded variation. We prove the lower semicontinuity of the functionals at stake and existence of minimizers for the corresponding variational problems. Then, we embed the latter into a bilevel scheme in order to automatically compute the regularization parameters. These parameters are considered to be spatially varying, thus allowing for good flexibility and preservation of details in the reconstructed image. After identifying a series of spatially inhomogeneous regularization functionals commonly used in image processing that are included in our framework, we substantiate its feasibility by performing numerical denoising examples in which the spatial dependence of the integrand is measurable. Specifically, we use Huber versions of the first and second order total variation (and their sum) with both the Huber and the regularization parameter being spatially varying. Notably, the spatially varying version of second order total variation produces high quality reconstructions when compared to regularizations of similar type, and the introduction of the low regularity spatially dependent Huber parameter leads to a further enhancement of the image details. We expect that our theoretical investigations and our numerical feasibility study will support future work on setting up schemes where general differential operators with spatially dependent coefficients will also be part of the optimization scheme.

Quantitative magnetic resonance imaging (qMRI) is concerned with estimating (in physical units) values of magnetic and tissue parameters, e.g., relaxation times $T_1$, $T_2$, or proton density $\rho$. Recently, in [Ma et al., Nature, 495 (2013), pp. 187--193], magnetic resonance fingerprinting (MRF) was introduced as a technique being capable of simultaneously recovering such quantitative parameters by using a two-step procedure: (i) given a probe, a series of magnetization maps are computed and then (ii) matched to (quantitative) parameters with the help of a precomputed dictionary which is related to the Bloch manifold. In this paper, we first put MRF and its variants into perspective with optimization and inverse problems to gain mathematical insights concerning identifiability of parameters under noise and interpretation in terms of optimizers. Motivated by the fact that the Bloch manifold is nonconvex and that the accuracy of the MRF-type algorithms is limited by the “discretization size” of the dictionary, a novel physics-based method for qMRI is proposed. In contrast to the conventional two-step method, our model is dictionary-free and is rather governed by a single nonlinear equation, which is studied analytically. This nonlinear equation is efficiently solved via robustified Newton-type methods. The effectiveness of the new method for noisy and undersampled data is shown both analytically and via extensive numerical examples, for which improvement over MRF and its variants is also documented.