Abstract Small load carriers (SLCs) are standardized reusable containers used to transport and protect customer goods in many manufacturers. Throughout the life cycle of the SLCs, they will be collected, manually checked for defects (wear, cracks, and residue on the surface), and cleaned by specialized logistic companies. Human operators in small to medium-sized companies manually evaluate the defects due to the variety and degree of possible defects and varying customer needs. This manual evaluation is not scalable and prone to errors. This work aims to fill this gap by proposing a computer vision system that can recognize the SLC type for inventory management and perform defect detection automatically. First, we develop a camera portal, consisting of standard components, that capture the relevant surfaces of the SLC. A labeled dataset of 17,530 images of 34 different SLCs with their defect status was recorded using this camera portal. We trained a classification model (ConvNeXt) using our dataset to predict the different types of SLCs achieving 100% class prediction accuracy. For defect detection, we explore eight state-of-the-art (SOTA) anomaly detection models that achieved high rankings in the MVTec industrial anomaly detection benchmark. These models are trained using default hyperparameters and the two highest-scoring models were chosen and fine-tuned. The best-fine-tuned models based on “Area under the Receiver Operating Characteristic Curve (AUROC)” are PatchCore (0.811) and DRAEM (0.748). These results indicate that there is still potential for improvement in the automation of defect detection of SLCs.

Mumford--Shah models are well-established and powerful variational tools for the regularization of noisy data. In the case of images this includes regularizing both the edge set as well as the image values itself. Thus, these models may be used as a basis for a segmentation pipeline or for smoothing the data. In this paper we consider higher order Mumford--Shah functionals which penalize the deviation from piecewise polynomials instead of piecewise constant functions as first order Mumford--Shah functionals do. Minimizing Mumford--Shah functionals, which are nonsmooth and nonconvex functionals, are NP hard problems. Compared with first order Mumford--Shah functionals, numerically solving higher order models is even more challenging, and in contrast to work on more theoretical aspects there are only very few works dealing with the algorithmic side. In this paper, we propose a new algorithmic framework for second order Mumford--Shah regularization. It is based on a proposed reformulation of higher order Mumford--Shah problems in terms of Taylor jets and a corresponding discretization. Using an ADMM approach, we split the discrete jet-based problem into subproblems which we can solve efficiently, noniteratively, and exactly. We derive numerically stable and fast solvers for these subproblems. In summary, we obtain an efficient overall algorithm. Our method requires a priori knowledge on neither the gray or color levels nor the shape of the discontinuity set of a solution. We demonstrate the applicability of the proposed methods in various numerical experiments. In particular, we quantitatively and qualitatively compare the proposed scheme with the algorithms proposed in the literature.

We propose an algorithm to efficiently compute approximate solutions of the piecewise affine Mumford-Shah model. The algorithm is based on a novel reformulation of the underlying optimization problem in terms of Taylor jets. A splitting approach leads to linewise segmented jet estimation problems for which we propose an exact and efficient solver. The proposed method has the combined advantages of prior algorithms: it directly yields a partition, it does not need an initialization procedure, and it is highly parallelizable. The experiments show that the algorithm has lower computation times and that the solutions often have lower functional values than the state-of-the-art.

Three-dimensional freehand imaging techniques are gaining wider adoption due to their ?exibility and cost ef?ciency. Typical examples for such a combination of a tracking system with an imaging device are freehand SPECT or freehand 3D ultrasound. However, the quality of the resulting image data is heavily dependent on the skill of the human operator and on the level of noise of the tracking data. The latter aspect can introduce blur or strong artifacts, which can signi?cantly hamper the interpretation of image data. Unfortunately, the most commonly used tracking systems to date, i.e. optical and electromagnetic, present a trade-off between invading the surgeon's workspace (due to line-of-sight requirements) and higher levels of noise and sensitivity due to the interference of surrounding metallic objects. In this work, we propose a novel approach for total variation regularization of data from tracking systems (which we term pose signals) based on a variational formulation in the manifold of Euclidean transformations. The performance of the proposed approach was evaluated using synthetic data as well as real ultrasound sweeps executed on both a Lego phantom and human anatomy, showing signi?cant improvement in terms of tracking data quality and compounded ultrasound images. Source code can be found at https://github.com/IFL-CAMP/pose_regularization.

In many machine learning tasks it is desirable that a model's prediction transforms in an equivariant way under transformations of its input. Convolutional neural networks (CNNs) implement translational equivariance by construction; for other transformations, however, they are compelled to learn the proper mapping. In this work, we develop Steerable Filter CNNs (SFCNNs) which achieve joint equivariance under translations and rotations by design. The proposed architecture employs steerable filters to efficiently compute orientation dependent responses for many orientations without suffering interpolation artifacts from filter rotation. We utilize group convolutions which guarantee an equivariant mapping. In addition, we generalize He's weight initialization scheme to filters which are defined as a linear combination of a system of atomic filters. Numerical experiments show a substantial enhancement of the sample complexity with a growing number of sampled filter orientations and confirm that the network generalizes learned patterns over orientations. The proposed approach achieves state-of-the-art on the rotated MNIST benchmark and on the ISBI 2012 2D EM segmentation challenge.

Median filtering is among the most utilized tools for smoothing real-valued data, as it is robust, edge-preserving, value-preserving, and yet can be computed efficiently. For data living on the unit circle, such as phase data or orientation data, a filter with similar properties is desirable. For these data, there is no unique means to define a median; so we discuss various possibilities. The arc distance median turns out to be the only variant which leads to robust, edge-preserving and value-preserving smoothing. However, there are no efficient algorithms for filtering based on the arc distance median. Here, we propose fast algorithms for filtering of signals and images with values on the unit circle based on the arc distance median. For non-quantized data, we develop an algorithm that scales linearly with the filter size. The runtime of our reference implementation is only moderately higher than the Matlab implementation of the classical median filter for real-valued data. For quantized data, we obtain an algorithm of constant complexity w.r.t. the filter size. We demonstrate the performance of our algorithms for real life data sets: phase images from interferometric synthetic aperture radar, planar flow fields from optical flow, and time series of wind directions.

In this paper we introduce the notion of second-order total generalized variation (TGV) regularization for manifold-valued data in a discrete setting. We provide an axiomatic approach to formalize reasonable generalizations of TGV to the manifold setting and present two possible concrete instances that fulfill the proposed axioms. We provide well-posedness results and present algorithms for a numerical realization of these generalizations to the manifold setup. Further, we provide experimental results for synthetic and real data to further underpin the proposed generalization numerically and show its potential for applications with manifold-valued data.

Current algorithmic approaches for piecewise affine motion estimation are based on alternating motion segmentation and estimation. We propose a new method to estimate piecewise affine motion fields directly without intermediate segmentation. To this end, we reformulate the problem by imposing piecewise constancy of the parameter field, and derive a specific proximal splitting optimization scheme. A key component of our framework is an efficient one-dimensional piecewise-affine estimator for vector-valued signals. The first advantage of our approach over segmentation-based methods is its absence of initialization. The second advantage is its lower computational cost which is independent of the complexity of the motion field. In addition to these features, we demonstrate competitive accuracy with other piecewise-parametric methods on standard evaluation benchmarks. Our new regularization scheme also outperforms the more standard use of total variation and total generalized variation.

Features that capture well the textural patterns of a certain class of images are crucial for the performance of texture segmentation methods. The manual selection of features or designing new ones can be a tedious task. Therefore, it is desirable to automatically adapt the features to a certain image or class of images. Typically, this requires a large set of training images with similar textures and ground truth segmentation. In this work, we propose a framework to learn features for texture segmentation when no such training data is available. The cost function for our learning process is constructed to match a commonly used segmentation model, the piecewise constant Mumford-Shah model. This means that the features are learned such that they provide an approximately piecewise constant feature image with a small jump set. Based on this idea, we develop a two-stage algorithm which first learns suitable convolutional features and then performs a segmentation. We note that the features can be learned from a small set of images, from a single image, or even from image patches. The proposed method achieves a competitive rank in the Prague texture segmentation benchmark, and it is effective for segmenting histological images.

We develop a fast algorithm for segmenting 3D images from linear measurements based on the Potts model (or piecewise constant Mumford-Shah model). To that end, we first derive suitable space discretizations of the 3D Potts model, which are capable of dealing with 3D images defined on non-cubic grids. Our discretization allows us to utilize a specific splitting approach, which results in decoupled subproblems of moderate size. The crucial point in the 3D setup is that the number of independent subproblems is so large that we can reasonably exploit the parallel processing capabilities of the graphics processing units (GPUs). Our GPU implementation is up to 18 times faster than the sequential CPU version. This allows to process even large volumes in acceptable runtimes. As a further contribution, we extend the algorithm in order to deal with non-negativity constraints. We demonstrate the efficiency of our method for combined image deconvolution and segmentation on simulated data and on real 3D wide field fluorescence microscopy data.

Magnetic particle imaging (MPI) is an emerging medical imaging modality which is based on the non-linear response of magnetic nanoparticles to an applied magnetic field. It is an important feature of MPI that even fast dynamic processes can be captured for 3D volumes. The high temporal resolution in turn leads to large amounts of data which have to be handled efficiently. But as the system matrix of MPI is non-sparse, the image reconstruction gets computationally demanding. Therefore, currently only basic image reconstruction methods such as Tikhonov regularization are used. However, Tikhonov regularization is known to oversmooth edges in the reconstructed image and to have only a limited noise reducing effect. In this work, we develop an efficient edge preserving and noise reducing reconstruction method for MPI. As regularization model, we propose to use the nonnegative fused lasso model, and we devise a discretization that is adapted to the acquisition geometry of the preclinical MPI scanner considered in this work. We develop a customized solver based on a generalized forward-backward scheme which is particularly suitable for the dense and not well-structured system matrices in MPI. Already a non-optimized prototype implementation processes a 3D volume within a few seconds so that processing several frames per second seems amenable. We demonstrate the improvement in reconstruction quality over the state-of-the-art method in an experimental medical setup for an in-vitro angioplasty of a stenosis.

This paper presents a method for the simultaneous segmentation and regularization of a series of shapes from a corresponding sequence of images. Such series arise as time series of 2D images when considering video data, or as stacks of 2D images obtained by slicewise tomographic reconstruction. We first derive a model where the regularization of the shape signal is achieved by a total variation prior on the shape manifold. The method employs a modified Kendall shape space to facilitate explicit computations together with the concept of Sobolev gradients. For the proposed model, we derive an efficient and computationally accessible splitting scheme. Using a generalized forward-backward approach, our algorithm treats the total variation atoms of the splitting via proximal mappings, whereas the data terms are dealt with by gradient descent. The potential of the proposed method is demonstrated on various application examples dealing with 3D data. We explain how to extend the proposed combined approach to shape fields which, for instance, arise in the context of 3D+t imaging modalities, and show an application in this setup as well.

In this paper, we consider combined TV denoising and diffusion tensor fitting in DTI using the affine-invariant Riemannian metric on the space of diffusion tensors. Instead of first fitting the diffusion tensors, and then denoising them, we define a suitable TV type energy functional which incorporates the measured DWIs (using an inverse problem setup) and which measures the nearness of neighboring tensors in the manifold. To approach this functional, we propose generalized forward- backward splitting algorithms which combine an explicit and several implicit steps performed on a decomposition of the functional. We validate the performance of the derived algorithms on synthetic and real DTI data. In particular, we work on real 3D data. To our knowledge, the present paper describes the first approach to TV regularization in a combined manifold and inverse problem setup.

This paper introduces the concept of shape signals, i.e., series of shapes which have a natural temporal or spatial ordering, as well as a variational formulation for the regularization of these signals. The proposed formulation can be seen as the shape-valued generalization of the Rudin-Osher-Fatemi (ROF) functional for intensity images. We derive a variant of the classical finite-dimensional representation of Kendall, but our framework is generic in the sense that it can be combined with any shape space. This representation allows for the explicit computation of geodesics and thus facilitates the efficient numerical treatment of the variational formulation by means of the cyclic proximal point algorithm. Similar to the ROF-functional, we demonstrate experimentally that ℓ1-type penalties both for data fidelity term and regularizer perform best in regularizing shape signals. Finally, we show applications of our method to shape signals obtained from synthetic, photometric, and medical data sets.

We present a method for the unsupervised segmentation of textured images using Potts functionals, which are a piecewise-constant variant of the Mumford and Shah functionals. We propose a minimization strategy based on the alternating direction method of multipliers and dynamic programming. The strategy allows us to process large feature spaces because the computational cost grows only linearly in the feature dimension. In particular, our algorithm has more favorable computational costs for high-dimensional data than graph cuts. Our feature vectors are based on monogenic curvelets. They incorporate multiple resolutions and directional information. The advantage over classical curvelets is that they yield smoother amplitudes due to the envelope effect of the monogenic signal.

We propose a fast splitting approach to the classical variational formulation of the image partitioning problem, which is frequently referred to as the Potts or piecewise constant Mumford--Shah model. For vector-valued images, our approach is significantly faster than the methods based on graph cuts and convex relaxations of the Potts model which are presently the state-of-the-art. The computational costs of our algorithm only grow linearly with the dimension of the data space which contrasts the exponential growth of the state-of-the-art methods. This allows us to process images with high-dimensional codomains such as multispectral images. Our approach produces results of a quality comparable with that of graph cuts and the convex relaxation strategies, and we do not need an a priori discretization of the label space. Furthermore, the number of partitions has almost no influence on the computational costs, which makes our algorithm also suitable for the reconstruction of piecewise constant (color or vectorial) images.

We consider total variation (TV) minimization for manifold-valued data. We propose a cyclic proximal point algorithm and a parallel proximal point algorithm to minimize TV functionals with $\ell^p$-type data terms in the manifold case. These algorithms are based on iterative geodesic averaging which makes them easily applicable to a large class of data manifolds. As an application, we consider denoising images which take their values in a manifold. We apply our algorithms to diffusion tensor images and interferometric SAR images as well as sphere- and cylinder-valued images. For the class of Cartan--Hadamard manifolds (which includes the data space in diffusion tensor imaging) we show the convergence of the proposed TV minimizing algorithms to a global minimizer.

We reconsider the continuous curvelet transform from a signal processing point of view. We show that the analyzing elements of the curvelet transform, the curvelets, can be understood as analytic signals in the sense of the partial Hilbert transform. We then generalize the usual curvelets by the monogenic curvelets, which are analytic signals in the sense of the Riesz transform. They yield a new transform, called the monogenic curvelet transform. This transform has the useful property that it behaves at the fine scales like the usual curvelet transform and at the coarse scales like the monogenic wavelet transform. In particular, the new transform is highly anisotropic at the fine scales and yields a well-interpretable amplitude/phase decomposition of the transform coefficients over all scales. We illustrate the advantage of this new directional multiscale amplitude/phase decomposition for the estimation of directional regularity.

In this article, we reconsider the continuous curvelet transform from a signal processing point of view. We show that the analyzing elements of the curvelet transform, the curvelets, can be understood as analytic signals in the sense of the partial Hilbert transform. We then replace the usual curvelets by the monogenic curvelets, which are analytic signals in the sense of the Riesz transform. They yield a new transform, called the monogenic curvelet transform, which has the interesting property that it behaves at the fine scales like the usual curvelet transform and at the coarse scales like the monogenic wavelet transform. In particular, the new transform is highly anisotropic at the fine scales and yields a well-interpretable amplitude/phase decomposition of the transform coefficients over all scales.

We consider an extension of the 1-D concept of analytical wavelet to n-D which is by construction compatible with rotations. This extension, called a monogenic wavelet, yields a decomposition of the wavelet coefficients into amplitude, phase, and phase direction. The monogenic wavelet is based on the hypercomplex monogenic signal which is defined using Riesz transforms and perfectly isotropic wavelets frames. Employing the new concept of Clifford frames, we can show that the monogenic wavelet generates a wavelet frame. Furthermore, this approach yields wavelet frames that are steerable with respect to direction. Applications to descreening and contrast enhancement illustrate the versatility of this approach to image analysis and reconstruction.