Abstract. Scattering networks yield powerful and robust hierarchical image descriptors which do not require lengthy training and which work well with very few training data. However, they rely on sampling the scale dimension. Hence, they become sensitive to scale variations and are unable to generalize to unseen scales. In this work, we define an alternative feature representation based on the Riesz transform. We detail and analyze the mathematical foundations behind this representation. In particular, it inherits scale equivariance from the Riesz transform and completely avoids sampling of the scale dimension. Additionally, the number of features in the representation is reduced by a factor four compared to scattering networks. Nevertheless, our representation performs comparably well for texture classification with an interesting addition: scale equivariance. Our method yields very good performance when dealing with scales outside of those covered by the training dataset. The usefulness of the equivariance property is demonstrated on the digit classification task, where accuracy remains stable even for scales four times larger than the one chosen for training. As a second example, we consider classification of textures. Finally, we show how this representation can be used to build hybrid deep learning methods that are more stable to scale variations than standard deep networks.

The Gaussian kernel and its derivatives have already been employed for Convolutional Neural Networks in several previous works. Most of these papers proposed to compute filters by linearly combining one or several bases of fixed or slightly trainable Gaussian kernels with or without their derivatives. In this article, we propose a high-level configurable layer based on anisotropic, oriented and shifted Gaussian derivative kernels which generalize notions encountered in previous related works while keeping their main advantage. The results show that the proposed layer has competitive performance compared to previous works and that it can be successfully included in common deep architectures such as VGG16 for image classification and U-net for image segmentation.

Symmetry is present in many tasks in computer vision, where the same class of objects can appear transformed, e.g. rotated due to different camera orientations, or scaled due to perspective. The knowledge of such symmetries in data coupled with equivariance of neural networks can improve their generalization to new samples. Differential invariants are equivariant operators computed from the partial derivatives of a function. In this paper we use differential invariants to define equivariant operators that form the layers of an equivariant neural network. Specifically, we derive invariants of the Special Euclidean Group SE(2), composed of rotations and translations, and apply them to construct a SE(2)-equivariant network, called SE(2) Differential Invariants Network (SE2DINNet). The network is subsequently tested in classification tasks which require a degree of equivariance or invariance to rotations. The results compare positively with the state-of-the-art, even though the proposed SE2DINNet has far less parameters than the compared models.

Empirical mode decomposition (EMD) is a fully data driven method for multiscale decomposing signals into a set of components known as intrinsic mode functions. EMD is based on lower and upper envelopes of the signal in an iterated decomposition scheme. In this paper, we put forward a simple yet effective method to learn EMD from data by means of morphological operators. We propose an end-to-end framework by incorporating morphological EMD operators into deeply learned representations, trained using standard backpropagation principle and gradient descent-based optimization algorithms. Three generalizations of morphological EMD are proposed: (a) by varying the family of structuring functions, (b) by varying the pair of morphological operators used to calculate the envelopes, and (c) by considering a convex sum of envelopes instead of the mean point used in classical EMD. We discuss in particular the invariances that are induced by the morphological EMD representation. Experimental results on supervised classification of hyperspectral images by one-dimensional convolutional networks demonstrate the interest of our method.

Classical morphological filters suffer from well performing in a noisy environment, and intrinsic image structures are not taken into account. The authors propose here an alternative to overcome such weaknesses, by properly using robust shock filters and inhomogeneity. Thus, they obtain multiscale morphological operators by using image edge functions as local weights in inhomogeneous Hamiltonians in classical multiscale dilations/erosions formulated with partial differential equations (PDEs). They provide the equivalent sup–inf‐based formulations, and derive sharpening/enhancement methods. In addition, they establish the PDE associated with the asymptotical iterations of the proposed robust and adaptive filters. The good behaviours of the proposed sup–inf and PDE‐based methods are illustrated on synthetic, greyscale, and colour images; results are analysed both qualitatively and quantitatively.

The authors propose here to overcome lacks of robustness against noise and adaptability to image features for which classical morphological operators suffer from. For doing this, they propose to deal with partial differential equations (PDEs) for generalised Cauchy problems, and they show that the proposed PDEs are equivalent to impose both robustness and adaptability to structuring functions of the corresponding sup‐inf operators. This allows them to introduce spatially adaptability in levellings, and it turns out that the proposed approach constitutes a PDE formulation and a generalisation of a larger class of levellings, the so‐called extended levellings, for which one of them are characterised by quasi‐flat zones. They show the efficiency of the proposed approach on synthetic, grey, and colour images with different types of noises.

We extend recent work on mathematical morphology for signal processing on weighted graphs, based on discrete tropical algebra. The framework is general and can be applied to any scalar function defined on a graph. We show applications in structure tensors analysis and the regularisation of greyscale images.

The high variability in fundus image databases is an important limiting drawback for detecting some retinal pathologies automatically. Age, human retinal pigmentation or lighting conditions affects in the colour of the acquired images. In this paper a colour-normalization method is presented as an initial pre-processing step in order to reduce the heterogeneity of retinal databases. The proposed method is based on geometric transformations applied to the chromaticity diagram of a target image taking into account a reference image. With the aim of quantifying the effect of the proposed colour normalization, a bright lesion detection from pathological images is carried out. A home-made system based on texture analysis and Support Vector Machine classification is used for this purpose. An improvement around a three percent in the detection accuracy demonstrates the importance of a retinal image colour pre-processing before any specific analysis.

This paper analyzes the kernel density estimation on spaces of Gaussian distributions endowed with different metrics. Expressions of kernels are provided for the 2-Wasserstein metric on the space of multivariate Gaussians. For the Fisher metric the kernels are provided only for univariate Gaussians and multivariate centered Gaussians. The density estimation is successfully applied to a classification problem of electro-encephalographic signals.

A technique to describe the spatial / spectral features of hyperspectral images is introduced. These descriptors aim at representing the content of the image while considering invariances related to the texture and to its geometric transformations, so called spatial invariances. Moreover, we also consider spectral invariances which are related to the composition of the pixels. Our approach is based on the scattering transform, which provides an useful framework for deep learning classification. The goal through these descriptors is to improve pixel-wise classification of hyperspectral images.

We propose a novel approach for pixel classification in hyperspectral images, leveraging on both the spatial and spectral information in the data. The introduced method relies on a recently proposed framework for learning on distributions - by representing them with mean elements in reproducing kernel Hilbert spaces (RKHS) and formulating a classification algorithm therein. In particular, we associate each pixel to an empirical distribution of its neighbouring pixels, a judicious representation of which in an RKHS, in conjunction with the spectral information contained in the pixel itself, give a new explicit set of features that can be fed into a suite of standard classification techniques - we opt for a well established framework of support vector machines (SVM). Furthermore, the computational complexity is reduced via random Fourier features formalism. We study the consistency and the convergence rates of the proposed method and the experiments demonstrate strong performance on hyperspectral data with gains in comparison to the state-of-the-art results.

In this paper, we discuss the problem of total ordering for mathematical morphology. We restrain the study to color images, but results are valid in any metric space. The discontinuities issue of total orders has already been evoked in mathematical morphology, but remains rarely studied. This phenomenon is highlighted and formalised in this paper. We propose a new approach to avoid discontinuities based on a recursive algorithm. The key point of the proposed method is to adapt the order to the studied image. The proposed framework presents invariance to isometric transformations of the color space. Promising results are presented.

This introductory paper aims at summarizing some problems and state-of-the-art techniques encountered in image processing for material analysis and design. Developing generic methods for this purpose is a complex task given the variability of the different image acquisition modalities (optical, scanning or transmission electron microscopy; surface analysis instrumentation, electron tomography, micro-tomography ...), and material composition (porous, fibrous, granular, hard materials, membranes, surfaces and interfaces ...). This paper presents an overview of techniques that have been and are currently developed to address this diversity of problems, such as segmentation, texture analysis, multiscale and directional features extraction, stochastic models and rendering, among others. Finally, it provides references to enter the issues, challenges and opportunities in materials characterization.

In this paper, a probability density function of object contours based on the stochastic watershed transform is carried out. The watershed transform produces an over-segmentation of the image due to noise, illumination problems, low contrast, etc., because each regional minimum of the image gives place to a region in the output image. To solve this problem, the efforts are focused on the definition of markers to impose new minima in the image, and enhancing the gradient image. The stochastic watershed performs a probability density function (pdf) of the object contours based on a MonteCarlo simulation of random markers. A variation of the method for defining this pdf based on regional regularization of the image is carried out. The objective is to obtain a pdf of the object contours with less noise and better contrast than that produced by the stochastic watershed to use it as a new gradient image for segmentation purposes.

This paper deals with the computation of the histogram of tensor images, that is, images where at each pixel is given a n n × n positive definite symmetric matrix, SPD(n). An approach based on orthogonal series density estimation is introduced, which is particularly useful for the case of measures based on Riemannian metrics. By considering SPD(n) as the space of the covariance matrices of multivariate gaussian distributions, we obtain the corresponding density estimation for the measure of both the Fisher metric and the Wasserstein metric. Experimental results on the application of such histogram estimation to DTI image segmentation, texture segmentation and texture recognition are included.

Classical adaptive mathematical morphology is based on operators which locally adapt the structuring elements to the image properties. Connected morphological operators act on the level of the flat zones of an image, such that only flat zones are filtered out, and hence the object edges are preserved. Area opening (resp. area closing) is one of the most useful connected operators, which filters out the bright (resp. dark) regions. It intrinsically involves the adaptation of the shape of the structuring element parameterized by its area. In this paper, we introduce the notion of reference-driven adaptive area opening according to two spatially-variant paradigms. First, the parameter of area is locally adapted by the reference image. This approach is applied to processing intensity depth images where the depth image is used to adapt the scale-size processing. Second, a self-dual area opening, where the reference image determines if the area filter is an opening or a closing with respect to the relationship between the image and the reference. Its natural application domain are the video sequences.

A current challenging topic in mathematical morphology is the construction of locally adaptive operators, i.e., structuring functions that are dependent on the input image itself at each position. Development of spatially variant filtering is well established in the theory and practice of Gaussian filtering. The aim of the first part (second and third sections) of the paper is to study how to generalize these convolution-based approaches in order to introduce adaptive nonlinear filters that asymptotically correspond to spatially variant morphological dilation and erosion. In particular, starting from the bilateral filtering framework and using the notion of counter-harmonic mean, our goal is to propose a new low complexity approach to defining spatially variant bilateral structuring functions. Then, in the second part (fourth section) of the paper, an original formulation of spatially variant flat morphological filters is proposed, where the adaptive structuring elements are obtained by thresholding the bilateral structuring functions. The methodological results of the paper are illustrated with various comparative examples.

The algorithm proposed in this paper allows to automatically segment the optic disc from a fundus image. The goal is to facilitate the early detection of certain pathologies and to fully automate the process so as to avoid specialist intervention. The method proposed for the extraction of the optic disc contour is mainly based on mathematical morphology along with principal component analysis (PCA). It makes use of different operations such as generalized distance function (GDF), a variant of the watershed transformation, the stochastic watershed, and geodesic transformations. The input of the segmentation method is obtained through PCA. The purpose of using PCA is to achieve the grey-scale image that better represents the original RGB image. The implemented algorithm has been validated on five public databases obtaining promising results. The average values obtained (a Jaccard's and Dice's coefficients of 0.8200 and 0.8932, respectively, an accuracy of 0.9947, and a true positive and false positive fractions of 0.9275 and 0.0036) demonstrate that this method is a robust tool for the automatic segmentation of the optic disc. Moreover, it is fairly reliable since it works properly on databases with a large degree of variability and improves the results of other state-of-the-art methods.

We present here discrete and continuous partial differential equation (PDE) -based methods for image enhancement/ sharpening. Using more robust and spatially adaptive PDEs, multiscale morphological operators that account image features are introduced, and then, used to provide a discrete enhancement operator, thanks to the former Kramer and Bruckner filter. A novel PDE associated to the introduced enhancement operator is established in 2D. Both the discrete and PDE-based sharpening filters are illustrated on synthetic, binary and real images.

The lack of a natural ordering on the sphere presents an inherent problem when defining morphological operators extended to unit sphere. We analyze here the notion of averaging over the unit sphere to obtain a local origin which can used to formulate ordering based operators. The notion of local supremum and infimum is introduced, which allows to define the dilation and erosion for images valued on the sphere. The algorithms are illustrated using polarimetric images.

In this paper, a new method for supervised hyperspectral data classification is proposed. In particular, the notion of stochastic minimum spanning forest (MSF) is introduced. For a given hyperspectral image, a pixelwise classification is first performed. From this classification map, M marker maps are generated by randomly selecting pixels and labeling them as markers for the construction of MSFs. The next step consists in building an MSF from each of the M marker maps. Finally, all the M realizations are aggregated with a maximum vote decision rule in order to build the final classification map. The proposed method is tested on three different data sets of hyperspectral airborne images with different resolutions and contexts. The influences of the number of markers and of the number of realizations M on the results are investigated in experiments. The performance of the proposed method is compared to several classification techniques (both pixelwise and spectral-spatial) using standard quantitative criteria and visual qualitative evaluation.

The paper deals with a generalization of the complex diffusion in order to introduce pseudo-morphological complex filters which mimic dilation/erosion operators. The non- linearization paradigm is based on the counter-harmonic mean. The physical model underlying complex diffusion is the free Schrodinger equation and consequently the proposed operators can be interpreted as the asymptotic "pseudo-morphological" solution of this fundamental equation. Theoretical results are illustrated with some image filtering examples.

A new method for supervised hyperspectral data classification is proposed. In particular, the notion of Stochastic Minimum Spanning Forests (MSFs) is introduced. For a given hyper-spectral image, a pixelwise classification is first performed. From this classification map, M marker maps are generated by randomly selecting pixels and labeling them as markers for the construction of MSFs. The next step consists in building an MSF from each of the M marker maps. Finally, all the M realizations are aggregated with a maximum vote decision rule, resulting in a final classification map. The experimental results presented on an AVIRIS image of the vegetation area show that the proposed approach yields accurate classification maps, and thus is attractive for hyperspectral data analysis.

This paper presents an algorithm for a 3D segmentation of the aorta artery in magnetic resonance images (MRI). The purpose is to project the 3D segmented aorta in the patient's abdomen with an augmented reality (AR) system to help the surgeon in laparoscopic interventions. In order to obtain accurate results in the segmentation process a marker-controlled watershed algorithm is used. Since this method requires a robust gradient image and two marker sets, a preprocessing step is carried out in each image. The algorithm is automatic and the results are promising with a Jaccard coefficient (JC) of 0.8107 ± 0.0228.

This paper presents significant improvements of Gray Level Size Zone Matrix (GLSZM) which is a bivariate statistical representation of texture, based on the co-occurrences of size/intensity of each flat zone (connected pixels of the same gray level). The first improvement is a multi-scale extension of the matrix which merges various quantizations of gray levels. A second alternative is proposed to take into account radial distribution of zone intensities. The third variant is a generalization of the matrix structure which allows to analyze fibrous textures, by changing the pair intensity/size for the pair length/orientation of each region. The interest of these improved descriptors is illustrated by texture classification problems arising from quantitative cell biology.

This paper deals with the theory and applications of spatially-variant discrete mathematical morphology. We review and formalize the definition of spatially variant dilation/erosion and opening/closing for binary and gray-level images using exclusively the structuring function, without resorting to complement. This theoretical framework allows to build morphological operators whose structuring elements can locally adapt their shape and orientation across the dominant direction of the structures in the image. The shape and orientation of the structuring element at each pixel are extracted from the image under study: the orientation is given by means of a diffusion process of the average square gradient field, which regularizes and extends the orientation information from the edges of the objects to the homogeneous areas of the image; and the shape of the orientated structuring elements can be linear or it can be given by the distance to relevant edges of the objects. The proposed filters are used on binary and gray-level images for enhancement of anisotropic features such as coherent, flow-like structures. Results of spatially-variant erosions/dilations and openings/closings-based filters prove the validity of this theoretical sound and novel approach.

A novel approach for vector ordering is introduced in this paper. The generic framework is based on a supervised learning formulation which leads to reduced orderings. A training set for the background and another training set for the foreground are needed as well as a supervised method to construct the ordering mapping. Two particular cases of learning techniques are considered in detail: 1) kriging-based vector ordering and 2) support vector machines-based vector ordering. These supervised orderings may then be used for the extension of mathematical morphology to vector images. In particular, in this paper, we focus on the application of morphological processing to hyperspectral images, illustrating the performance with practical examples.

Standard formulation of morphological operators is translation invariant in the space and in the intensity: the same processing is considered for each point of the image. A current challenging topic in mathematical morphology is the construction of adaptive operators. In previous works, the adaptive operators are based either on spatially variable neighbourhoods according to the local regularity, or on size variable neighbourhoods according to the local intensity. This paper introduces a new framework: the structurally adaptive mathematical morphology. More precisely, the rationale behind the present approach is to work on a nonlinear multi-scale image decomposition, and then to adapt intrinsically the size of the operator to the local scale of the structures. The properties of the derived operators are investigated and their practical performances are compared with respect to standard morphological operators using natural image examples.

This paper shows a comparison of spatially-variant discrete operators for denoising gray-level images. These non-iterative operators use a neighborhood that varies over space, adapting their shape and orientation according to the data of the image under study. The orientation of the neighborhood is computed by means of a diffusion process of the average square gradient field, which regularizes and extends the orientation information from the edges of the objects to the homogeneous areas of the image; and the shape of the orientated neighborhood can be either a linear segment or a rectangle of anisotropy given by the distance to relevant edges of the objects. Results on gray-level images show the ability of spatially-variant morphological operators for adaptively preserving the main structures in the image while reducing the noise.

A novel approach for vectorial ordering is introduced in this paper. The generic framework is based on a supervised learning formulation which leads to reduced orderings. A training set for the background and another training set for the foreground are needed as well as a supervised method to construct the ordering mapping. In particular, we consider here a kriging-based vectorial ordering. This supervised ordering may then used for the extension of mathematical morphology to vectorial images. Application of morphological processing to hyperspectral image illustrates the performance of proposal operators.

The aim of this paper is to propose tools for statistical analysis of shape families using morphological operators. Given a series of shape families (or shape categories), the approach consists in empirically computing shape statistics (i.e., mean shape and variance of shape) and then to use simple algorithms for random shape generation, for empirical shape confidence boundaries computation and for shape classification using Bayes rules. The main required ingredients for the present methods are well known in image processing, such as watershed on distance functions or log-polar transformation. Performance of classification is presented in a well-known shape database.

This paper deals with unsupervised segmentation of hyper-spectral images. It is based on the stochastic watershed, an approach to estimate a probability density function (pdf) of contours of an image using Monte Carlo simulations of watershed segmentations. In particular, it is introduced for the first time a multiscale framework for the computation of the pdf of contours using the stochastic watershed. Two multiscale approaches are considered: i) a linear scale-space using Gaussian filters, ii) a nonlinear morphological scale-space pyramid using levelings. In addition, a multiscale pyramid obtained by modifying the size of the random markers is also studied. Then, it is shown how the pdf of contours can finally be segmented using the non-parametric waterfalls algorithm. The performances of the proposed methods are compared using two examples of standard remote sensing hyperspectral images.

The application of nonlinear manifold learning for hyperspectral image analysis has been widely studied in last years. One of the main ingredients of these data reduction techniques is the distance used to compare the spectral band images. By means of this distance the pairwise similarity matrix is built and then, the matrix is used to explore the intrinsic dimensionality of the hyperspectral image. There are two main families of image distances which have been considered in previous works: i) the distance between the pixels using Minkowski metrics, such as the Euclidean distance or the L1distance; ii) the distances between the image histograms, such as the Kullback-Leibler Divergence or the chi-squared distance. The aim of this paper is to propose two new families of spatial image distances for spectral band comparison. Both are based on notions from mathematical morphology, a nonlinear image processing methodology based on the application of lattice theory to spatial structures. The first distance is based on the formulation using morphological dilations of Hausdorff distance for gray-scale images. The second distance is more original and it is founded in the leveling operator. Levelings are geodesic filters which modify, without blurring the contours, one of the images according to the other image. The application of these morphological distances for hyperspectral dimensionality exploration is illustrated with two powerful nonlinear data analysis techniques: Kernel-PCA and ISOMAP. Using standard image examples, their performance is studied in comparison with other image distances such as Euclidean distance and KL-divergence.

We propose in this paper to perform mathematical morphology operators in a geometric transformation of an image. As a result of this procedure, processing images with regular structuring elements in the transformed domain is equivalent to working with deformed structuring elements in the original representation. More specifically, the conversion into polar-logarithmic coordinates provides satisfying results in image analysis applied to round objects, if they are roughly origin-centered. We have illustrated the interest of the derived cyclic morphology with two pattern recognition examples: erythrocyte shape analysis and multiscale description of iris textures.

The paper presents a new segmentation and classification scheme to analyze hyperspectral (HS) data. The Robust Color Morphological Gradient of the HS image is computed, and the watershed transformation is applied to the obtained gradient. After the pixel-wise Support Vector Machines classification, the majority voting within the watershed regions is performed. Experimental results are presented on a 103-airborne ROSIS image, of the University of Pavia, Italy. The integration of the spatial information from the watershed segmentation into the HS image classification improves the classification accuracies, when compared to the pixel-wise classification.

This paper deals with color image simplification using levelings. This class of connected filters suppresses details but preserves the contours of the remaining structures or objects. As the notion of "color structure" is not trivial, the formulation of morphological operators for color images involves many open issues. The principle choice of a well-defined color space is crucial and it is proposed to work on a luminance/saturation/hue representation defined by the norm L1. A family of morphological color operators is then introduced using the classical formulation with total orderings by means of lexicographic cascades. In this framework, a methodology for color image simplification is introduced, which takes advantage of a saturation-controlled combination of the chromatic and the achromatic (or the spectral and the spatio-geometric) components. More precisely, it is based on the application of a color leveling to each significant region, specifically adapted to the nature (chromatic/achromatic) of the region and which needs an initial image partition into the homogenous regions. Experimental results illustrate the performance of the new developed algorithms.

The paper deals with the use of the various color pieces of information for segmenting color images and sequences with mathematical morphology operators. It is divided in four parts. The first one is concerning the choice of the color space suitable for morphological processing. The choice of a connection which induces a specific segmentation is discussed. The authors then present the color segmentation approach which is based on a nonparametric pyramid of watersheds, with a comparative study of different color gradients. Another multiscale color segmentation algorithm is then introduced, relying on the merging of chromatic-achromatic partitions ordered by the saturation component.

MOTIVATION: DNA microarrays are an experimental technology which consists in arrays of thousands of discrete DNA sequences that are printed on glass microscope slides. Image analysis is an important aspect of microarray experiments. The aim of this step is to reduce an image of spots into a table with a measure of the intensity for each spot. Efficient, accurate and automatic analysis of DNA spot images is essential in order to use this technology in laboratory routines. RESULTS: We present an automatic non-supervised set of algorithms for a fast and accurate spot data extraction from DNA microarrays using morphological operators which are robust to both intensity variation and artefacts. The approach can be summarised as follows. Initially, a gridding algorithm yields the automatic segmentation of the microarray image into spot quadrants which are later individually analysed. Then the analysis of the spot quadrant images is achieved in five steps. First, a pre-quantification, the spot size distribution law is calculated. Second, the background noise extraction is performed using a morphological filtering by area. Third, an orthogonal grid provides the first approach to the spot locus. Fourth, the spot segmentation or spot boundaries definition is carried out using the watershed transformation. And fifth, the outline of detected spots allows the signal quantification or spot intensities extraction; in this respect, a noise model has been investigated. The performance of the algorithm has been compared with two packages: ScanAlyze and Genepix, showing its robustness and precision.

This paper deals with techniques to detect abrupt scene transitions when random brightness variations (flicker) are present. This is normally the case when trying to restore or index old films. The application of conventional techniques in this situation tends to produce a large number of false positive detection of cuts. The paper is intently restricted to techniques which require low computation (no motion estimation).