Point clouds are a set of data points in space to represent the 3D geometry of objects. A fundamental step in the processing is to identify a subset of points to represent the shape. While traditional sampling methods often ignore to incorporate geometrical information, recent developments in learning-based sampling models have achieved significant levels of performance. With the integration of geometrical priors, the ability to learn and preserve the underlying structure can be enhanced when sampling. To shed light into the shape, a qualitative skeleton serves as an effective descriptor to guide sampling for both local and global geometries. In this paper, we introduce MorphoSkel3D11Code: https://github.com/Pierreoo/MorphoSkel3D as a new technique based on morphology to facilitate an efficient skeletonization of shapes. With its low computational cost, MorphoSkel3D is a unique, rule-based algorithm to benchmark its quality and performance on two large datasets, ModelNet and ShapeNet, under different sampling ratios. The results show that training with MorphoSkel3D leads to an informed and more accurate sampling in the practical application of object classification and point cloud retrieval.

Abstract. Classical mathematical morphology on images relies on two translation equivariant operators which are often considered as the nonlinear counterparts of convolution. Observing the development of convolutional neural networks, mathematical morphology is transitioning to a deep learning framework. This paper is an attempt (extending the paper of Penaud–Polge, Velasco-Forero, and Angulo [ Proceedings of the International Conference on Discrete Geometry and Mathematical Morphology, 2024]) to build theoretical foundations to adapt mathematical morphology to group equivariant deep learning. The proposed theory generalizes existing framework of translation equivariant morphological operators by considering a special case of group morphology, introduced by Roerdink in the early 2000s, and deriving it in the context of nonabelian group actions. A theoretical aperture is given by (i) a generalized expression of [Formula: see text]-operators, proposed by Heijmans in the early 90s, for group equivariance and (ii) a group equivariant version of a recent smooth approximation of morphological operators by Hermary et al. [ J. Math. Imaging Vision, 64 (2022), pp. 736–753]. The theoretical results lead to the proposition of several group equivariant morphological layers. Finally, the proposed layers are assessed using the Fashion-MNIST dataset in the case of translations and [Formula: see text] rotations. The experiments show that the proposed morphological networks, trained only with upright samples, classify rotated images without a loss of performance.

The Gaussian kernel and its derivatives have already been employed for Convolutional Neural Networks in several previous works. Most of these papers proposed to compute filters by linearly combining one or several bases of fixed or slightly trainable Gaussian kernels with or without their derivatives. In this article, we propose a high-level configurable layer based on anisotropic, oriented and shifted Gaussian derivative kernels which generalize notions encountered in previous related works while keeping their main advantage. The results show that the proposed layer has competitive performance compared to previous works and that it can be successfully included in common deep architectures such as VGG16 for image classification and U-net for image segmentation.

Symmetry is present in many tasks in computer vision, where the same class of objects can appear transformed, e.g. rotated due to different camera orientations, or scaled due to perspective. The knowledge of such symmetries in data coupled with equivariance of neural networks can improve their generalization to new samples. Differential invariants are equivariant operators computed from the partial derivatives of a function. In this paper we use differential invariants to define equivariant operators that form the layers of an equivariant neural network. Specifically, we derive invariants of the Special Euclidean Group SE(2), composed of rotations and translations, and apply them to construct a SE(2)-equivariant network, called SE(2) Differential Invariants Network (SE2DINNet). The network is subsequently tested in classification tasks which require a degree of equivariance or invariance to rotations. The results compare positively with the state-of-the-art, even though the proposed SE2DINNet has far less parameters than the compared models.

Empirical mode decomposition (EMD) is a fully data driven method for multiscale decomposing signals into a set of components known as intrinsic mode functions. EMD is based on lower and upper envelopes of the signal in an iterated decomposition scheme. In this paper, we put forward a simple yet effective method to learn EMD from data by means of morphological operators. We propose an end-to-end framework by incorporating morphological EMD operators into deeply learned representations, trained using standard backpropagation principle and gradient descent-based optimization algorithms. Three generalizations of morphological EMD are proposed: (a) by varying the family of structuring functions, (b) by varying the pair of morphological operators used to calculate the envelopes, and (c) by considering a convex sum of envelopes instead of the mean point used in classical EMD. We discuss in particular the invariances that are induced by the morphological EMD representation. Experimental results on supervised classification of hyperspectral images by one-dimensional convolutional networks demonstrate the interest of our method.

An adaptive three-dimensional Kalman filter for the tracking of maneuvering targets in three dimensions is proposed. In the radar industry, numerous trackers are based on a constant velocity model, with a process noise covariance matrix Q which is adapted in real time to enhance tracking: it is kept at moderate values during straight lines where the constant velocity assumption applies and is increased during maneuvers. In the present paper we advocate a novel method to increase Q during maneuvers (and hence the Kalman gains) based on a recurrent neural network (RNN). The difficulty and the interest of our approach lies in the fact the neural network is trained together with the filter, by backpropagation through the filter, and hence learns the covariance matrix such as to directly maximize the accuracy of the final output.

Scene understanding of large-scale 3D point clouds of an outer space is still a challenging task. Compared with simulated 3D point clouds, the raw data from LiDAR scanners consist of tremendous points returned from all possible reflective objects and they are usually non-uniformly distributed. Therefore, its cost-effective to develop a solution for learning from raw large-scale 3D point clouds. In this track, we provide large-scale 3D point clouds of street scenes for the semantic segmentation task. The data set consists of 80 samples with 60 for training and 20 for testing. Each sample with over 2 million points represents a street scene and includes a couple of objects. There are five meaningful classes: building, car, ground, pole and vegetation. We aim at localizing and segmenting semantic objects from these large-scale 3D point clouds. Four groups contributed their results with different methods. The results show that learning-based methods are the trend and one of them achieves the best performance on both Overall Accuracy and mean Intersection over Union. Next to the learning-based methods, the combination of hand-crafted detectors are also reliable and rank second among comparison algorithms.

We extend recent work on mathematical morphology for signal processing on weighted graphs, based on discrete tropical algebra. The framework is general and can be applied to any scalar function defined on a graph. We show applications in structure tensors analysis and the regularisation of greyscale images.

A hierarchy is a series of nested partitions in which a coarser partition results from merging regions of finer ones. Each hierarchy derived from an image provides a particular structural description of the image content, depending upon the criteria for merging neighboring regions. Distinct hierarchies derived from a same image reflect its various facets and the distances between them nicely characterize its content. In this paper the hierarchies are constructed with the versatile stochastic watershed algorithm and their inter-distances are measured with the Gromov-Hausdorff distance. Experiments conducted on images simulated by dead leaves model illustrate the advantages of our approach in terms of learning efficiency and understandability of the results.

This work addresses minimum spanning tree (MST) construction in streaming for images. We study the problem of computation a MST on streaming in which image columns from a continuous stream are processed in blocks of a given size. The correctness of proposed algorithm is proved and confirmed in the case of morphological segmentation of remote sensing images.

Recent works on image co-segmentation aim to segment common objects among image sets. These methods can co-segment simple images well, but their performance may degrade significantly on more cluttered images. In order to co-segment both simple and complex images well, this paper proposes a novel paradigm to rank images and to propagate the segmentation results from the simple images to more and more complex ones. In the experiments, the proposed paradigm demonstrates its effectiveness in segmenting large image sets with a wide variety in object appearance, sizes, orientations, poses, and multiple objects in one image. It outperformed the current state-of-the-art algorithms significantly, especially in difficult images.

Anomaly Detection methods are used when there is not enough information about the target to detect. These methods search for pixels in the image with spectral characteristics that differ from the background. The most widespread detection test, the RX-detector, is based on the Mahalanobis distance and on the background statistical characterization through the mean vector and the covariance matrix. Although non-Gaussian distributions have already been introduced for background modeling in Hyperspectral Imaging, the parameters estimation is still performed using the Maximum Likelihood Estimates for Gaussian distribution. This paper describes robust estimation procedures more suitable for non-Gaussian environment. Therefore, they can be used as plug-in estimators for the RX-detector leading to some great improvement in the detection process. This theoretical improvement has been evidenced over two real hyperspectral images.

Edges are crucial descriptors for image analysis and rely mostly on local spectral distances. In this paper, local spectral changes are assimilated to the local statistical dependences which are measured by the local mutual information. This metric is shown to be invariant to unknown bijective transforms which makes it a good candidate for analyzing consistently satellite images which are affected by uncontrolled spectral distortions. Such an edge property is assessed with WorldView-2 multi-angular sequence of images, where spectral distortions arise with far nadir view angles.

A novel approach for vector ordering is introduced in this paper. The generic framework is based on a supervised learning formulation which leads to reduced orderings. A training set for the background and another training set for the foreground are needed as well as a supervised method to construct the ordering mapping. Two particular cases of learning techniques are considered in detail: 1) kriging-based vector ordering and 2) support vector machines-based vector ordering. These supervised orderings may then be used for the extension of mathematical morphology to vector images. In particular, in this paper, we focus on the application of morphological processing to hyperspectral images, illustrating the performance with practical examples.

Standard formulation of morphological operators is translation invariant in the space and in the intensity: the same processing is considered for each point of the image. A current challenging topic in mathematical morphology is the construction of adaptive operators. In previous works, the adaptive operators are based either on spatially variable neighbourhoods according to the local regularity, or on size variable neighbourhoods according to the local intensity. This paper introduces a new framework: the structurally adaptive mathematical morphology. More precisely, the rationale behind the present approach is to work on a nonlinear multi-scale image decomposition, and then to adapt intrinsically the size of the operator to the local scale of the structures. The properties of the derived operators are investigated and their practical performances are compared with respect to standard morphological operators using natural image examples.

A novel approach for vectorial ordering is introduced in this paper. The generic framework is based on a supervised learning formulation which leads to reduced orderings. A training set for the background and another training set for the foreground are needed as well as a supervised method to construct the ordering mapping. In particular, we consider here a kriging-based vectorial ordering. This supervised ordering may then used for the extension of mathematical morphology to vectorial images. Application of morphological processing to hyperspectral image illustrates the performance of proposal operators.

The aim of this paper is to propose tools for statistical analysis of shape families using morphological operators. Given a series of shape families (or shape categories), the approach consists in empirically computing shape statistics (i.e., mean shape and variance of shape) and then to use simple algorithms for random shape generation, for empirical shape confidence boundaries computation and for shape classification using Bayes rules. The main required ingredients for the present methods are well known in image processing, such as watershed on distance functions or log-polar transformation. Performance of classification is presented in a well-known shape database.

This paper deals with unsupervised segmentation of hyper-spectral images. It is based on the stochastic watershed, an approach to estimate a probability density function (pdf) of contours of an image using Monte Carlo simulations of watershed segmentations. In particular, it is introduced for the first time a multiscale framework for the computation of the pdf of contours using the stochastic watershed. Two multiscale approaches are considered: i) a linear scale-space using Gaussian filters, ii) a nonlinear morphological scale-space pyramid using levelings. In addition, a multiscale pyramid obtained by modifying the size of the random markers is also studied. Then, it is shown how the pdf of contours can finally be segmented using the non-parametric waterfalls algorithm. The performances of the proposed methods are compared using two examples of standard remote sensing hyperspectral images.

The application of nonlinear manifold learning for hyperspectral image analysis has been widely studied in last years. One of the main ingredients of these data reduction techniques is the distance used to compare the spectral band images. By means of this distance the pairwise similarity matrix is built and then, the matrix is used to explore the intrinsic dimensionality of the hyperspectral image. There are two main families of image distances which have been considered in previous works: i) the distance between the pixels using Minkowski metrics, such as the Euclidean distance or the L1distance; ii) the distances between the image histograms, such as the Kullback-Leibler Divergence or the chi-squared distance. The aim of this paper is to propose two new families of spatial image distances for spectral band comparison. Both are based on notions from mathematical morphology, a nonlinear image processing methodology based on the application of lattice theory to spatial structures. The first distance is based on the formulation using morphological dilations of Hausdorff distance for gray-scale images. The second distance is more original and it is founded in the leveling operator. Levelings are geodesic filters which modify, without blurring the contours, one of the images according to the other image. The application of these morphological distances for hyperspectral dimensionality exploration is illustrated with two powerful nonlinear data analysis techniques: Kernel-PCA and ISOMAP. Using standard image examples, their performance is studied in comparison with other image distances such as Euclidean distance and KL-divergence.

Spatial smoothing over the original hyperspectral data based on wavelet and anisotropic partial differential equations is incorporated using composite kernel in graph-based classifiers. The kernels combine spectral-spatial relationships using the smoothed and original hyperspectral images. Experiments with different real hyperspectral scenarios are presented. Comparison with recent graph-based methods shows that the proposed scheme gives better classification with lower computational cost.

Spatial smoothing over the original hyperspectral data based on wavelet and partial differential equations (PDEs) are incorporated in the classifiers using composite kernel with kNN graphs. The kernels combine spectral-spatial relationships using the smoothed and original images. Experiments with real hyperspectral scenarios are presented. Comparison with recent graph based methods show that the proposed scheme improves existing methods.