Point clouds are a set of data points in space to represent the 3D geometry of objects. A fundamental step in the processing is to identify a subset of points to represent the shape. While traditional sampling methods often ignore to incorporate geometrical information, recent developments in learning-based sampling models have achieved significant levels of performance. With the integration of geometrical priors, the ability to learn and preserve the underlying structure can be enhanced when sampling. To shed light into the shape, a qualitative skeleton serves as an effective descriptor to guide sampling for both local and global geometries. In this paper, we introduce MorphoSkel3D11Code: https://github.com/Pierreoo/MorphoSkel3D as a new technique based on morphology to facilitate an efficient skeletonization of shapes. With its low computational cost, MorphoSkel3D is a unique, rule-based algorithm to benchmark its quality and performance on two large datasets, ModelNet and ShapeNet, under different sampling ratios. The results show that training with MorphoSkel3D leads to an informed and more accurate sampling in the practical application of object classification and point cloud retrieval.

Scene understanding of large-scale 3D point clouds of an outer space is still a challenging task. Compared with simulated 3D point clouds, the raw data from LiDAR scanners consist of tremendous points returned from all possible reflective objects and they are usually non-uniformly distributed. Therefore, its cost-effective to develop a solution for learning from raw large-scale 3D point clouds. In this track, we provide large-scale 3D point clouds of street scenes for the semantic segmentation task. The data set consists of 80 samples with 60 for training and 20 for testing. Each sample with over 2 million points represents a street scene and includes a couple of objects. There are five meaningful classes: building, car, ground, pole and vegetation. We aim at localizing and segmenting semantic objects from these large-scale 3D point clouds. Four groups contributed their results with different methods. The results show that learning-based methods are the trend and one of them achieves the best performance on both Overall Accuracy and mean Intersection over Union. Next to the learning-based methods, the combination of hand-crafted detectors are also reliable and rank second among comparison algorithms.

Ultimate levelings are operators that extract important image contrast information from a scale-space based on levelings. Along with the residual extraction process, some residues usually come from undesirable regions, and they should be filtered out. For this, some strategies can be applied to filter these undesirable residues. In this paper, we introduce a new approach to detect desirable regions from ultimate levelings through a new hierarchical structure called residual tree. From this structure, we extract attribute vectors to build a machine learning model which gives a matching value between ground truth regions and residual tree nodes. Thus, from the selected residual tree nodes, we present a new approach to choose the best disjoint residual nodes which gives the regions of the ultimate levelings. Finally, from the ultimate levelings, we use its partition associated function to solve the segmentation problem. In order to evaluate our new approach, some experiments were carried out with a plant dataset and results report the state-of-the-art performance in plant segmentation.

We present Kernel Point Convolution (KPConv), a new design of point convolution, i.e. that operates on point clouds without any intermediate representation. The convolution weights of KPConv are located in Euclidean space by kernel points, and applied to the input points close to them. Its capacity to use any number of kernel points gives KPConv more flexibility than fixed grid convolutions. Furthermore, these locations are continuous in space and can be learned by the network. Therefore, KPConv can be extended to deformable convolutions that learn to adapt kernel points to local geometry. Thanks to a regular subsampling strategy, KPConv is also efficient and robust to varying densities. Whether they use deformable KPConv for complex tasks, or rigid KPconv for simpler tasks, our networks outperform state-of-the-art classification and segmentation approaches on several datasets. We also offer ablation studies and visualizations to provide understanding of what has been learned by KPConv and to validate the descriptive power of deformable KPConv.

This paper introduces a new definition of multiscale neighborhoods in 3D point clouds. This definition, based on spherical neighborhoods and proportional subsampling, allows the computation of features with a consistent geometrical meaning, which is not the case when using k-nearest neighbors. With an appropriate learning strategy, the proposed features can be used in a random forest to classify 3D points. In this semantic classification task, we show that our multiscale features outperform state-of-the-art features using the same experimental conditions. Furthermore, their classification power competes with more elaborate classification approaches including Deep Learning methods.

This work addresses minimum spanning tree (MST) construction in streaming for images. We study the problem of computation a MST on streaming in which image columns from a continuous stream are processed in blocks of a given size. The correctness of proposed algorithm is proved and confirmed in the case of morphological segmentation of remote sensing images.

In this study, a multiscale local blur estimation is proposed based on the existing local focus measure that combines gradient and toggle mapping. This method evaluates the quality of images regardless of their content (not in an autofocus context) and can predict Optical Character Recognition accuracy based on local blur. The resulting approach outperforms state of the art blur detection methods. Quantitative results are given on DIQA database. Moreover, the authors demonstrate its usefulness for extracting a region of interest from partially blurry images. Results are shown on images acquired by a project devoted to smartphone based text extraction for visually impaired people. In this study, sharp region extraction is essential since it allows warning the users when their picture is unusable. Moreover, it saves computing time.

This work presents a new class of residual operators called ultimate levelings which are powerful image operators based on numerical residues. Within a multi-scale framework, these operators analyze a given image under a series of levelings. Thus, contrasted objects can be detected if a relevant residue is generated when they are filtered out by one of these levelings. Our approach consists of, firstly, (i) representing the input image as a morphological tree; then, (ii) showing that a certain operation on this tree results in a leveling operator; and finally (iii) demonstrating that a sequential application of this operation on the tree is able to produce a family of levelings that satisfies scale-space properties. Besides, other contributions of this paper include: (i) the statement of properties of ultimate levelings, (ii) the presentation of an efficient algorithm for their computation, (iii) the provision of strategies for choosing families of primitives, (iv) the presentation of strategies for filtering undesirable residues, and (v) the provision of some illustrative examples of application of ultimate levelings. Furthermore, ultimate levelings are computationally efficient and their performance evaluations are comparable to the state of art methods for filtering and image segmentation.

The authors address, in this study, a new adaptive binarisation method on images captured by smartphones. This work is part of an application for visually impaired people assistance, which aims at making text information accessible to people who cannot read it. The main advantage of the proposed method is that the windows underlying the local thresholding process are automatically adapted to the image content. This avoids the problematic parameter setting of local thresholding approaches, difficult to adapt to a heterogeneous database. The adaptive windows are extracted based on ultimate opening (a morphological operator) and then used as thresholding windows to perform a local Otsu's algorithm. The authors’ method is evaluated and compared with the Niblack, Sauvola, Wolf, toggle mapping morphological segmentation (TMMS) and maximally stable extremal regions methods on a new challenging database introduced by them. Their database is acquired by visually impaired people in real conditions. It contains 4000 annotated characters (available online for research purposes). Experiments show that the proposed method outperforms classical binarisation methods for degraded images such as low‐contrasted or blurred images, very common in their application.

Residual operators analyze the evolution of an image subject to the application of a series of transformations, for example a series of openings of increasing size. When a significant object is filtered out by a transformation corresponding to its size, an important residue is observed. Maximal residues are kept for each pixel, indicating the most significant objects present in the image. Different families of operators have been used in the literature: morphological openings or closings, attribute openings or openings by reconstruction. In this paper we propose to compute residues on a hierarchy of partitions, computing the differences between regions at different hierarchical levels based on the classical earth mover's distance. The advantage of our approach is that it is autodual and generic as it can be applied with any hierarchical approach.

Abstract: This paper describes a publicly available 3D database from the rue Madame, a street in the 6th Parisian district. Data have been acquired by the Mobile Laser Scanning (MLS) system L3D2 and correspond to a 160 m long street section. Annotation has been carried out in a manually assisted way. An initial annotation is obtained using an automatic segmentation algorithm. Then, a manual refinement is done and a label is assigned to each segmented object. Finally, a class is also manually assigned to each object. Available classes include facades, ground, cars, motorcycles, pedestrians, traffic signs, among others. The result is a list of (X, Y, Z, reflectance, label, class) points. Our aim is to offer, to the scientific community, a 3D manually labeled dataset for detection, segmentation and classification benchmarking. With respect to other databases available in the state of the art, this dataset has been exhaustively annotated in order to include all available objects and to allow point-wise comparison. 1

Text detection in natural images remains a very challenging task. For instance, in an urban context, the detection is very difficult due to large variations in terms of shape, size, color, orientation, and the image may be blurred or have irregular illumination, etc. In this paper, we describe a robust and accurate multiresolution approach to detect and classify text regions in such scenarios. Based on generation/validation paradigm, we first segment images to detect character regions with a multiresolution algorithm able to manage large character size variations. The segmented regions are then filtered out using shape-based classification, and neighboring characters are merged to generate text hypotheses. A validation step computes a region signature based on texture analysis to reject false positives. We evaluate our algorithm in two challenging databases, achieving very good results.

We offer, in this paper, a new method to segment text in natural scenes. This method is based on the use of a morphological operator: the Toggle Mapping. The efficiency of the method is illustrated and the method is compared, according to various criteria, with common methods issued from the state of the art. This comparison shows that our method gives better results and is faster than the state of the art methods. Our method reduces also the number of segmented regions. This can lead to time saving in a complete scheme (executing time of multiple processing steps usually depends on the number of regions) and proves that our algorithm is more relevant.

In this paper, we describe an automatic method for segmentation of building facade images. First, individual facades are isolated from general city block images. This step is based on accumulation of directional color gradients, assuming that facade structures are aligned. Then sky region is detected based on segmentation approach and color marker extraction. Finally, the images are split in floors using directional color gradient accumulation, as well. Our approach introduces several morphological filters to augment the robustness to problems such as: textured balconies, some specular reflections of the bright windows and small obstacles in images. The experimental results show the performance of our approach.

This paper discusses the usefulness of a partition-based image representation in the context of user-assisted segmentation, and compares it with the two other most common image representations in this framework: pixel-based and transition-based. Partition-based image representations allow user-interaction at the region level, which is a very natural and friendly manner to interact with images. The paper describes various region-based implementations of the most common types of user interaction; namely, initial object selection, object refinement, imposing object characteristics, selection of specific objects, selection of objects that are similar to a previously selected one. In all cases, partition-based image representations, and their associated region-based tools for user interaction, show up to be very suitable for interactive image segmentation.

In this paper we describe an interactive video object segmentation tool developed in the framework of the ACTS-AC098 MOMUSYS project. The Video Object Generator with User Environment (VOGUE) combines three different sets of automatic and semi-automatic-tool (spatial segmentation, object tracking and temporal segmentation) with general purpose tools for user interaction. The result is an integrated environment allowing the user-assisted segmentation of any sort of video sequences in a friendly and efficient manner.

This paper presents a toolbox for interactive image segmentation based on a series of nested partitions of increasing coarseness. The user can navigate among the different resolution levels and select regions with simple mouse clicks. The whole family of partitions can be created during a single morphological flooding of the image, leading to a very fast algorithm. The information is stored in a minimum spanning tree.

This paper presents a generic video coding algorithm allowing the content-based manipulation of objects. This manipulation is possible thanks to the definition of a spatiotemporal segmentation of the sequences. The coding strategy relies on a joint optimization in the rate-distortion sense of the partition definition and of the coding techniques to be used within each region. This optimization creates the link between the analysis and synthesis parts of the coder. The analysis defines the time evolution of the partition, as well as the elimination or the appearance of regions that are homogeneous either spatially or in motion. The coding of the texture as well as of the partition relies on region-based motion compensation techniques. The algorithm offers a good compromise between the ability to track and manipulate objects and the coding efficiency.

This paper presents a technique for tracking areas of interest in the framework of segmentation-based video coding. The technique is independent of the type of segmentation technique used in the coding approach. Therefore, it can also be used in block-based coding schemes. The algorithm relies on a double segmentation of the image based on morphological tools. This double segmentation permits to obtain the position and shape of the previous area of interest in the current image. In order to demonstrate the potentialities of this algorithm, it is applied in a specific coding scheme so that content-based selective coding is allowed.

We present a coding scheme that achieves, for each image in the sequence, the best segmentation in terms of rate-distortion theory. It is obtained from a set of initial regions and a set of available coding techniques. The segmentation combines spatial and motion criteria. It selects at each area of the image the most adequate criterion for defining a partition in order to obtain the best compromise between cost and quality. In addition, the proposed scheme is very suitable for addressing content-based functionalities.