Due to advancements in digital cameras, it is easy to gather multiple images (or videos) from an object under different conditions. Therefore, image-set classification has attracted more attention, and different solutions were proposed to model them. A popular way to model image sets is subspaces, which form a manifold called the Grassmann manifold. In this contribution, we extend the application of Generalized Relevance Learning Vector Quantization to deal with Grassmann manifold. The proposed model returns a set of prototype subspaces and a relevance vector. While prototypes model typical behaviours within classes, the relevance factors specify the most discriminative principal vectors (or images) for the classification task. They both provide insights into the model's decisions by highlighting influential images and pixels for predictions. Moreover, due to learning prototypes, the model complexity of the new method during inference is independent of dataset size, unlike previous works. We applied it to several recognition tasks including handwritten digit recognition, face recognition, activity recognition, and object recognition. Experiments demonstrate that it outperforms previous works with lower complexity and can successfully model the variation, such as handwritten style or lighting conditions. Moreover, the presence of relevances makes the model robust to the selection of subspaces' dimensionality.

The $\alpha $ -tree is an effective hierarchical image representation used for connected filtering or segmentation in remote sensing and other image applications. The $\alpha $ -tree constructs a tree based on the dissimilarities of the pixels in an image. Compared to other hierarchical image representations such as the component tree, the $\alpha $ -tree provides a better representation of the granularity of images and is easier to apply to multichannel images. The major drawback of the $\alpha $ -tree is its processing speed, due to the large amount of data to be processed and the lack of studies on an efficient algorithms, especially on multichannel and high dynamic range images. In this study, we introduce a novel adaptation of the hybrid component tree algorithm on the $\alpha $ -tree for fast parallel $\alpha $ -tree construction in any dynamic range of pixel dissimilarity. We tested the hybrid $\alpha $ -tree algorithm on Sentinel-2 remote sensing images from the European Space Agency (ESA) as well as randomly generated images, on the Hábrók high performance computing cluster. Experimental results show that the hybrid $\alpha $ -tree algorithm achieves the processing speed of 10-30Mpix/s and the speedup of 10-30 on a 128-core computer, proving the efficiency of the first parallel $\alpha $ -tree algorithm in high dynamic range, to the best of our knowledge.

Alpha trees, and derived $\alpha $ - $\omega $ -hierarchies are powerful tools for hierarchical image representation in computer vision. However, the quality of $\alpha $ - $\omega $ -hierarchies has not been fully evaluated, limiting their further development and application. In our study, an algorithm for evaluating the quality of $\alpha $ - $\omega $ -hierarchies based on horizontal cut filters is proposed. With the aim to automatically select optimal parameters and dissimilarity measures for $\alpha $ - $\omega $ -hierarchy constructions, key factors including maximum accuracy, construction complexity, and efficiency of $\alpha $ - $\omega $ -hierarchies are systematically considered. Notably, remote sensing images based experiments were conducted to demonstrate the usefulness of this algorithm. In addition, our algorithm can be potentially extended to qualify other types of hierarchical trees, making it useful for the automatic selection of optimal hierarchical segmentation methods.

The α-tree algorithm is a useful hierarchical representation technique which facilitates comprehension of images such as remote sensing and medical images. Most α-tree algorithms make use of priority queues to process image edges in a correct order, but because traditional priority queues are inefficient in α-tree algorithms using extreme-dynamic-range pixel dissimilarities, they run slower compared with other related algorithms such as component tree. In this paper, we propose a novel hierarchical heap priority queue algorithm that can process α-tree edges much more efficiently than other state-of-the-art priority queues. Experimental results using 48-bit Sentinel-2 A remotely sensed images and randomly generated images have shown that the proposed hierarchical heap priority queue improved the timings of the flooding α-tree algorithm by replacing the heap priority queue with the proposed queue: 1.68 times in 4-N and 2.41 times in 8-N on Sentinel-2 A images, and 2.56 times and 4.43 times on randomly generated images.

Component trees are powerful image processing tools to analyze the connected components of an image. One attractive strategy consists in building the nested relations at first and then deriving the components’ attributes afterward, such that the user can switch between different attribute functions without having to re-compute the entire tree. Only sequential algorithms allow such an approach, while no parallel algorithm is available. In this paper, we extend a recent method using distributed memory techniques to enable posterior attribute computation in a parallel or distributed manner. This novel approach significantly reduces the computational time needed for combining several attribute functions interactively in Giga and Tera-Scale data sets.

Connected filters and multi-scale tools are region-based operators acting on the connected components of an image. Component trees are image representations to efficiently perform these operations as they represent the inclusion relationship of the connected components hierarchically. This paper presents disccofan (DIStributed Connected COmponent Filtering and ANalysis), a new method that extends the previous 2D implementation of the Distributed Component Forests (DCFs) to handle 3D processing and higher dynamic range data sets. disccofan combines shared and distributed memory techniques to efficiently compute component trees, user-defined attributes filters, and multi-scale analysis. Compared to similar methods, disccofan is faster and scales better on low and moderate dynamic range images, and is the only method with a speed-up larger than 1 on a realistic, astronomical floating-point data set. It achieves a speed-up of 11.20 using 48 processes to compute the DCF of a 162 Gigapixels, single-precision floating-point 3D data set, while reducing the memory used by a factor of 22. This approach is suitable to perform attribute filtering and multi-scale analysis on very large 2D and 3D data sets, up to single-precision floating-point value.

Component-graphs provide powerful and complex structures for multi-band image processing. We propose a multiband astronomical source detection framework with the component-graphs relying on a new set of component attributes. We propose two modules to differentiate nodes belong to distinct objects and to detect partial object nodes. Experiments demonstrate an improved capacity at detecting faint objects on a multi-band astronomical dataset.

Extraction of depth from images is of great importance for various computer vision applications. Methods based on convolutional neural networks are very accurate but have high computation requirements, which can be achieved with GPUs. However, GPUs are difficult to use on devices with low power requirements like robots and embedded systems. In this light, we propose a stereo matching method appropriate for applications in which limited computational and energy resources are available. The algorithm is based on a hierarchical representation of image pairs which is used to restrict disparity search range. We propose a cost function that takes into account region contextual information and a cost aggregation method that preserves disparity borders. We tested the proposed method on the Middlebury and KITTI benchmark data sets and on the TrimBot2020 synthetic data. We achieved accuracy and time efficiency results that show that the method is suitable to be deployed on embedded and robotics systems.

The standard representations known as component trees, used in morphological connected attribute filtering and multi-scale analysis, are unsuitable for cases in which either the image itself or the tree do not fit in the memory of a single compute node. Recently, a new structure has been developed which consists of a collection of modified component trees, one for each image tile. It has to-date only been applied to fairly simple image filtering based on area. In this paper, we explore other applications of these distributed component forests, in particular to multi-scale analysis such as pattern spectra, and morphological attribute profiles and multi-scale leveling segmentations.

Max-trees, or component trees, are graph structures that represent the connected components of an image in a hierarchical way. Nowadays, many application fields rely on images with high-dynamic range or floating point values. Efficient sequential algorithms exist to build trees and compute attributes for images of any bit depth. However, we show that the current parallel algorithms perform poorly already with integers at bit depths higher than 16 bits per pixel. We propose a parallel method combining the two worlds of flooding and merging max-tree algorithms. First, a pilot max-tree of a quantized version of the image is built in parallel using a flooding method. Later, this structure is used in a parallel leaf-to-root approach to compute efficiently the final max-tree and to drive the merging of the sub-trees computed by the threads. We present an analysis of the performance both on simulated and actual 2D images and 3D volumes. Execution times are about better than the fastest sequential algorithm and speed-up goes up to on 64 threads.

We propose a local region descriptor based on connected pattern spectra, and combined with normalized central moments. The descriptors are calculated for MSER regions of the image, and their performance compared against SIFT. The MSER regions were chosen because they can be efficiently selected by constructing a max-tree, a structure used to calculate both descriptors and region moments. Experiments on the UCID database show an improvement over SIFT in two out of five experimental setups, and comparable performance in two other experiments. The new descriptors are only half the size of SIFT, resulting in 4 times faster query times when performing exact search on descriptor index built from 262 images.

Estimation of the background is an essential step in automated extraction of faint, extended objects from large-scale, optical surveys in astronomy. In this paper we present an improvement on the background estimation method of a commonly used tool in this field: Source Extractor (SEx-tractor). We show that the original method suffers from bias caused by presence of extended sources, and present an alternative which greatly reduces this effect, leading to much better preservation of faint extended structures.

Component trees are important data structures for computation of connected attribute filters. Though some of the available algorithms are suitable for high-dynamic range, and in particular floating point data, none are suitable for computation of component trees for so-called second-generation, and mask-based connectivity. The latter allow generalization of the traditional notion of connected components, to allow considering e.g. a star cluster as a single entity. This paper provides an O(N log N) algorithm for component trees, suitable for standard and mask-based connectivity. At 24 bits per pixel, the new algorithm outperforms the existing by a factor of 20 to 77 in cpu-time, on 3 megapixel images, depending on the image content.

In this paper, we present a new method for attribute filtering, combining contrast and structural information. Using hyperconnectivity based on k-flat zones, we improve the ability of attribute filters to retain internal details in detected objects. Simultaneously, we improve the suppression of small, unwanted detail in the background. We extend the theory of attribute filters to hyperconnectivity and provide a fast algorithm to implement the new method. The new version is only marginally slower than the standard Max-Tree algorithm for connected attribute filters, and linear in the number of pixels or voxels. It is two orders of magnitude faster than anisotropic diffusion. The method is implemented in the form of a filtering rule suitable for handling both increasing (size) and nonincreasing (shape) attributes. We test this new framework on nonincreasing shape filters on both 2D images from astronomy, document processing, and microscopy, and 3D CT scans, and show increased robustness to noise while maintaining the advantages of previous methods.

Attribute filters allow enhancement and extraction of features without distorting their borders, and never introduce new image features. These are highly desirable properties in biomedical imaging, where accurate shape analysis is paramount. However, setting the attribute-threshold parameters has to date only been done manually. This paper explores simple, fast and automated methods of computing attribute threshold parameters based on image segmentation, thresholding and data clustering techniques. Though several techniques perform well on blood-vessel filtering, the choice of technique appears to depend on the imaging mode.

In medical imaging, anatomical and other structures such as urinary stones, are often extracted with the aid of active contour/ surface models. Active surface-based methods have robustness limitations and are computationally expensive. In this paper we present a morphological method based on attribute filters and the newly presented sphericity attribute. The operators involved, extract the targeted objects in their entirety without shape/size distortions and proceed rapidly. Experiments on three real 3D data-sets demonstrate their efficiency and their performance is discussed.

Openings-by-reconstruction are the oldest connected filters, and indeed, reconstruction methodology lies at the heart of many connected operators such as levelings. Starting out from the basic reconstruction principle of iterated geodesic dilations, extensions such as the use of reconstruction criteria, which constrain the reconstruction process, are discussed. The latter prevent linking distinct objects connected by narrow bridges during the reconstruction process, whilst maintaining as much edge preservation as possible. A far faster variant of filtering with reconstruction criteria is presented, which can be implemented in an O(N) algorithm in stead of O(N2).

Morphological attribute filters have not previously been parallelized, mainly because they are both global and non-separable. We propose a parallel algorithm that achieves efficient parallelism for a large class of attribute filters, including attribute openings, closings, thinnings and thickenings, based on Salembier's Max-Trees and Min-trees. The image or volume is first partitioned in multiple slices. We then compute the Max-trees of each slice using any sequential Max-Tree algorithm. Subsequently, the Max-trees of the slices can be merged to obtain the Max-tree of the image. A C-implementation yielded good speed-ups on both a 16-processor MIPS 14000 parallel machine, and a dual-core Opteron-based machine. It is shown that the speed-up of the parallel algorithm is a direct measure of the gain with respect to the sequential algorithm used. Furthermore, the concurrent algorithm shows a speed gain of up to 72 percent on a single-core processor, due to reduced cache thrashing.

An efficient algorithm is presented for the computation of grayscale morphological operations with arbitrary 2-D flat structuring elements (S.E.). The required computing time is independent of the image content and of the number of gray levels used. It always outperforms the only existing comparable method, which was proposed in the work by Van Droogenbroeck and Talbot, by a factor between 3.5 and 35.1, depending on the image type and shape of S.E. So far, filtering using multiple S.E.s is always done by performing the operator for each size and shape of the S.E. separately. With our method, filtering with multiple S.E.s can be performed by a single operator for a slightly reduced computational cost per size or shape, which makes this method more suitable for use in granulometries, dilation-erosion scale spaces, and template matching using the hit-or-miss transform. The discussion focuses on erosions and dilations, from which other transformations can be derived.

In this paper connected operators from mathematical morphology are extended to a wider class of operators, which are based on connectivities in higher dimensional spaces, similar to scale spaces, which will be called attribute-spaces. Though some properties of connected filters are lost, granulometries can be defined under certain conditions, and pattern spectra in most cases. The advantage of this approach is that regions can be split into constituent parts before filtering more naturally than by using partitioning connectivities. Furthermore, the approach allows dealing with overlap, which is impossible in connectivity. A theoretical comparison to hyperconnectivity suggests the new concept is different. The theoretical results are illustrated by several examples. These show how attribute-space connected filters merge the ability of filtering based on local structure using classical, structuring-element-based filters to the object-attribute-based filtering of connected filters, and how this differs from similar attempts using second-generation connectivity.

Connected filters are edge-preserving morphological operators, which rely on a notion of connectivity. This is usually the standard 4 and 8-connectivity, which is often too rigid since it cannot model generalized groupings such as object clusters or partitions. In the set-theoretical framework of connectivity, these groupings are modeled by the more general second-generation connectivity. In this paper, we present both an extension of this theory, and provide an efficient algorithm based on the Max-Tree to compute attribute filters based on these connectivities. We first look into the drawbacks of the existing framework that separates clustering and partitioning and is directly dependent on the properties of a preselected operator. We then propose a new type of second-generation connectivity termed mask-based connectivity which eliminates all previous dependencies and extends the ways the image domain can be connected. A previously developed Dual-Input Max-Tree algorithm for area openings is adapted for the wider class of attribute filters on images characterized by second-generation connectivity. CPU-times for the new algorithm are comparable to the original algorithm, typically deviating less than 10 percent either way.

In this paper, we describe a multiscale and multishape morphological method for pattern-based analysis and classification of gray-scale images using connected operators. Compared with existing methods, which use structuring elements, our method has three advantages. First, in our method, the time needed for computing pattern spectra does not depend on the number of scales or shapes used, i.e., the computation time is independent of the dimensions of the pattern spectrum. Second, size and strict shape attributes can be computed, which we use for the construction of joint 2D shape-size pattern spectra. Third, our method is significantly less sensitive to noise and is rotation-invariant. Although rotation invariance can also be approximated by methods using structuring elements at different angles, this tends to be computationally intensive. The classification performance of these methods is discussed using four image sets: Brodatz, COIL-20, COIL-100, and diatoms. The new method obtains better or equal classification performance to the best competitor with a 5 to 9-fold speed gain.

The Max-Tree designed for morphological attribute filtering in image processing, is a data structure in which the nodes represent connected components for all threshold levels in a data set. Attribute filters compute some attribute describing the shape or size of each connected component and then decide which components to keep or to discard. In this paper, we augment the basic Max-Tree data structure such that interactive volumetric filtering and visualization becomes possible. We introduce extensions that allow (1) direct, splatting-based, volume rendering; (2) representation of the Max-Tree on graphics hardware; and (3) fast active cell selection for isosurface generation. In all three cases, we can use the Max-Tree representation for visualization directly, without needing to reconstruct the volumetric data explicitly. We show that both filtering and visualization can be performed at interactive frame rates, ranging between 2.4 and 32 frames per seconds. In contrast, a standard texture-based volume visualization method manages only between 0.5 and 1.8 frames per second. For isovalue browsing, the experimental results show that the performance is comparable to the performance of an interval tree, where our method has the advantage that both filter threshold browsing and isolevel browsing are fast. It is shown that the methods using graphics hardware can be extended to other connected filters.

An efficient algorithm is presented for the computation of gray-scale morphological operations with 2-D structuring elements (S.E.). The required computing time is independent of the image content and of the number of gray levels used. For circular S.E.s, it always outperforms the only existing comparable method, which was proposed by Van Droogenbroeck and Talbot (1996), by a factor between 1.8 and 8.6, depending on the image type. So far, filtering using multiple S.E.s is always done by performing the operator for each size and shape of the S.E. separately. With our method filtering with multiple S.E.s can be performed by a single operator for a reduced computational cost per size or shape, which makes this method more suitable for use in granulometries, dilation-erosion scale spaces, and template matching using the hit-or-miss transform.

A multiscale, morphological method for the purpose of shape-based object recognition is presented. A connected operator similar to the morphological hat-transform is defined, and two scale-space representations are built, using the curvature function as the underlying one-dimensional signal. Each peak and valley of the curvature is extracted and described by its maximum and average heights and by its extent and represents an entry in the top or bottom hat-transform scale spaces. We demonstrate object recognition based on hat-transform scale spaces for three large data sets, a set of diatom contours, the set of silhouettes from the MPEG-7 database and the set of two-dimensional views of three-dimensional objects from the COIL-20 database. Our approach outperforms other methods for which comparative results exist.

A new theoretical development is presented for handling the over-segmentation problem in partitioning-based connected openings. The definition we propose treats singletons generated with the earlier method, as elements of a larger connected component. Unlike the existing formalism, this new method allows detection of filamentous structures linking larger objects. This is demonstrated in attribute filtering on neuron images.

A novel, physically motivated deformable model for shape recovery and segmentation is presented. The model, referred to as the charged-particle model (CPM), is inspired by classical electrodynamics and is based on a simulation of charged particles moving in an electrostatic field. The charges are attracted towards the contours of the objects of interest by an electrostatic field, whose sources are computed based on the gradient-magnitude image. The electric field plays the same role as the potential forces in the snake model, while internal interactions are modeled by repulsive Coulomb forces. We demonstrate the flexibility and potential of the model in a wide variety of settings: shape recovery using manual initialization, automatic segmentation, and skeleton computation. We perform a comparative analysis of the proposed model with the active contour model and show that specific problems of the latter are surmounted by our model. The model is easily extendable to 3D and copes well with noisy images.

In this paper we present a multiscale morphological method for use in texture classification. A connected operator similar to the morphological hat-transform is defined, and two scale-space representations are built. The most important features are extracted from the scale spaces by unsupervised cluster analysis, and the resulting pattern vectors provide the input of a decision tree classifier. We obtain 93.5 % correct classification for the Brodatz texture database.

Two moving-window methods, using either flat or Gaussian weighted windows, for local thresholding with robust automatic threshold selection are developed. The results show that fast segmentation of blood vessels against a varying background and noise is possible at modest computational cost. Volumes of 128 x 256/sup 2/ and 256/sup 3/ can be segmented in 3.1 s and 6.6 s, for flat, and 12.6 s and 30.8 s for Gaussian windows, respectively, on a 1.9 GHz Pentium 4.

The implementation of morphological connected set operators for image filtering and pattern recognition is discussed. Two earlier algorithms based on priority queues and hierarchical queues, respectively, are compared to a more recent union-find approach. Unlike the earlier algorithms which process regional extrema in the image sequentially, the union-find method allows simultaneous processing of extrema. In the context of area openings, closings, and pattern spectra, the union-find algorithm outperforms the previous methods on almost all natural and synthetic images tested. Finally, extensions to pattern spectra and the more general class of attribute operators are presented for all three algorithms, and memory usages are compared.

An area based counterpart of the binary structural opening spectra is developed. It is shown that these area opening and closing spectra can be computed using an adaptation of Tarjan's (1975) union-find algorithm. These spectra provide rotation, translation, and scale invariant pattern vectors for texture analysis.

A method for shape analysis of diatoms (single-cell algae with silica shells) based on extraction of features on the contour of the cells by multi-scale mathematical morphology is presented. After building a morphological contour curvature scale space, we present a method for extracting the most prominent features by unsupervised cluster analysis. The number of extracted features matches well with those found visually in 92% of the 350 diatom images examined.