The required number of users’ actions and the response time can critically affect user experience during interactive image segmentation. In this work, we revisit a recent graph-based algorithm, namely Dynamic Trees (DT), which has shown to be more effective than several well-established methods from the literature of graph-based image segmentation. DT solves segmentation by growing optimum-path trees rooted at seed pixels, such that the arc weights are estimated on the fly from image properties of the growing trees, defining the objects as optimum-path forests rooted at their internal seeds. Depending on the application (e.g., 3D medical images), the response time to correct segmentation by adding and removing seeds can seriously compromise the method’s efficiency. We present a differential dynamic trees (DDT) algorithm that adds and removes trees updating optimum paths only in the required regions of the image. We demonstrate that the DDT algorithm can preserve the high effectiveness of DT, being one order of magnitude faster than DT. The experiments also show the advantages of DDT over those well-established counterparts.

Oil and gas reservoirs are well studied in petroleum engineering, using seismic data to estimate fluid flow and well placement. However, seismic data cannot capture fractures due to their scale, and fractures may affect rock porosity and permeability. Consequently, rock fracture segmentation and quantification from aerial images of analogous outcrops can input essential information into those studies. This paper presents a new method, namedinteractive Forest Growing(iFG), for fracture segmentation. The image is initially segmented into superpixels, defining a superpixel graph. The user selects seed superpixels, a path-cost threshold, and fractures are delineated by growing one optimum-path tree from each seed with path costs limited to the selected threshold. iFG considerably increases efficiency while reducing human effort in fracture segmentation compared to pixel-by-pixel manual annotation. We evaluate iFG with three specialists and against aninteractive Region Growing(iRG) method using 15 images to measure bias in user interpretations, verify efficiency gain over a similar approach, and generate a dataset with consolidated annotation for future work. The experiments show that iFG reduces user interventions from 19% to 33% compared to iRG, users with more experience in fracture analysis complete segmentation 4-5 times faster, and segmentation effectiveness is independent of user experience since the average F1 scores between users uing both methods ranged from 0.966 to 0.979, allowing us to create a consolidated segmentation.

Image segmentation is a challenging and essential task in remote sensing. Deep neural networks (DNNs) have successfully segmented images from different domains, but the models usually require time-consuming and expensive pixel-level data annotation. In this letter, we exploit a recent technique to learn features (an encoder) from a few markers placed by the user in relevant image regions, build an encoder–decoder model from a small set of regions delineated by click-based segmentation, and use that model to annotate the remaining pixels. Such user-guided data expansion modeling can be repeated as the encoder–decoder network improves, and by selecting well-annotated regions, the user considerably expands the pixel set to train DNNs with little supervision. We show the role of feature learning from image markers (FLIM) and that our data expansion model can significantly improve the generalization performance of a state-of-the-art DNN when segmenting buildings in aerial images of distinct cities.

Identifying species of trees in aerial images is essential for land-use classification, plantation monitoring, and impact assessment of natural disasters. The manual identification of trees in aerial images is tedious, costly, and error-prone, so automatic classification methods are necessary. Convolutional neural network (CNN) models have well succeeded in image classification applications from different domains. However, CNN models usually require intensive manual annotation to create large training sets. One may conceptually divide a CNN into convolutional layers for feature extraction and fully connected layers for feature space reduction and classification. We present a method that needs a minimal set of user-selected images to train the CNN’s feature extractor, reducing the number of required images to train the fully connected layers. The method learns the filters of each convolutional layer from user-drawn markers in image regions that discriminate classes, allowing better user control and understanding of the training process. It does not rely on optimization based on backpropagation, and we demonstrate its advantages on the binary classification of coconut-tree aerial images against one of the most popular CNN models.

John E. Vargas Muñoza*, Devis Tuiab & Alexandre X. Falcãoaa Laboratory of Image Data Science, Institute of Computing University of Campinas, Campinas, Brazilb Laboratory of Geo-information Science and Remote Sensing, Wageningen University & Research, Wageningen, The NetherlandsJohn E. Vargas Muñoz received the B.Sc. degree in informatics engineering from the National University of San Antonio Abad in Cusco, Cusco, Peru, in 2010, and the master's degree in computer science from the University of Campinas, Campinas, Brazil, in 2015. During 2017-2018, he worked on applications of machine learning to open geographical data, as a visiting Ph.D. student, in the Laboratory of Geo-information Science and Remote Sensing at Wageningen University, the Netherlands. In 2019, he received a Ph.D. in computer science from the University of Campinas, Campinas, Brazil. His research interests include machine learning, image processing, remote sensing image classification, and crowdsourced geographic information analysis.Devis Tuia (S'07, M’09, SM’15) received the Ph.D in environmental sciences at the University of Lausanne, Switzerland, in 2009. He was a Postdoc at the University of Valencia, the University of Colorado, Boulder, CO and EPFL Lausanne. Between 2014 and 2017, he was Assistant Professor at the University of Zurich. He is now Full Professor at the Geo-Information Science and Remote Sensing Laboratory at Wageningen University, the Netherlands. He is interested in algorithms for information extraction and data fusion of geospatial data (including remote sensing) using machine learning and computer vision. He serves as Associate Editor for IEEE TGRS and the Journal of the ISPRS. More info on http://devis.tuia.googlepages.com/Alexandre X. Falcão is full professor at the Institute of Computing, University of Campinas, Campinas, SP, Brazil. He received a B.Sc. in Electrical Engineering from the Federal University of Pernambuco, Recife, PE, Brazil, in 1988. He has worked in biomedical image processing, visualization and analysis since 1991. In 1993, he received a M.Sc. in Electrical Engineering from the University of Campinas, Campinas, SP, Brazil. During 1994-1996, he worked with the Medical Image Processing Group at the Department of Radiology, University of Pennsylvania, PA, USA, on interactive image segmentation for his doctorate. He got his doctorate in Electrical Engineering from the University of Campinas in 1996. In 1997, he worked in a project for Globo TV at a research center, CPqD-TELEBRAS in Campinas, developing methods for video quality assessment. His experience as professor of Computer Science and Engineering started in 1998 at the University of Campinas. His main research interests include image/video processing, visualization, and analysis; graph algorithms and dynamic programming; image annotation, organization, and retrieval; machine learning and pattern recognition; and image analysis applications in Biology, Medicine, Biometrics, Geology, and Agriculture.CONTACT John E. Vargas Muñoz [email protected] populations in rural areas of developing countries has attracted the attention of humanitarian mapping projects since it is important to plan actions that affect vulnerable areas. Recent efforts have tackled this problem as the detection of buildings in aerial images. However, the quality and the amount of rural building annotated data in open mapping services like OpenStreetMap (OSM) is not sufficient for training accurate models for such detection. Although these methods have the potential of aiding in the update of rural building information, they are not accurate enough to automatically update the rural building maps. In this paper, we explore a human-computer interaction approach and propose an interactive method to support and optimize the work of volunteers in OSM. The user is asked to verify/correct the annotation of selected tiles during several iterations and therefore improving the model with the new annotated data. The experimental results, with simulated and real user annotation corrections, show that the proposed method greatly reduces the amount of data that the volunteers of OSM need to verify/correct. The proposed methodology could benefit humanitarian mapping projects, not only by making more efficient the process of annotation but also by improving the engagement of volunteers.

As constituent parts of image objects, superpixels can improve several higher-level operations. However, image segmentation methods might have their accuracy severely compromised for reduced numbers of superpixels. To mitigate the problem, we introduce Dynamic Iterative Spanning Forest (DISF), a seed-based method that improves all components in the Iterative Spanning Forest (ISF) framework for superpixel segmentation. DISF relies on a new strategy for seed estimation that can find more relevant seeds, reconstruct relevant edges along with iterations, and guarantee the desired number of superpixels. DISF also assures optimal spanning forests for path costs based on dynamic arc-weight estimation, being faster as the desired number of superpixels grows. We show that DISF can improve effectiveness on three datasets with distinct object properties, requiring significantly fewer iterations than all seed-based baselines.

The detection and counting of coconut trees in aerial images are important tasks for environment monitoring and post-disaster assessment. Recent deep-learning-based methods can attain accurate results, but they require a reasonably high number of annotated training samples. In order to obtain such large training sets with considerably reduced human effort, we present a semi-automatic sample annotation method based on the 2D t-SNE projection of the sample feature space. The proposed approach can facilitate the construction of effective training sets more efficiently than using the traditional manual annotation, as shown in our experimental results with VHR images from the Kingdom of Tonga.

Finding the ideal number of layers and size for each layer is a key challenge in deep neural network design. Two approaches for such networks exist: filter learning and architecture learning. While the first one starts with a given architecture and optimizes model weights, the second one aims to find the best architecture. Recently, several visual analytics (VA) techniques have been proposed to understand the behavior of a network, but few VA techniques support designers in architectural decisions. We propose a hybrid methodology based on VA to improve the architecture of a pre-trained network by reducing/increasing the size and number of layers. We introduce Activation Occurrence Maps that show how likely each image position of a convolutional kernel’s output activates for a given class, and Class Selectivity Maps, that show the selectiveness of different positions in a kernel’s output for a given label. Both maps help in the decision to drop kernels that do not significantly add to the network’s performance, increase the size of a layer having too few kernels, and add extra layers to the model. The user interacts from the first to the last layer, and the network is retrained after each layer modification. We validate our approach with experiments in models trained with two widely-known image classification datasets and show how our method helps to make design decisions to improve or to simplify the architectures of such models.

Superpixel segmentation has emerged as an important research problem in the areas of image processing and computer vision. In this paper, we propose a framework, namely Iterative Spanning Forest (ISF), in which improved sets of connected superpixels (supervoxels in 3D) can be generated by a sequence of image foresting transforms. In this framework, one can choose the most suitable combination of ISF components for a given application-i.e., 1) a seed sampling strategy; 2) a connectivity function; 3) an adjacency relation; and 4) a seed pixel recomputation procedure. The superpixels in ISF structurally correspond to spanning trees rooted at those seeds. We present five ISF-based methods to illustrate different choices for those components. These methods are compared with a number of state-of-the-art approaches with respect to effectiveness and efficiency. Experiments are carried out on several datasets containing 2D and 3D objects with distinct texture and shape properties, including a high-level application, named sky image segmentation. The theoretical properties of ISF are demonstrated in the supplementary material and the results show ISF-based methods rank consistently among the best for all datasets.

Mapping rural buildings in developing countries is crucial to monitor and plan in those vulnerable areas. Despite the existence of some rural building annotations in OpenStreetMap (OSM), those are of insufficient quantity and quality to train models able to map large areas accurately. In particular, these annotations are very often misaligned with respect to the buildings that are present in updated aerial imagery. We propose a Markov Random Field (MRF) method to correct misaligned rural building annotations. To do so, our method uses i) the correlation between candidate aligned OSM annotations and buildings roughly detected on aerial images and ii) the local consistency of the alignment vectors.

In this paper, we propose a post classification smoothing method aimed at improving the accuracy and visual appearance of sub-decimeter image classification results. Starting from the class confidence maps of a supervised classifier, we find a set of high confidence markers and propagate labels on an extended region adjacency graph. We apply the proposed method on a challenging 5cm resolution dataset over Potsdam, Germany. The proposed algorithm outperforms state-of-the-art post classification smoothing algorithms both when the classifier is trained specifically on the image and when it is trained and tested in different set of images.

In machine learning, pattern classification assigns high-dimensional vectors (observations) to classes based on generalization from examples. Artificial neural networks currently achieve state-of-the-art results in this task. Although such networks are typically used as black-boxes, they are also widely believed to learn (high-dimensional) higher-level representations of the original observations. In this paper, we propose using dimensionality reduction for two tasks: visualizing the relationships between learned representations of observations, and visualizing the relationships between artificial neurons. Through experiments conducted in three traditional image classification benchmark datasets, we show how visualization can provide highly valuable feedback for network designers. For instance, our discoveries in one of these datasets (SVHN) include the presence of interpretable clusters of learned representations, and the partitioning of artificial neurons into groups with apparently related discriminative roles.

The performance of pattern classifiers depends on the separability of the classes in the feature space - a property related to the quality of the descriptors - and the choice of informative training samples for user labeling - a procedure that usually requires active learning. This work is devoted to improve the quality of the descriptors when samples are superpixels from remote sensing images. We introduce a new scheme for superpixel description based on Bag of visual Words, which includes information from adjacent superpixels, and validate it by using two remote sensing images and several region descriptors as baselines.

Biometrics systems have significantly improved person identification and authentication, playing an important role in personal, national, and global security. However, these systems might be deceived (or spoofed) and, despite the recent advances in spoofing detection, current solutions often rely on domain knowledge, specific biometric reading systems, and attack types. We assume a very limited knowledge about biometric spoofing at the sensor to derive outstanding spoofing detection systems for iris, face, and fingerprint modalities based on two deep learning approaches. The first approach consists of learning suitable convolutional network architectures for each domain, whereas the second approach focuses on learning the weights of the network via back propagation. We consider nine biometric spoofing benchmarks - each one containing real and fake samples of a given biometric modality and attack type - and learn deep representations for each benchmark by combining and contrasting the two learning approaches. This strategy not only provides better comprehension of how these approaches interplay, but also creates systems that exceed the best known results in eight out of the nine benchmarks. The results strongly indicate that spoofing detection systems based on convolutional networks can be robust to attacks already known and possibly adapted, with little effort, to image-based attacks that are yet to come.

Recent advances in interactive image segmentation focused on eliminating the user bias during evaluation by simulating their behavior using robot users. However, these robots only work for region-based methods, excluding the important class of approaches that rely on the boundary-tracking paradigm. We propose completely novel robot users that are able to simulate the human user behavior when segmenting an image through the addition of anchor points close to the object's boundary. Our robots constantly evaluate the optimum-boundary segment being computed from a previously selected anchor point to the current virtual mouse position, seeking for the longest possible segment with minimum acceptable error. A new anchor is added when the error is too high and the robots iterate until closing the contour, just like real users. We validate our robots by conducting a user study and extensive experiments, considering two boundary-tracking methods: live-wire-on-the-fly and Riverbed. We further show how our robot users can be used to assess hybrid approaches that combine boundary-tracking with region-based delineation, such as LiveMarkers, while conjecturing that robots might lead to new methods for automatic foreground segmentation.

In this paper, we show how to improve the Radial Basis Function Neural Networks effectiveness by using the Optimum-Path Forest clustering algorithm, since it computes the number of clusters on-the-fly, which can be very interesting for finding the Gaussians that cover the feature space. Some commonly used approaches for this task, such as the well-known fc-means, require the number of classes/clusters previous its performance. Although the number of classes is known in supervised applications, the real number of clusters is extremely hard to figure out, since one class may be represented by more than one cluster. Experiments over 9 datasets together with statistical analysis have shown the suitability of OPF clustering for the RBF training step.

The development of effective and efficient ways of handling real-world applications is becoming increasingly widespread, yet it still faces a number of practical challenges. First and foremost, we have the limited availability of labeled data in contrast to an unbounded number of unlabeled ones. Despite some efforts in active semi-supervised learning, their success depends on an approach suitable to be applied to real massive data. In this paper, we introduce a novel integration of semi-supervised learning and a priori-reduction and organization criteria for active learning based on Optimum-Path Forest classifiers. Encouraging results on both public and real data show the synergy of these strategies jointly. Our approach iteratively generates semi-supervised classifiers that attain high accuracy by selecting the most representative labeled set, while decreasing the propagated errors on the unlabeled set. In addition, it is able to identify samples from all classes quickly while keeping user interaction to a minimum throughout the learning iterations.

Land cover classification has been paramount in the last years. Since the amount of information acquired by satellite on-board imaging systems has increased, there is a need for automatic tools that can tackle such problem. Despite the fact that one can find several works in the literature, we propose a novel methodology for land cover classification by means of the optimum-path forest (OPF) framework, which has never been applied to this context up to date. Experiments were conducted in supervised and unsupervised situations against some state-of-the-art pattern recognition techniques, such as support vector machines, Bayesian classifier, k-means, and mean shift. We had shown that supervised OPF can outperform such approaches, being much faster than all. In regard to clustering techniques, all classifiers have achieved similar results.

Humans are natural face recognition experts, far out-performing current automated face recognition algorithms, especially in naturalistic, “in the wild” settings. However, a striking feature of human face recognition is that we are dramatically better at recognizing highly familiar faces, presumably because we can leverage large amounts of past experience with the appearance of an individual to aid future recognition. Meanwhile, the analogous situation in automated face recognition, where a large number of training examples of an individual are available, has been largely underexplored, in spite of the increasing relevance of this setting in the age of social media. Inspired by these observations, we propose to explicitly learn enhanced face representations on a per-individual basis, and we present two methods enabling this approach. By learning and operating within person-specific representations, we are able to significantly outperform the previous state-of-the-art on PubFig83, a challenging benchmark for familiar face recognition in the wild, using a novel method for learning representations in deep visual hierarchies. We suggest that such person-specific representations aid recognition by introducing an intermediate form of regularization to the problem.

Interactive image segmentation methods normally rely on cues about the foreground imposed by the user as region constraints (markers/brush strokes) or boundary constraints (anchor points). These paradigms often have complementary strengths and weaknesses, which can be addressed to improve the interactive experience by reducing the user’s effort. We propose a novel hybrid paradigm based on a new form of interaction called live markers, where optimum boundary-tracking segments are turned into internal and external markers for region-based delineation to effectively extract the object. We present four techniques within this paradigm: 1) LiveMarkers; 2) RiverCut; 3) LiveCut; and 4) RiverMarkers. The homonym LiveMarkers couples boundary-tracking via live-wire-on-the-fly (LWOF) with optimum seed competition by the image foresting transform (IFT-SC). The IFT-SC can cope with complex object silhouettes, but presents a leaking problem on weaker parts of the boundary that is solved by the effective live markers produced by LWOF. Conversely, in RiverCut, the long boundary segments computed by Riverbed around complex shapes provide markers for Graph Cuts by the Min-Cut/Max-Flow algorithm (GCMF) to complete segmentation on poorly defined sections of the object’s border. LiveCut and RiverMarkers further demonstrate that live markers can improve segmentation even when the combined approaches are not complementary (e.g., GCMFs shrinking bias is also dramatically prevented when using it with LWOF). Moreover, since delineation is always region based, our methodology subsumes both paradigms, representing a new way of extending boundary tracking to the 3D image domain, while speeding up the addition of markers close to the object’s boundary-a necessary but time consuming task when done manually. We justify our claims through an extensive experimental evaluation on natural and medical images data sets, using recently proposed robot users for boundary-tracking methods.

Many methods have been recently proposed to deal with the large amount of data provided by high-resolution remote sensing technologies. Several of these methods rely on the use of image segmentation algorithms for delineating target objects. However, a common issue in geographic object-based applications is the definition of the appropriate data representation scale, a problem that can be addressed by exploiting multiscale segmentation. The use of multiple scales, however, raises new challenges related to the definition of effective and efficient mechanisms for extracting features. In this paper, we address the problem of extracting histogram-based features from a hierarchy of regions for multiscale classification. The strategy, called H-Propagation, exploits the existing relationships among regions in a hierarchy to iteratively propagate features along multiple scales. The proposed method speeds up the feature extraction process and yields good results when compared with global low-level extraction approaches.

We introduce an image segmentation algorithm GCsummax, which combines, in a novel manner, the strengths of two popular algorithms: Relative Fuzzy Connectedness (RFC) and (standard) Graph Cut (GC). We show, both theoretically and experimentally, that GCsummaxpreserves robustness of RFC with respect to the seed choice (thus, avoiding “shrinking problem” of GC), while keeping GC's bigger control over “leaking though the weak boundary.” The theoretical analysis of GCsummaxis greatly facilitated by our recent theoretical results that RFC belongs to the Generalized GC (GGC) segmentation algorithms framework. In our implementation of GCsummaxwe use, as a subroutine, a version of RFC algorithm (based on Image Foresting Transform) that runs (provably) in linear time with respect to the image size. This results in GCsummaxrunning in a time close to linear.

Coffee crop recognition in remote sensing images is a complex task. It poses several challenges due to different spectral responses and texture patterns that can be extracted from coffee regions. This paper presents a novel framework for combining different classifiers using support vector machine technique (SVM), which try to learn with each one of classifiers previews experiences (meta-learning). We investigate the combination of seven learning methods and seven image descriptors aiming at creating low-cost classifiers for coffee crops recognition. The objective is to provide an effective mechanism for coffee crop recognition by fusion of region-based classifiers in remote sensing images. The experiments showed that the proposed framework for fusion of classifiers produces better results than the traditional majority voting fusion approach and all base classifiers tested.

We have developed interactive tools for graph-based segmentation of natural images, in which the user guides object delineation by drawing strokes (markers) inside and outside the object. A suitable arc-weight estimation is paramount to minimize user time and maximize segmentation accuracy in these tools. However, it depends on discriminative image properties for object and background. These properties can be obtained from some marker pixels, but their identification is a hard problem during delineation. Careless arc-weight re-estimation reduces user control and drops performance, while interactive arc-weight estimation in a step before interactive object extraction is the best option so far, albeit it is not intuitive for nonexpert users. We present an effective solution using the unified framework of the image foresting transform (IFT) with three operators: clustering for interpreting user interaction and determining when and where arc weights need to be re-estimated; fuzzy classification for arc-weight estimation; and marker competition based on optimum connectivity for object extraction. For validation, we compared the proposed approach with another interactive IFT-based method, which computes arc weights before extraction. Evaluation involved multiple users (experts and nonexperts), a dataset with several natural images, and measurements to quantify accuracy, precision, efficiency (user time and computation time), and user control, being some of them novel measurements, proposed in this work.

A huge effort has been applied in image classification to create high-quality thematic maps and to establish precise inventories about land cover use. The peculiarities of remote sensing images (RSIs) combined with the traditional image classification challenges made RSI classification a hard task. Our aim is to propose a kind of boost-classifier adapted to multiscale segmentation. We use the paradigm of boosting, whose principle is to combine weak classifiers to build an efficient global one. Each weak classifier is trained for one level of the segmentation and one region descriptor. We have proposed and tested weak classifiers based on linear support vector machines (SVM) and region distances provided by descriptors. The experiments were performed on a large image of coffee plantations. We have shown in this paper that our approach based on boosting can detect the scale and set of features best suited to a particular training set. We have also shown that hierarchical multiscale analysis is able to reduce training time and to produce a stronger classifier. We compare the proposed methods with a baseline based on SVM with radial basis function kernel. The results show that the proposed methods outperform the baseline.

This paper presents an optimum user-steered boundary tracking approach for image segmentation, which simulates the behavior of water flowing through a riverbed. The riverbed approach was devised using the image foresting transform with a never-exploited connectivity function. We analyze its properties in the derived image graphs and discuss its theoretical relation with other popular methods such as live wire and graph cuts. Several experiments show that riverbed can significantly reduce the number of user interactions (anchor points), as compared to live wire for objects with complex shapes. This paper also includes a discussion about how to combine different methods in order to take advantage of their complementary strengths.

In this paper we deal with the problem of feature selection by introducing a new approach based on Gravitational Search Algorithm (GSA). The proposed algorithm combines the optimization behavior of GSA together with the speed of Optimum-Path Forest (OPF) classifier in order to provide a fast and accurate framework for feature selection. Experiments on datasets obtained from a wide range of applications, such as vowel recognition, image classification and fraud detection in power distribution systems are conducted in order to asses the robustness of the proposed technique against Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA) and a Particle Swarm Optimization (PSO)-based algorithm for feature selection.

Most face recognition methods rely on a common feature space to represent the faces, in which the face aspects that better distinguish among all the persons are emphasized. This strategy may be inadequate to represent more appropriate aspects of a specific person's face, since there may be some aspects that are good at distinguishing only a given person from the others. Based on this idea and sup- ported by some findings in the human perception of faces, we propose a face recognition framework that associates a feature space to each person that we intend to recognize. Such feature spaces are conceived to underline the discriminating face aspects of the persons they represent. In order to recognize a probe, we match it to the gallery in all the feature spaces and fuse the results to establish the identity. With the help of an algorithm that we devise, the Discriminant Patch Selection, we were capable of carrying out experiments to intuitively compare the traditional approaches with the person-specific representation. In the performed experiments, the person-specific face representation always resulted in a better identification of the faces.

Automatic MR-image segmentation of brain tissues is an important issue in neuroimaging. For instance, it is a key methodological component of a popular technique denominated voxel-based morphometry (VBM), which quantifies gray-matter (GM) volumes from MR images. However, segmentation accuracy in some subcortical regions on the basis of extant methods is not satisfactory, compromising VBM results. We combine a probabilistic atlas and a fast clustering approach based on optimum connectivity between voxels in their feature space. The algorithm exploits local image properties and global information from the atlas as features to group GM and white-matter (WM) voxels in distinct clusters, and uses the total probability values inside the clusters to label them as GM or WM. This new method is validated in the region of the thalamus and outperformed two widely used methods packaged in SPM and FSL.

This work presents an optimum user-steered boundary tracking approach for image segmentation, which simulates the behavior of water flowing through a riverbed. The riverbed approach was devised using the Image Foresting Transform with a never exploited connectivity function. We analyze its properties in the derived image graphs and discuss its theoretical relation with other popular methods, such as live wire and graph cuts. Riverbed can significantly reduce the number of user interactions (anchor points) as compared to live wire for objects with complex shapes.

Although non-technical losses automatic identification has been massively studied, the problem of selecting the most representative features in order to boost the identification accuracy has not attracted much attention in this context. In this paper, we focus on this problem applying a novel feature selection algorithm based on Particle Swarm Optimization and Optimum-Path Forest. The results demonstrated that this method can improve the classification accuracy of possible frauds up to 49% in some datasets composed by industrial and commercial profiles.

The applications of Automatic Vowel Recognition (AVR), which is a sub-part of fundamental importance in most of the speech processing systems, vary from automatic interpretation of spoken language to biometrics. State-of-the-art systems for AVR are based on traditional machine learning models such as Artificial Neural Networks (ANNs) and Support Vector Machines (SVMs), however, such classifiers can not deal with efficiency and effectiveness at the same time, existing a gap to be explored when real-time processing is required. In this work, we present an algorithm for AVR based on the Optimum-Path Forest (OPF), which is an emergent pattern recognition technique recently introduced in literature. Adopting a supervised training procedure and using speech tags from two public datasets, we observed that OPF has outperformed ANNs, SVMs, plus other classifiers, in terms of training time and accuracy.

Traditional pattern recognition techniques can not handle the classification of large datasets with both efficiency and effectiveness. In this context, the Optimum-Path Forest (OPF) classifier was recently introduced, trying to achieve high recognition rates and low computational cost. Although OPF was much faster than Support Vector Machines for training, it was slightly slower for classification. In this paper, we present the Efficient OPF (EOPF), which is an enhanced and faster version of the traditional OPF, and validate it for the automatic recognition of white matter and gray matter in magnetic resonance images of the human brain.

The decision correctness in expert systems strongly depends on the accuracy of a pattern classifier, whose learning is performed from labeled training samples. Some systems, however, have to manage, store, and process a large amount of data, making also the computational efficiency of the classifier an important requirement. Examples are expert systems based on image analysis for medical diagnosis and weather forecasting. The learning time of any pattern classifier increases with the training set size, and this might be necessary to improve accuracy. However, the problem is more critical for some popular methods, such as artificial neural networks and support vector machines (SVM), than for a recently proposed approach, the optimum-path forest (OPF) classifier. In this letter, we go beyond by presenting a robust approach to reduce the training set size and still preserve good accuracy in OPF classification. We validate the method using some data sets and for rainfall occurrence estimation based on satellite image analysis. The experiments use SVM and OPF without pruning of training patterns as baselines.

This paper proposes a new texture descriptor to guide the search and retrieval in image databases. It extracts rich information from global and local primitives of textured images. At a higher level, the global macro-features in textured images are characterized by exploiting the multiresolution properties of the Steerable Pyramid Decomposition. By doing this, the global texture configurations are highlighted. At a finer level, the local arrangements of texture micro-patterns are encoded by the Local Binary Pattern operator.Experiments were carried out on the standard Vistex dataset aiming to compare our descriptors against popular texture extraction methods with regard to their retrieval accuracies. The comparative evaluations allowed us to show the superior descriptive properties of our feature representation methods.

The image foresting transform (IFT) is e a generic technique that uses simple variations of the same core algorithm to construct many image processing operators like watershed transforms, edge tracking, geodesic paths, among others. In this paper we propose the silicon IFT (SIFT), an FPGA-based architecture that leverages on the IFT flexibility to build a fast image processing architecture capable of implementing IFT operators in hardware. Our experiments have shown that SIFT can reach speedups of 5600 upon the correspondent software implementation. Moreover, they exhibit excellent execution times as compared to recent dedicated image processing architectures.

Tensor Scale is a morphometric parameter that unifies the representation of local structure thickness, orientation, and anisotropy, which can be used in several image processing tasks. This paper introduces a new application for tensor scale, which is the detection of saliences on a given contour, based on the tensor scale orientations computed for the entire object and mapped to its contour. For validation purposes, we present a shape descriptor that uses the detected contour saliences. Experimental results are provided, comparing the proposed method with our previous Contour Salience Descriptor (CS). We show that the proposed method can be not only faster and more robust in the detection of salience points than the CS method, but also more effective as a shape descriptor.

In this paper, we propose a novel framework using Genetic Programming to combine image database descriptors for content-based image retrieval (CBIR). Our framework is validated through several experiments involving two image databases and specific domains, where the images are retrieved based on the shape of their objects.

The absence of object information very often asks for considerable human assistance in medical image segmentation. Many interactive two-dimensional and three-dimensional (3-D) segmentation methods have been proposed, but their response time to user's actions should be considerably reduced to make them viable from the practical point of view. We circumvent this problem in the framework of the image foresting transform (IFT)--a general tool for the design of image operators based on connectivity--by introducing a new algorithm (DIFT) to compute sequences of IFTs in a differential way. We instantiate the DIFT algorithm for watershed-based and fuzzy-connected segmentations under two paradigms (single-object and multiple-object) and evaluate the efficiency gains of both approaches with respect to their linear-time implementation based on the nondifferential IFT. We show that the DIFT algorithm provides efficiency gains from 10 to 17, reducing the user's waiting time for segmentation with 3-D visualization on a common PC from 19-36 s to 2-3 s. We also show that the multiple-object approach is more efficient than the single-object paradigm for both segmentation methods.

In many medical imaging applications, due to the limited field of view of imaging devices, acquired images often include only a part of a structure. In such situations, it is impossible to guarantee that the images will contain exactly the same physical extent of the structure at different scans, which leads to difficulties in registration and in many other tasks, such as the analysis of the morphology, architecture, and kinematics of the structures. To facilitate such analysis, we developed a general method, referred to as iso-shaping, that generates structures of the same shape from segmented image sequences. The basis for this method is to automatically find a set of key points, called shape centers, in the segmented partial anatomic structure such that these points are present in all images and that they represent the same physical location in the object, and then trim the structure using these points as reference. The application area considered here is the analysis of the morphology, architecture, and kinematics of the joints of the foot from magnetic resonance images acquired at different joint positions and load conditions. The accuracy of the method is analyzed by utilizing ten data sets for iso-shaping the tibia and the fibula via four evaluative experiments. The analysis indicates that iso-shaping produces results as predicted by the theoretical framework.

In this work we discuss an approach to characterize the human cortex in MRI data through the segmentation of the cortical layer with deep penetration into the sulci, using the Image Foresting Transform. This technique, based on graph theory, provides a sound framework for the implementation of many image processing operators, and has been successfully applied to the solution of many problems, including some in medical imaging. We show the results of applying the technique to slices of MRI data from four subjects and compare these to the computation of the morphological skeleton.

This paper presents \bic (Border/Interior pixel Classification), a compact and efficient CBIR approach suitable for broad image domains. It has three main components: (1) a simple and powerful image analysis algorithm that classifies image pixels as either border or interior, (2) a new logarithmic distance (dLog) for comparing histograms, and (3) a compact representation for the visual features extracted from images. Experimental results show that the BIC approach is consistently more compact, more efficient and more effective than state-of-the-art CBIR approaches based on sophisticated image analysis algorithms and complex distance functions. It was also observed that the dLog distance function has two main advantages over vectorial distances (e.g., L1): (1) it is able to increase substantially the effectiveness of (several) histogram-based CBIR approaches and, at the same time, (2) it reduces by 50% the space requirement to represent a histogram.

The authors present a novel content based image retrieval (CBIR) approach, for image databases, based on cluster analysis. CBIR relies on the representation (metadata) of images' visual content. In order to produce such metadata, we propose an efficient and adaptive clustering algorithm to segment the images into regions of high similarity. This approach contrasts with those that use a single color histogram for the whole image (global methods), or local color histograms for a fixed number of image cells (partition based methods). Our experimental results show that our clustering approach offers high retrieval effectiveness with low space overhead. For example, using a database of 20000 images, we obtained higher retrieval effectiveness than partition based methods with about the same space overhead of global methods, which are typically regarded as storage-wise compact.

We have been developing user-steered image segmentation methods for situations which require considerable human assistance in object definition. In the past, we have presented two paradigms, referred to as live-wire and live-lane, for segmenting 2D/3D/4D object boundaries in a slice-by-slice fashion, and demonstrated that live-wire and live-lane are more repeatable, with a statistical significance level of P < 0.03, and are 1.5-2.5 times faster, with a statistical significance level of P < 0.02, than manual tracing. In this paper, we introduce a 3D generalization of the live-wire approach for segmenting 3D/4D object boundaries which further reduces the time spent by the user in segmentation. In a 2D live-wire, given a slice, for two specified points (pixel vertices) on the boundary of the object, the best boundary segment is the minimum-cost path between the two points, described as a set of oriented pixel edges. This segment is found via Dijkstra's algorithm as the user anchors the first point and moves the cursor to indicate the second point. A complete 2D boundary is identified as a set of consecutive boundary segments forming a "closed", "connected", "oriented" contour. The strategy of the 3D extension is that, first, users specify contours via live-wiring on a few slices that are orthogonal to the natural slices of the original scene. If these slices are selected strategically, then we have a sufficient number of points on the 3D boundary of the object to subsequently trace optimum boundary segments automatically in all natural slices of the 3D scene. A 3D object boundary may define multiple 2D boundaries per slice. The points on each 2D boundary form an ordered set such that when the best boundary segment is computed between each pair of consecutive points, a closed, connected, oriented boundary results. The ordered set of points on each 2D boundary is found from the way the users select the orthogonal slices. Based on several validation studies involving segmentation of the bones of the foot in MR images, we found that the 3D extension of live-wire is more repeatable, with a statistical significance level of P < 0.0001, and 2-6 times faster, with a statistical significance level of P < 0.01, than the 2D live-wire method, and 3-15 times faster than manual tracing.

We have been developing general user steered image segmentation strategies for routine use in applications involving a large number of data sets. In the past, we have presented three segmentation paradigms: live wire, live lane, and a three-dimensional (3-D) extension of the live-wire method. In this paper, we introduce an ultra-fast live-wire method, referred to as live wire on the fly, for further reducing user's time compared to the basic live-wire method. In live wire, 3-D/four-dimensional (4-D) object boundaries are segmented in a slice-by-slice fashion. To segment a two-dimensional (2-D) boundary, the user initially picks a point on the boundary and all possible minimum-cost paths from this point to all other points in the image are computed via Dijkstra's algorithm. Subsequently, a live wire is displayed in real time from the initial point to any subsequent position taken by the cursor. If the cursor is close to the desired boundary, the live wire snaps on to the boundary. The cursor is then deposited and a new live-wire segment is found next. The entire 2-D boundary is specified via a set of live-wire segments in this fashion. A drawback of this method is that the speed of optimal path computation depends on image size. On modestly powered computers, for images of even modest size, some sluggishness appears in user interaction, which reduces the overall segmentation efficiency. In this work, we solve this problem by exploiting some known properties of graphs to avoid unnecessary minimum-cost path computation during segmentation. In live wire on the fly, when the user selects a point on the boundary the live-wire segment is computed and displayed in real time from the selected point to any subsequent position of the cursor in the image, even for large images and even on low-powered computers. Based on 492 tracing experiments from an actual medical application, we demonstrate that live wire on the fly is 1.3-31 times faster than live wire for actual segmentation for varying image sizes, although the pure computational part alone is found to be about 120 times faster.

In multidimensional image analysis, there are, and will continue to be, situations wherein automatic image segmentation methods fail, calling for considerable user assistance in the process. The main goals of segmentation research for such situations ought to be (i) to provideeffective controlto the user on the segmentation processwhileit is being executed, and (ii) to minimize the total user's time required in the process. With these goals in mind, we present in this paper two paradigms, referred to aslive wireandlive lane, for practical image segmentation in large applications. For both approaches, we think of the pixel vertices and oriented edges as forming a graph, assign a set of features to each oriented edge to characterize its ``boundariness,'' and transform feature values to costs. We provide training facilities and automatic optimal feature and transform selection methods so that these assignments can be made with consistent effectiveness in any application. In live wire, the user first selects an initial point on the boundary. For any subsequent point indicated by the cursor, an optimal path from the initial point to the current point is found and displayed in real time. The user thus has a live wire on hand which is moved by moving the cursor. If the cursor goes close to the boundary, the live wire snaps onto the boundary. At this point, if the live wire describes the boundary appropriately, the user deposits the cursor which now becomes the new starting point and the process continues. A few points (live-wire segments) are usually adequate to segment the whole 2D boundary. In live lane, the user selects only the initial point. Subsequent points are selected automatically as the cursor is moved within a lane surrounding the boundary whose width changes as a function of the speed and acceleration of cursor motion. Live-wire segments are generated and displayed in real time between successive points. The users get the feeling that the curve snaps onto the boundary as and while they roughly mark in the vicinity of the boundary. We describe formal evaluation studies to compare the utility of the new methods with that of manual tracing based on speed and repeatability of tracing and on data taken from a large ongoing application. The studies indicate that the new methods are statistically significantly more repeatable and 1.5–2.5 times faster than manual tracing.