Abstract Several convolutional neural network architectures have been proposed for handwritten character recognition. However, most of the conventional architectures demand large scale training data and long training time to obtain satisfactory results. These requirements prevent the use of these methods in a broader range of applications. As an alternative to cope with these problems, we present a new convolutional network for handwritten character recognition based on the Fukunaga–Koontz transform (FKT). Our approach lies in the assumption that Fukunaga–Koontz convolutional kernels can be efficiently learned from subspaces and directly employed to produce high discriminant features in a shallow network architecture. When representing image classes by subspaces, the within-class separability is reduced, since the subspaces form clusters in a low-dimensional space. To increase the between-class separability, we compute a discriminative space from the training subspaces using FKT. By learning convolutional kernels from subspaces, it is possible to extract representative and discriminative features from an image with only a few parameters. Another contribution of the proposed network is the use of pooling layers, which further improves its performance. The proposed method, called Fukunaga–Koontz Network (FKNet), is suitable for solving practical problems, especially when training and processing times are constraints. Four publicly available handwritten character datasets are employed to evaluate the advantages of FKNet. In addition, we demonstrate the flexibility of the proposed method by experiments on LFW dataset.

We propose an online method for concept drift detection based on dynamic classifier ensemble selection. The proposed method generates a pool of ensembles by promoting diversity among classifier members and chooses expert ensembles according to global prequential accuracy values. Unlike current dynamic ensemble selection approaches that use only local knowledge to select the most competent ensemble for each instance, our method focuses on selection taking into account the decision space. Consequently, it is well adapted to the context of drift detection in data stream problems. The results of the experiments show that the proposed method attained the highest detection precision and the lowest number of false alarms, besides competitive classification accuracy rates, in artificial datasets representing different types of drifts. Moreover, it outperformed baselines in different real-problem datasets in terms of classification accuracy.

Several real-world prediction problems are subject to changes over time due to their dynamic nature. These changes, named concept drift, usually lead to immediate and disastrous loss in classifier's performance. In order to cope with such a serious problem, drift detection methods have been proposed in the literature. However, current methods cannot be widely used since they are based either on performance monitoring or on fully labeled data, or even both. Focusing on overcoming these drawbacks, in this work we propose using density variation of the most significant instances as an explicit unsupervised trigger for concept drift detection. Here, density variation is based on Active Learning, and it is calculated from virtual margins projected onto the input space according to classifier confidence. In order to investigate the performance of the proposed method, we have carried out experiments on six databases, precisely four synthetic and two real databases focusing on setting up all parameters involved in our method and on comparing it to three baselines, including two supervised drift detectors and one Active Learning-based strategy. The obtained results show that our method, when compared to the supervised baselines, reached better recognition rates in the majority of the investigated databases, while keeping similar or higher detection rates. In terms of the Active Learning-based strategies comparison, our method outperformed the baseline taking into account both recognition and detection rates, even though the baseline employed much less labeled samples. Therefore, the proposed method established a better trade-off between amount of labeled samples and detection capability, as well as recognition rate.

Indoor positioning systems consist of identifying the physical location of devices inside buildings. They are usually based on the signal strength of a device packet received by a set of WiFi access points. Among the most precise solutions, are those based on machine learning algorithms, such as kNN (k-Nearest Neighbors). This technique is known as fingerprint positioning. Even though kNN is one of the most used classification methods due to its high precision results, it lacks scalability since an instance we need to classify must be compared to all other instances in the training base. In this work, we use a novel hierarchical navigable small world graph technique to fit the training database so that the samples can be efficiently classified in the online phase of the fingerprint positioning, allowing it to be used in large-scale scenarios and/or to be executed in resource-limited devices. We evaluated the performance of this solution using both synthetic and real-world training data and compared its performance to other known kNN variants such as kd-tree and ball-tree. Our results clearly show the performance gains of the graph-based solution, while still being able to maintain or even reduce the positioning error.

In recent years, several convolutional neural networks-based architectures (CNN) have been proposed for handwritten character recognition. However, most of the conventional proposed models demand large scale training data and long training time to compute the parameters and achieve satisfactory results. These requirements prevent the use of these methods in a wider range of applications. To solve these problems, we present a novel convolutional network for handwritten character recognition based on subspace method. Our approach lies on the assumption that convolutional kernels can be efficiently learned from subspaces and directly employed to produce high discriminant features in a CNN architecture. When representing each image class by subspaces, we decrease the inter-class similarity, since the subspaces form clusters in a low dimensional space. In order to enlarge the intra-class similarity, we estimate a discriminative space from the training subspaces. By learning convolutional kernels from subspaces we can obtain representative and discriminative information produced by the network with few parameters, creating a light weight network. This use of a flexible architecture and its straightforward implementation make the proposed method quite attractive in practical terms. Our experimental evaluation shows that we achieve competitive results compared to the state-of-the-art methods.

Recently, convolutional neural network has attracted an increasing amount of attention in machine learning and computer vision areas, improving the performance of several related applications. Currently, many deep learning network architectures such as principal component analysis network (PCANet), linear discriminant analysis network (LDANet) and canonical correlation analysis network (CCANet) have been proposed for object and face classification. These architecture solutions have demonstrated high efficiency, with a simple implementation, providing a fast prototyping of efficient image classification applications. However, these solutions take advantage of filters that may not extract high discriminative features in more complicated computer vision problems (e.g. containing high degree of overlap between the distributions of the data). To generate more discriminative information, we introduce a discriminative canonical correlation network (DCCNet), that employs filters constructed from discriminative canonical correlations analysis (DCC). Learning filters from DCC ensures that the network will produce discriminative features, generating more representative and discriminative information. We demonstrate the applicability of DCCNet through experiments on four datasets.

Object recognition on large-scale video has recently attracted considerable research interest due to the huge amount of data available on the Internet, surveillance systems, social media networks and autonomous vehicles. By representing large-scale videos as image sets, we can handle the complex data variations such as viewpoint, illumination, and pose. In this paper, we propose an efficient and robust method for image set recognition based on Kernel Orthogonal Mutual Subspace Method (KOMSM), where sets of images are expressed as nonlinear subspaces. In our method, we formulate the image sets as nonlinear 2D subspaces by applying K2D-PCA and variants of 2D-PCA. Comparing to KOMSM, the proposed method requires less memory resource since it inherits the computational advantages of 2D-PCA and variants. In addition, the subspaces produced by K2D-PCA preserves the spatial relation between image pixels, generating more informative subspaces than KOMSM. The introduced method has the advantage of representing the subspaces in a more compact manner, achieving lower time complexity, confirming the suitability of employing 2D-PCA and variants. These results have been revealed through comprehensive experimentation conducted on five publicly available datasets.

The Amazon Rainforest degradation is a worldwide concern. The rainforest has been endangered by the illegal wood extraction without control even in the preservation areas. Due to the large geography extension prevent these crimes with an unmanned aerial vehicle (UAV) is not always possible. The Wireless Acoustics Sensor Network (WASNs) technology can alleviate this problem. Here, we present an acoustical framework to detect the sounds produced by several chainsaws. Our framework was developed to be embedded in a sensor node, combining the Mel-Fourier Cepstral Coefficients (MFCCs) with One-class classification technique. This classification method, that is based on a kernel density approach, allows us to recognize only chainsaw sounds rejecting all the other possible environmental sounds, such as: animal's calls, weather noises or boat engines. In the experiments, we varied the number MFCCs coefficients and the Kernel bandwidth performing a leave-one-out cross validation to find the best combination. Finally, we found that the best parameter combination achieve 98% of accuracy showing a low FNR and a high TPR, fact that enhances the credibility of the system avoiding false alarms and making it an optimal choice for an WSN application.

In real-world applications, machine learning algorithms can be employed in spam detection, environmental monitoring, fraud detection, medical diagnoses, among others. Most of these problems present an environment which changes over time. The problem involving classification tasks in dynamic environments has become one of the major challenges in machine learning domain in the last decades. Currently in the literature, methods based on accuracy monitoring are commonly used to detect changes explicitly. However, these methods may become infeasible in some real-world applications especially due to two aspects: they may need human operator feedback, and may depend on a significant decrease of accuracy to be able to detect changes. In addition, most of these methods are also incremental learning-based, since they update the decision model for every incoming example. However, this may lead to system unnecessary updates. In order to overcome these problems, this paper proposes a method to detect changes based on data distribution dissimilarity and to update the decision model only after drift detection. The achievement of this method is relevant to allow change detection and reaction be applicable in several practical problems. The experiments conducted indicate that the proposed method attains performance and detection rates similar to the traditional DDM and EDDM, which are incremental learning-based and performance monitoring-based drift detectors.

The schema matching problem consists of finding semantic correspondences between elements (e.g., attributes) of two database schemas. Typically, methods to solve this problem first use pair-wise functions called matchers to generate similarity scores values between pairs of elements from the two schemas. These scores are used as estimations of the correspondence between elements. Next, matchers are combined to establish which pairs of elements must be mapped when integrating the two schemas. For this, the best-known schema matching methods rely on fixed heuristics. We consider that using fixed heuristics is not always helpful to cope with a variety of database and element mismatch cases and argue that machine learning (ML) techniques comprise suitable alternatives to properly combine matchers. In this paper we propose ALMa (Active Learning Matching), a novel method for combining matchers based on active learning, which is an interesting and effective machine learning technique that efficiently exploits the users' expertise on the matching task. We report the results of experiments we carried out comparing ALMa with COMA, a well-known schema matching method in the literature based on fixed heuristics, and with YAM, a recent schema matching method based on supervised learning. In the experiments, ALMa achieved results better or at least similar to the baselines, while demanded less user effort, confirming the suitability of using active learning for combining matchers.

Machine Learning solutions for concept drift detection problems try to decide to what extent a particular set of examples still represents the current concept rather than treating all data equally. Monitoring the set of relevant features used to generate the classification model may be an effective strategy for concept drift detection. This paper focuses on analyzing the possibility of detecting drifts through feature evolution monitoring in the spam detection problem. Results of the experiments show that the relevant features of the target domain are significantly different from the relevant features of the source domain. This offers a new possibility for analyzing the relationship between feature evolution and misclassification rate. The experiments were conducted using two databases: a public database composed of samples collected between 2003 and 2004; and a new private database composed of samples collected between 2012 and 2013.

The analysis of well logs is important on reservoir characterization. One goal of this analysis is to classify lithofacies in order to estimate the reserve of petroleum. This analysis is traditionally conducted as a semi-automated process, where graphs of curves are used to allow human experts to perform the classification task. On the other hand, this problem has been dealt with in the literature as an automatic pattern recognition task. In this context, several aspects have been investigated such as, comparison among classification methods, reservoirs with heterogeneity and data from different wells. This paper addresses the aspect of using data obtained from one well to train an ensemble of heterogeneous classifiers in order to combine their decision to assign labels to data extracted from other wells. All investigated wells are from the same reservoir. In addition, we show that the classifier ensemble does not outperform its members when the training and the test sets are composed of samples obtained from the same well. This comparison indicates that once a human expert has manually classified lithofacies from one well, this information may be used to train a classifier ensemble, which will be able to use this knowledge to achieve high accuracy on classifying samples from wells within the neighboring region, at the same reservoir.

Anurans (frogs or toads) are commonly used by biologists as early indicators of ecological stress. The reason is that anurans are closely related to the ecosystem. Although several sources of data may be used for monitoring these animals, anuran calls lead to a non-intrusive data acquisition strategy. Moreover, wireless sensor networks (WSNs) may be used for such a task, resulting in more accurate and autonomous system. However, it is essential save resources to extend the network lifetime. In this paper, we evaluate the impact of reducing data dimension for automatic classification of bioacoustic signals when a WSN is involved. Such a reduction is achieved through a wrapper-based feature subset selection strategy that uses genetic algorithm (GA). We use GA to find the subset of features that maximizes the cost-benefit ratio. In addition, we evaluate the impact of reducing the original feature space, when sampling frequencies are also reduced. Experimental results indicate that we can reduce the number of features, while increasing classification rates (even when smaller sampling frequencies of transmission are used).

The structure of dynamic websites comprised of a set of objects such as HTML tags, script functions, hyperlinks and advanced features in browsers lead to numerous resources and interactiveness in services currently provided on the Internet. However, these features have also increased security risks and attacks since they allow malicious codes injection or XSS (Cross-Site Scripting). XSS remains at the top of the lists of the greatest threats to web applications in recent years. This paper presents the experimental results obtained on XSS automatic classification in web pages using Machine Learning techniques. We focus on features extracted from web document content and URL. Our results demonstrate that the proposed features lead to highly accurate classification of malicious page.

Dynamic classifier ensemble selection is focused on selecting the most confident classifier ensemble to predict the class of a particular test pattern. The overproduce-and-choose strategy is a dynamic classifier ensemble selection method which is divided into optimization and dynamic selection phases. The first phase involves the test of different candidate ensembles in order to produce a population composed of the highest performing candidate ensembles. Then, the second phase calculates the domain of expertise of each candidate ensemble to pick up the solution with highest degree of certainty of its decision to classify the unknown test samples. It has been shown that the optimization phase decreases oracle, the upper bound of dynamic selection processes. In this paper we propose a hybrid algorithm to perform the optimization phase of overproduce and-choose strategy. The proposed algorithm combines stochastic initialization of candidate ensembles of different sizes, with the traditional forward search greedy method. The objective is to apply oracle as search criterion during the optimization phase. We show experimentally that choosing the population of classifier ensembles taking into account the population oracle leads to increase the upper bound of the dynamic selection phase. Moreover, experimental results conducted to compare the proposed method to a multi-objective genetic algorithm (MOGA), demonstrate that our method outperforms MOGA on generating population of candidate ensembles with higher oracle rates.

Classifier ensemble selection may be formulated as a learning task since the search algorithm operates by minimizing/maximizing the objective function. As a consequence, the selection process may be prone to overfitting. The objectives of this paper are: (1) to show how overfitting can be detected when the selection is performed by two classical search algorithms: Genetic Algorithm and Particle Swarm Optimization; and (2) to verify which algorithm is more prone to overfitting. The experimental results demonstrate that GA appears to be more affected by overfitting.

The overproduce-and-choose strategy involves the generation of an initial large pool of candidate classifiers and it is intended to test different candidate ensembles in order to select the best performing solution. The ensemble's error rate, ensemble size and diversity measures are the most frequent search criteria employed to guide this selection. By applying the error rate, we may accomplish the main objective in Pattern Recognition and Machine Learning, which is to find high-performance predictors. In terms of ensemble size, the hope is to increase the recognition rate while minimizing the number of classifiers in order to meet both the performance and low ensemble size requirements. Finally, ensembles can be more accurate than individual classifiers only when classifier members present diversity among themselves. In this paper we apply two Pareto front spread quality measures to analyze the relationship between the three main search criteria used in the overproduce-and-choose strategy. Experimental results conducted demonstrate that the combination of ensemble size and diversity does not produce conflicting multi-objective optimization problems. Moreover, we cannot decrease the generalization error rate by combining this pair of search criteria. However, when the error rate is combined with diversity or the ensemble size, we found that these measures are conflicting objective functions and that the performances of the solutions are much higher.

Dynamic classifier selection has traditionally focused on selecting the most accurate classifier to predict the class of a particular test pattern. In this paper we propose a new dynamic selection method to select, from a population of ensembles, the most confident ensemble of classifiers to label the test sample. Such a level of confidence is measured by calculating the ambiguity of the ensemble on each test sample. We show theoretically and experimentally that choosing the ensemble of classifiers, from a population of high accurate ensembles, with lowest ambiguity among its members leads to increase the level of confidence of classification, consequently, increasing the generalization performance. Experimental results conducted to compare the proposed method to static selection and DCS-LA, demonstrate that our method outperforms both DCS-LA and static selection strategies when a population of high accurate ensembles is available.

Many recent works have investigated methods to select subsets of classifiers instead of combining all available classifiers. The majority of these works has concluded that the combiner error rate is better than diversity to guide the selection process in order to identify the best performing subset of classifiers. However, the classifier selection process has to take into account three different aspects: complexity, overfitting and performance. These aspects of the selection process have not yet been tackled simultaneously in the literature. The study presented in this paper, deals with these three aspects in a handwritten digit recognition problem. Different search criteria such as diversity, error rate and number of classifiers are applied in single and multi-objective optimization approaches using genetic algorithms. In our experiments, we observed that error rate applied in a single optimization approach was the best objective function to increase performance. The generalized diversity and interrater agreement measures, combined with error rate in pairs of objective functions were the best measures to reduce complexity and keep good performance in a multi-objective optimization approach. Finally, the performance of the solutions found in both, single and multi-objective optimization processes were increased by applying a global validation method to reduce overfitting.