Most machine learning methods assume fixed probability distributions, limiting their applicability in nonstationary real-world scenarios. While continual learning methods address this issue, current approaches often rely on closed-box models or require extensive user intervention for interpretability. We propose SyMPLER (Systems Modeling through Piecewise Linear Evolving Regression), an explainable model for time series forecasting in nonstationary environments based on dynamic piecewise-linear approximations. Unlike other locally linear models, SyMPLER uses generalization bounds from Statistical Learning Theory to automatically determine when to add new local models based on prediction errors, eliminating the need for explicit clustering of the data. Experiments show that SyMPLER can achieve comparable performance to both closed-box and existing explainable models while maintaining a human-interpretable structure that reveals insights about the system's behavior. In this sense, our approach conciliates accuracy and interpretability, offering a transparent and adaptive solution for forecasting nonstationary time series.

While large margin classifiers are originally an outcome of an optimization framework, support vectors (SVs) can be obtained from geometric approaches. This article presents advances in the use of Gabriel graphs (GGs) in binary and multiclass classification problems. For Chipclass, a hyperparameterless and optimization-less GG-based binary classifier, we discuss how activation functions and support edge (SE)-centered neurons affect the classification, proposing smoother functions and structural SV (SSV)-centered neurons to achieve margins with low probabilities and smoother classification contours. We extend the neural network architecture, which can be trained with backpropagation with a softmax function and a cross-entropy loss, or by solving a system of linear equations. A new subgraph-/distance-based membership function for graph regularization is also proposed, along with a new GG recomputation algorithm that is less computationally expensive than the standard approach. Experimental results with the Friedman test show that our method was better than previous GG-based classifiers and statistically equivalent to tree-based models.

In order to solve non-linear problems, kernel-based classifiers rely on implicit mappings to very high-dimensional spaces.These target spaces, although mathematically robust, often lack the property of visual interpretation, limiting the intuition of the problem at hand.In this work, the notion of a similarity space is presented, to which one can map input samples and visualize how they interact under a given kernel function.By exploring statistics in such space, a class separability measure is derived, which can be used to find optimal kernel parameters for binary classification.Experiments using support vector machines were conducted, showing the method's effectiveness when compared to grid-search approaches.

Although it requires simple computations, provides good performance on linear classification tasks and offers a suitable environment for active learning strategies, the Hebbian learning rule is very sensitive to how the training data relate to each other in the input space. Since this spatial arrangement is inherent to each set of samples, the practical application of this learning paradigm is limited. Thus, representation learning may play an important role in projecting the input data into a new space where linear separability is improved. Earlier methods based on orthogonal coding addressed this issue but presented many side effects, impoverishing the generalization of the model. Hence, this paper considers a recently proposed method based on kernel density estimators, which performs a likelihood-based projection where linear separability and generalization capacity are enhanced in an autonomous fashion. Results show that this novel method allows one to use linear classifiers to solve many binary classification problems and overcome the performance of well-established classifiers.

The number of connected embedded edge computing Internet of Things (IoT) devices has been increasing over the years, contributing to the significant growth of available data in different scenarios. Thereby, machine learning algorithms arise to enable task automation and process optimization based on those data. However, due to some learning methods' computational complexity implementing geometric classifiers, it is a challenge to map these on embedded systems or devices with limited resources in size, processing, memory, and power, to accomplish the desired requirements. This hampers the applicability of these methods to complex industrial embedded edge applications. This work evaluates strategies to reduce classifiers' implementation costs based on the CHIP-clas model, independent of hyperparameter tuning and optimization algorithms. The proposal aims to evaluate the tradeoff between numerical precision and model performance and analyze the hardware implementations of a distance-based classifier. Two 16 -b floating-point formats were compared to the 32 -b floating-point precision implementation. Also, a new hardware architecture was developed and then compared to the state-of-the-art reference. The results indicate that the model is robust to low precision computation, providing statistically equivalent results compared to the baseline model, also pointing out statistically equivalent performance and a global speed-up factor of approx 4.39 in processing time.

It is well known that there is an increasing interest in edge computing to reduce the distance between cloud and end devices, especially for machine learning (ML) methods. However, when related to latency-sensitive applications, little work can be found in ML literature on suitable embedded systems implementations. This article presents new ways to implement the decision rule of a large margin classifier based on Gabriel graphs as well as an efficient implementation of this on an embedded system. The proposed approach uses the nearest neighbor method as the decision rule, and the implementation starts from an RTL pipeline architecture developed for binary large margin classifiers and proposes the integration in a hardware/software co-design. Results showed that the proposed approach was statistically similar to the classifier and had a speedup factor of up to eight times compared to the classifier executed in software, with performance suitable for ML latency-sensitive applications.

Deep neural networks (NNs) have been proved to be efficient learning systems for supervised and unsupervised tasks. However, learning complex data representations using deep NNs can be difficult due to problems such as lack of data, exploding or vanishing gradients, high computational cost, or incorrect parameter initialization, among others. Deep representation and transfer learning (RTL) can facilitate the learning of data representations by taking advantage of transferable features learned by an NN model in a source domain, and adapting the model to a new domain.

This brief presents a geometrical approach for obtaining large margin classifiers. The method aims at exploring the geometrical properties of the data set from the structure of a Gabriel graph, which represents pattern relations according to a given distance metric, such as the Euclidean distance. Once the graph is generated, geometrical support vectors (SVs) (analogous to support vector machines (SVMs) SVs) are obtained in order to yield the final large margin solution from a Gaussian mixture model. Experiments with 20 data sets have shown that the solutions obtained with the proposed method are statistically equivalent to those obtained with SVMs. However, the present method does not require optimization and can also be extended to large data sets using the cascade SVM concept.

A bottleneck of laboratory analysis in process industries including steelmaking plants is the low sampling rate. Inference models using only variables measured online have then been used to made such information available in advance. This study develops predictive models for key mechanical properties of seamless steel tubes, by strength, ultimate tensile strength and hardness. A plant in Brazil was used as the case study. The sample sizes of some steel tube families given namely, yield a particular property are discrepant and sometimes very small. To overcome this sample imbalance and lack of representativeness, committees of predictive neural network models based on bagging predictors, a type of ensemble method, were adopted. As a result, all steel families for all properties have been satisfactorily described showing the correlations between targets and model estimates close to 99%. These results were compared to multiple linear regression, support vector machine and a simpler neural network. Such information available in advance favors corrective actions before complete tube production mitigating rework costs in general.

This article presents a novel representation of artificial neural networks (ANNs) that is based on a projection of weights into a new spherical space defined by a radius r and a vector of angles Θ . This spherical representation of ANNs further simplifies the multiobjective learning problem, which is usually treated as a constrained optimization problem that requires great computational effort to maintain the constraints. With the proposed spherical representation, the constrained optimization problem becomes unconstrained, which simplifies the formulation and computational effort required. In addition, it also allows the use of any nonlinear optimization method for the multiobjective learning of ANNs. Results presented in this article show that the proposed spherical representation of weights yields more accurate estimates of the Pareto set than the classical multiobjective approach. Regarding the final solution selected from the Pareto set, our approach was effective and outperformed some state-of-the-art methods on several data sets.

The following work presents a new approach to automatic selection of Tikhonov's regularization parameter, responsible for controlling the weight value of an ELM neural network. A strategy based on measurements obtained from data projection (Fisher-Score) is introduced. Seven datasets are tested and results are compared to those obtained when the regularization parameter is selected through cross-validation. The strategy shows satisfactory classification performance (in terms of p-value), while presenting significant training time reduction.

It is not uncommon for data obtained from systems to change after the model is learned. These occurrences are named dataset shifts and to deal with them models with the ability to adapt to data changes must be used. A strategy that can be easily integrated to other classifiers is proposed. It creates a geometrical representation of data that extracts information from both labelled and unlabelled data. Then data entropy and Jensen-Shannon dissimilarity tests are used during the model selection to handle cases where data shift. Results have shown that the proposed method is promising because of its simple integration with state of art classifiers and its performance in enhancing said classifiers accuracy in the studied cases.

This paper presents a geometrical approach for obtaining large margin classifiers. The method aims at exploring the geometrical properties of the dataset from the structure of a Gabriel graph, which represents pattern relations according to a given distance metric, such as the Euclidean distance. Once the graph is generated, geometric vectors, analogous to SVM's support vectors are obtained in order to yield the final large margin solution from a mixture model approach. A preliminary experimental study with five real-world benchmarks showed that the method is promising.

Traditional learning algorithms applied to complex and highly imbalanced training sets may not give satisfactory results when distinguishing between examples of the classes. The tendency is to yield classification models that are biased towards the overrepresented (majority) class. This paper investigates this class imbalance problem in the context of multilayer perceptron (MLP) neural networks. The consequences of the equal cost (loss) assumption on imbalanced data are formally discussed from a statistical learning theory point of view. A new cost-sensitive algorithm (CSMLP) is presented to improve the discrimination ability of (two-class) MLPs. The CSMLP formulation is based on a joint objective function that uses a single cost parameter to distinguish the importance of class errors. The learning rule extends the Levenberg-Marquadt's rule, ensuring the computational efficiency of the algorithm. In addition, it is theoretically demonstrated that the incorporation of prior information via the cost parameter may lead to balanced decision boundaries in the feature space. Based on the statistical analysis of results on real data, our approach shows a significant improvement of the area under the receiver operating characteristic curve and G-mean measures of regular MLPs.

A large number of unclassified sequences is still found in public databases, which suggests that there is still need for new investigations in the area. In this contribution, we present a methodology based on Artificial Neural Networks for protein functional classification. A new protein coding scheme, called here Extended-Sequence Coding by Sliding Windows, is presented with the goal of overcoming some of the difficulties of the well method Sequence Coding by Sliding Window. The new protein coding scheme uses more than one sliding window length with a weight factor that is proportional to the window length, avoiding the ambiguity problem without ignoring the identity of small subsequences Accuracy for Sequence Coding by Sliding Windows ranged from 60.1 to 77.7 percent for the first bacterium protein set and from 61.9 to 76.7 percent for the second one, whereas the accuracy for the proposed Extended-Sequence Coding by Sliding Windows scheme ranged from 70.7 to 97.1 percent for the first bacterium protein set and from 61.1 to 93.3 percent for the second one. Additionally, protein sequences classified inconsistently by the Artificial Neural Networks were analyzed by CD-Search revealing that there are some disagreement in public repositories, calling the attention for the relevant issue of error propagation in annotated databases due the incorrect transferred annotations.

This paper presents a bi-objective algorithm that optimizes the error and the sum of the absolute weights of a Multi-Layer Perceptron neural network. The algorithm is based on the linear Least Absolute Shrinkage and Selection Operator (LASSO) and provides simultaneous generalization and weight selection optimization. The algorithm searches for a set of optimal solutions called Pareto set from which a single weight vector with best performance and reduced number of weights is selected based on a validation criterion. The method is applied to classification and regression real problems and compared with the norm based multi-objective algorithm. Results show that the neural networks obtained have improved generalization performance and reduced topology.

We present in this paper a hybrid strategy for training least squares support vector machines (LS-SVMs), in order to eliminate their greatest drawback when comparing to original support vector machines (SVMs), the inexistence of support vectors' automatic detection, the so called loss of sparseness. The main characteristic of LS-SVMs is the low computational complexity comparing to SVMs, without quality loss in the solution, because the principles that both have been based are the same. In this paper, we use a sample selection technique called reduced remaining subset (RRS), which is based on a modified nearest neighbor rule, in order to choose the best samples to represent each class. After that, LS-SVMs use the selected samples as support vectors to find the decision surface between the classes. Some experiments are presented to compare the proposed approach with two existent methods that also aim to impose sparseness in LS-SVMs.

A new method for design of digital classification circuits is presented in this paper in order to implement them in hardware (FPGA, PAL, VLSI, ASIC, etc). The method works by first selecting a subset of the training data that is just off the separation margin between the classes. The subset is provided to a Boolean minimization algorithm that, by hypercube expansion, designs a classifier with a smoother separation surface between classes. The obtained circuits have performance comparable to support vector machines and multilayer perceptron trained with a generalization control algorithm.

This paper presents an evolutionary approach to the training of transductive support vector machines (TSVMs). A genetic algorithm (GA) is used to search for the best labeling of the test set, providing increased convergence performance and more globally optimized solutions. The stochastic nature of GAs makes this approach more likely to reach global minima than the standard transductive SVMs. A gene-dependent mutation operator, motivated by the k-nearest neighbor algorithm, is introduced, accelerating the convergence significantly.

This paper presents a new scheme for training MLPs which employs a relaxation method for multi-objective optimization. The algorithm works by obtaining a reduced set of solutions, from which the one with the best generalization is selected. This approach allows balancing between the training error and norm of network weight vectors, which are the two objective functions of the multi-objective optimization problem. The method is applied to classification and regression problems and compared with Weight Decay (WD), Support Vector Machines (SVMs) and standard Backpropagation (BP). It is shown that the systematic procedure for training proposed results on good generalization neural models, and outperforms traditional methods.

This paper shows two different methodologies, both based on sliding mode control to train multilayer perceptron. These two methods are compared with standard back propagation, momentum and RPROP algorithms. The results show that the use of this control theory can reduce the time to train multilayer perceptron and also provide an interesting tool to analyze the limits for the parameters involved in the algorithm.

The use of a linguistic representation for expressing knowledge acquired by learning systems is an important issue as regards to user understanding. Under this assumption, and to make sure that these systems will be welcome and used, several techniques have been developed by the artificial intelligence community, under both the symbolic and the connectionist approaches. This work discusses and investigates three knowledge extraction techniques based on these approaches. The first two techniques, the C4.5 and CN2 symbolic learning algorithms, extract knowledge directly from the data set. The last technique, the TREPAN algorithm extracts knowledge from a previously trained neural network. The CN2 algorithm induces if...then rules from a given data set. The C4.5 algorithm extracts decision trees, although it can also extract ordered rules, from the data set. Decision trees are also the knowledge representation used by the TREPAN algorithm.

Based on the classical backpropagation weight update equations, sliding mode control theory is introduced as a technique to adapt weights of a multi-layer perceptron. As will be demonstrated, the introduction of sliding mode has resulted in a much faster version of the standard backpropagation. The results show also that the proposed algorithm presents some important features of sliding mode control, which are robustness and high speed of learning. In addition to that, this paper shows also how control theory can be applied to train neural networks.