Abstract Distance-based supervised method, the minimal learning machine, constructs a predictive model from data by learning a mapping between input and output distance matrices. In this paper, we propose new methods and evaluate how their core component, the distance mapping, can be adapted to multi-label learning. The proposed approach is based on combining the distance mapping with an inverse distance weighting. Although the proposal is one of the simplest methods in the multi-label learning literature, it achieves state-of-the-art performance for small to moderate-sized multi-label learning problems. In addition to its simplicity, the proposed method is fully deterministic: Its hyper-parameter can be selected via ranking loss-based statistic which has a closed form, thus avoiding conventional cross-validation-based hyper-parameter tuning. In addition, due to its simple linear distance mapping-based construction, we demonstrate that the proposed method can assess the uncertainty of the predictions for multi-label classification, which is a valuable capability for data-centric machine learning pipelines.

Deep kernel learning (DKL) marries the uncertainty quantification of Gaussian processes (GPs) and the representational power of deep neural networks. However, training DKL is challenging and often leads to overfitting. Most notably, DKL often learns “non-local” kernels — incurring spurious correlations. To remedy this issue, we propose using amortized inducing points and a parameter-sharing scheme, which ties together the amortization and DKL networks. This design imposes an explicit dependency between the ELBO’s model fit and capacity terms. In turn, this prevents the former from dominating the optimization procedure and incurring the aforementioned spurious correlations. Extensive experiments show that our resulting method, amortized varitional DKL (AVDKL), i) consistently outperforms DKL and standard GPs for tabular data; ii) achieves significantly higher accuracy than DKL in node classification tasks; and iii) leads to substantially better accuracy and negative log-likelihood than DKL on CIFAR100.

Gaussian processes (GPs) can provide a principled approach to uncertainty quantification with easy-to-interpret kernel hyperparameters, such as the lengthscale, which controls the correlation distance of function values.However, selecting an appropriate kernel can be challenging. Deep GPs avoid manual kernel engineering by successively parameterizing kernels with GP layers, allowing them to learn low-dimensional embeddings of the inputs that explain the output data. Following the architecture of deep neural networks, the most common deep GPs warp the input space layer-by-layer but lose all the interpretability of shallow GPs. An alternative construction is to successively parameterize the lengthscale of a kernel, improving the interpretability but ultimately giving away the notion of learning lower-dimensional embeddings. Unfortunately, both methods are susceptible to particular pathologies which may hinder fitting and limit their interpretability. This work proposes a novel synthesis of both previous approaches: {Thin and Deep GP} (TDGP). Each TDGP layer defines locally linear transformations of the original input data maintaining the concept of latent embeddings while also retaining the interpretation of lengthscales of a kernel. Moreover, unlike the prior solutions, TDGP induces non-pathological manifolds that admit learning lower-dimensional representations. We show with theoretical and experimental results that i) TDGP is, unlike previous models, tailored to specifically discover lower-dimensional manifolds in the input data, ii) TDGP behaves well when increasing the number of layers, and iii) TDGP performs well in standard benchmark datasets.

Background: Bug-fixing is the crux of software maintenance. It entails tending to heaps of bug reports using limited resources. Using historical data, we can ask questions that contribute to better-informed allocation heuristics. The caveat here is that often there is not enough data to provide a sound response. This issue is especially prominent for young projects. Also, answers may vary from project to project. Consequently, it is impossible to generalize results without assuming a notion of relatedness between projects.

Multilateration (MLAT) is thede factotechnique to localize points of interest (POIs) in navigation and surveillance systems. Despite sensors being inherently noisy, most existing techniques i) are oblivious to noise patterns in sensor measurements; and ii) only provide point estimates of the POI. This often results in unreliable estimates with high variance,i.e., that are highly sensitive to measurement noise. To overcome this caveat, we advocate the use of Bayesian modeling. Using Bayesian statistics, we provide a comprehensive guide to handle uncertainties in MLAT, including principled choices for the likelihood function and the prior distributions. Notably, the resulting model is easy to implement and can leverage off-the-shelf Markov Chain Monte Carlo (MCMC) software for inference. Besides coping with unreliable measurements, our framework can also deal with sensors whose location is not completely known, which is an asset in mobile systems. Our solution also naturally incorporates multiple measurements per reference point, a common practical situation that is usually not handled directly by other approaches. Comprehensive experiments with both synthetic and real-world data indicate that our Bayesian approach to the MLAT task provides better position estimation and uncertainty quantification when compared to the available alternatives.

Gaussian Process Latent Variable Model (GPLVM) is a flexible framework to handle uncertain inputs in Gaussian Processes (GPs) and incorporate GPs as components of larger graphical models. Nonetheless, the standard GPLVM variational inference approach is tractable only for a narrow family of kernel functions. The most popular implementations of GPLVM circumvent this limitation using quadrature methods, which may become a computational bottleneck even for relatively low dimensions. For instance, the widely employed Gauss-Hermite quadrature has exponential complexity on the number of dimensions. In this work, we propose using the unscented transformation instead. Overall, this method presents comparable, if not better, performance than off-the-shelf solutions to GPLVM, and its computational complexity scales only linearly on dimension. In contrast to Monte Carlo methods, our approach is deterministic and works well with quasi-Newton methods, such as the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm. We illustrate the applicability of our method with experiments on dimensionality reduction and multistep-ahead prediction with uncertainty propagation.

Knowledge transmitted between generations by non-genetic means can be understood as culture. The capacity of individuals from certain species to teach and learn plays a fundamental role in directing the evolutionary process. The Neuroevolution of Augmenting Topologies (NEAT) framework enables evolving neural structures to iteratively solve a given learning problem. However, the NEAT approach does not consider cultural aspects in its formulation. In such a context, the aim of this paper is to propose and evaluate ways of enhancing the NEAT framework with additional learning approaches. The parameters involved in the analysis comprise the Backpropagation and the Extreme Learning Machine (ELM) learning algorithms, the individuals to be taught, the moment when culture manifests in the system, and the nature of the lessons to be learned. Empirical results on sequential learning tasks indicate that cultural enhancements, as well as some of the proposed variations, accelerate the neuroevolution convergence.

The electrocardiogram (ECG) is a non-invasive method to detect cardiovascular diseases (CVD), the most common cause of death in the world. The recognition of heartbeat morphologies present in the ECG signal is an effective way to detect CVDs prematurely. Many approaches were developed for this purpose, such as the use of Wavelets, High Order Statistics (HOS), Local Binary Patterns (LBP), Random Projection, Fiducial points, and Hermite Polynomials. Unfortunately, most parts of these approaches suffer from the high variability of ECG signal features and conditions. Also, it is common to use more than one of them simultaneously, which makes it hard to infer the contributions of each one. This work presents a new robust methodology to extract features for heartbeat morphology classification. Moreover, we introduce new labels for a small set of morphologies present in MIT-BIH Arrhythmia database, taking into account only the QRS complex (the 3 more representative waves of a heartbeat) instead of the whole heartbeat. We evaluate each approach in isolation and the results show that our method outperforms other well-known strategies.

The task of detecting anomalous data patterns is as important in practical applications as challenging. In the context of spatial data, recognition of unexpected trajectories brings additional difficulties, such as high dimensionality and varying pattern lengths. We aim to tackle such a problem from a probability density estimation point of view, since it provides an unsupervised procedure to identify out of distribution samples. More specifically, we pursue an approach based on normalizing flows, a recent framework that enables complex density estimation from data with neural networks. Our proposal computes exact model likelihood values, an important feature of normalizing flows, for each segment of the trajectory. Then, we aggregate the segments' likelihoods into a single coherent trajectory anomaly score. Such a strategy enables handling possibly large sequences with different lengths. We evaluate our methodology, named aggregated anomaly detection with normalizing flows (GRADINGS), using real world trajectory data and compare it with more traditional anomaly detection techniques. The promising results obtained in the performed computational experiments indicate the feasibility of the GRADINGS, specially the variant that considers autoregressive normalizing flows.

Huntington's disease is a hereditary condition in which brain nerve cells rupture over time. This work proposes a new method for the detection of Huntington's disease using digitised voice signals by diseased and healthy volunteers while they were reading Lithuanian poems. In this approach, the produced features by voice signals suffer a dimensionality reduction to optimize the prediction stage. The performance evaluation regarded 186 speech exams and 24 volunteers, combining twelve audio signal feature extractors with classification models. The results indicate an excellent performance, reaching precision and accuracy over 99 percent with prediction time below 1 second. This approach shows promising results indicating its usability to improve the medical diagnosis via computer-aided diagnosis.

Bayesian Optimization (BO) is a framework for black-box optimization that is especially suitable for expensive cost functions. Among the main parts of a BO algorithm, the acquisition function is of fundamental importance, since it guides the optimization algorithm by translating the uncertainty of the regression model in a utility measure for each point to be evaluated. Considering such aspect, selection and design of acquisition functions are one of the most popular research topics in BO. Since no single acquisition function was proved to have better performance in all tasks, a well-established approach consists of selecting different acquisition functions along the iterations of a BO execution. In such an approach, the GP-Hedge algorithm is a widely used option given its simplicity and good performance. Despite its success in various applications, GP-Hedge shows an undesirable characteristic of accounting on all past performance measures of each acquisition function to select the next function to be used. In this case, good or bad values obtained in an initial iteration may impact the choice of the acquisition function for the rest of the algorithm. This fact may induce a dominant behavior of an acquisition function and impact the final performance of the method. Aiming to overcome such limitation, in this work we propose a variant of GP-Hedge, named No-PASt-BO, that reduce the influence of far past evaluations. Moreover, our method presents a built-in normalization that avoids the functions in the portfolio to have similar probabilities, thus improving the exploration. The obtained results on both synthetic and real-world optimization tasks indicate that No-PASt-BO presents competitive performance and always outperforms GP-Hedge.