Abstract Gaussian Process regression is a kernel method successfully adopted in many real-life applications. Recently, there is a growing interest on extending this method to non-Euclidean input spaces, like the one considered in this paper, consisting of probability measures. Although a Positive Definite kernel can be defined by using a suitable distance—the Wasserstein distance— the common procedure for learning the Gaussian Process model can fail due to numerical issues, arising earlier and more frequently than in the case of an Euclidean input space and, as demonstrated, impossible to avoid by adding artificial noise ( nugget effect ) as usually done. This paper uncovers the main reason of these issues, that is a non-stationarity relation between the Wasserstein-based squared exponential kernel and its Euclidean counterpart. As a relevant result, we learn a Gaussian Process model by assuming the input space as Euclidean and then use an algebraic transformation, based on the uncovered relation, to transform it into a non-stationary and Wasserstein-based Gaussian Process model over probability measures. This algebraic transformation is simpler than log-exp maps used on data belonging to Riemannian manifolds and recently extended to consider the pseudo-Riemannian structure of an input space equipped with the Wasserstein distance.

Abstract It has been recently remarked that focusing only on accuracy in searching for optimal Machine Learning models amplifies biases contained in the data, leading to unfair predictions and decision supports. Recently, multi-objective hyperparameter optimization has been proposed to search for Machine Learning models which offer equally Pareto-efficient trade-offs between accuracy and fairness. Although these approaches proved to be more versatile than fairness-aware Machine Learning algorithms—which instead optimize accuracy constrained to some threshold on fairness—their carbon footprint could be dramatic, due to the large amount of energy required in the case of large datasets. We propose an approach named FanG-HPO: fair and green hyperparameter optimization (HPO), based on both multi-objective and multiple information source Bayesian optimization. FanG-HPO uses subsets of the large dataset to obtain cheap approximations (aka information sources) of both accuracy and fairness, and multi-objective Bayesian optimization to efficiently identify Pareto-efficient (accurate and fair) Machine Learning models. Experiments consider four benchmark (fairness) datasets and four Machine Learning algorithms, and provide an assessment of FanG-HPO against both fairness-aware Machine Learning approaches and two state-of-the-art Bayesian optimization tools addressing multi-objective and energy-aware optimization.

In this paper, we investigate the impact of optimal hyper-parameter configuration in relational topic models. The main goal is to validate the hypothesis that single-objective Bayesian Optimization (BO) can discover a hyper-parameter setting that leads a set of relational topic models to simultaneously ensure good prediction capabilities and significant topics from a qualitative perspective. Our research, as a result of a comparative analysis performed on 7 state-of-the-art models, 5 performance measures and 3 datasets, has highlighted three main findings: (1) the majority of relational topic models are not able to offer a good trade-off between classification capabilities and topic interpretability; (2) single-objective optimization of hyper-parameters, targeted on maximizing the F1-Measure, is able to create topics that are also optimal with respect to the Kullback Leibler divergence measure; (3) the Pareto frontiers across several performance metrics reveals that the most promising trade-off between the performance metrics can be obtained by Constrained Relational Topic Models.

Abstract This paper presents a sequential model based optimization framework for optimizing a black-box, multi-extremal and expensive objective function, which is also partially defined, that is it is undefined outside the feasible region. Furthermore, the constraints defining the feasible region within the search space are unknown. The approach proposed in this paper, namely SVM-CBO, is organized in two consecutive phases, the first uses a Support Vector Machine classifier to approximate the boundary of the unknown feasible region, the second uses Bayesian Optimization to find a globally optimal solution within the feasible region. In the first phase the next point to evaluate is chosen by dealing with the trade-off between improving the current estimate of the feasible region and discovering possible disconnected feasible sub-regions. In the second phase, the next point to evaluate is selected as the minimizer of the Lower Confidence Bound acquisition function but constrained to the current estimate of the feasible region. The main of the paper is a comparison with a Bayesian Optimization process which uses a fixed penalty value for infeasible function evaluations, under a limited budget (i.e., maximum number of function evaluations). Results are related to five 2D test functions from literature and 80 test functions, with increasing dimensionality and complexity, generated through the Emmental-type GKLS software. SVM-CBO proved to be significantly more effective as well as computationally efficient.

Abstract Searching for accurate machine and deep learning models is a computationally expensive and awfully energivorous process. A strategy which has been recently gaining importance to drastically reduce computational time and energy consumed is to exploit the availability of different information sources, with different computational costs and different “fidelity,” typically smaller portions of a large dataset. The multi-source optimization strategy fits into the scheme of Gaussian Process-based Bayesian Optimization. An Augmented Gaussian Process method exploiting multiple information sources (namely, AGP-MISO) is proposed. The Augmented Gaussian Process is trained using only “reliable” information among available sources. A novel acquisition function is defined according to the Augmented Gaussian Process. Computational results are reported related to the optimization of the hyperparameters of a Support Vector Machine (SVM) classifier using two sources: a large dataset—the most expensive one—and a smaller portion of it. A comparison with a traditional Bayesian Optimization approach to optimize the hyperparameters of the SVM classifier on the large dataset only is reported.

Abstract Modelling human function learning has been the subject of intense research in cognitive sciences. The topic is relevant in black-box optimization where information about the objective and/or constraints is not available and must be learned through function evaluations. In this paper, we focus on the relation between the behaviour of humans searching for the maximum and the probabilistic model used in Bayesian optimization. As surrogate models of the unknown function, both Gaussian processes and random forest have been considered: the Bayesian learning paradigm is central in the development of active learning approaches balancing exploration/exploitation in uncertain conditions towards effective generalization in large decision spaces. In this paper, we analyse experimentally how Bayesian optimization compares to humans searching for the maximum of an unknown 2D function. A set of controlled experiments with 60 subjects, using both surrogate models, confirm that Bayesian optimization provides a general model to represent individual patterns of active learning in humans.

This work stems from the Italian project H-CIM (Health-Care Intelligent Monitoring), aimed at developing a wearable sensor data streams based home-monitoring system to support self-rehabilitation of elderly outpatients. Different from the pervasive data stream applications, which are always accompanied by the evolution of unstable class concepts, this project requires stable standard and personalized rehabilitation exercises patterns be provided to assess outpatient's self-therapy progress at home. In this designed pipeline, the representation sequences of the personal standard rehabilitation exercises in wearable sensor streams is therefore first benchmarked, then an assessment system which integrates multistage data processing and analyzing is proposed to enable elders to manage their own rehabilitation progress properly. The system proved to be an effective tool for supporting compliance monitoring and personalized self-rehabilitation; it is currently under further development within the Italian project Home-IoT, with the aim to become a more general data stream analytics service, not only devoted to rehabilitation exercises assessment.

Bayesian optimization has become a widely used tool in the optimization and machine learning communities. It is suitable to problems as simulation/optimization and/or with an objective function computationally expensive to evaluate. Bayesian optimization is based on a surrogate probabilistic model of the objective whose mean and variance are sequentially updated using the observations and an “acquisition” function based on the model, which sets the next observation at the most “promising” point. The most used surrogate model is the Gaussian Process which is the basis of well-known Kriging algorithms. In this paper, the authors consider the pump scheduling optimization problem in a Water Distribution Network with both ON/OFF and variable speed pumps. In a global optimization model, accounting for time patterns of demand and energy price allows significant cost savings. Nonlinearities, and binary decisions in the case of ON/OFF pumps, make pump scheduling optimization computationally challenging, even for small Water Distribution Networks. The well-known EPANET simulator is used to compute the energy cost associated to a pump schedule and to verify that hydraulic constraints are not violated and demand is met. Two Bayesian Optimization approaches are proposed in this paper, where the surrogate model is based on a Gaussian Process and a Random Forest, respectively. Both approaches are tested with different acquisition functions on a set of test functions, a benchmark Water Distribution Network from the literature and a large-scale real-life Water Distribution Network in Milan, Italy.

A crucial mid-long term goal for the clinical management of chronic heart failure (CHF) patients is to detect in advance new decompensation events, for improving quality of outcomes while reducing costs on the healthcare system. Within the relevant clinical protocols and guidelines, a general consensus has not been reached on how further decompensations could be predicted, even though many different evidence-based indications are known. In this paper we present the Knowledge Discovery (KD) task which has been implemented and developed into the EU FP6 Project HEARTFAID (www.heartfaid.org), proposing an innovative knowledge based platform of services for effective and efficient clinical management of heart failure within elderly population. KD approaches have represented a practical and effective tool for analyzing data about 49 CHF patients who have been recurrently visited by cardiologist, measuring clinical parameters taken from clinical guidelines and evidence-based knowledge and that are also easy to be acquired at home setting. Several KD algorithms have been applied on collected data, obtaining different binary classifiers performing a plausible early detection of new decompensations, showing high accuracy on internal validation and independent test.