Claudio Pinhanez, Paulo Cavalin, Victor Henrique Alves Ribeiro, Ana Appel, Heloisa Candello, Julio Nogima, Mauro Pichiliani, Melina Guerra, Maira de Bayser, Gabriel Malfatti, Henrique Ferreira. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

This paper describes how machine learning training data and symbolic knowledge from curators of conversational systems can be used together to improve the accuracy of those systems and to enable better curatorial tools. This is done in the context of a real-world practice of curators of conversational systems who often embed taxonomically-structured meta-knowledge into their documentation. The paper provides evidence that the practice is quite common among curators, that is used as part of their collaborative practices, and that the embedded knowledge can be mined by algorithms. Further, this meta-knowledge can be integrated, using neuro-symbolic algorithms, to the machine learning-based conversational system, to improve its run-time accuracy and to enable tools to support curatorial tasks. Those results point towards new ways of designing development tools which explore an integrated use of code and documentation by machines.

Chatbots have received significant attention in the last years. These systems have improved operational efficiency by reducing the cost of customer service and more and more are used in customer service channels. More recently, with the addition of speech capabilities added to those systems, bot language modifications have to be done in order to improve user experience. In this paper, we analyze the language used in those systems using datasets collected from real chatbots. For that, we first propose models that are able to identify the language style (writing, speech, and computer-mediated) using as linguist features syntactic (Biber's dimensions) and sentence embeddings as well as state-of-art datasets. Our results show that our models were able to distinguish among the three classes with an accuracy of up to 66%. Finally, we evaluate real chatbots systems, either speech- and text-based ones, using our proposed models. We found that these real chatbots are generally using a computer-mediated style, but there is indication that their developers tend to adapt the language style to be more speech-like when text-to-speech is used.

In this work we present a multi-criteria multi-engine approach for text simplification. The main goal is to demonstrate a way to take advantage of a pool of systems, since in the literature several systems have been proposed for the task, and the results have been improving considerably. Note though, that such systems can behave differently, better or worse than the other ones, according to the input. For this reason, in this work we investigate the benefits of exploiting multiple systems at once, in a single-engine, in order to select the most appropriate simplification output from a pool of candidate outputs. In such an engine, a multi-critera decision making approach selects the final output considering simplicity and similarity scores, by comparing the candidates with the input. Results on both the Turk and WikiSmall corpora indicate that the proposed framework is able to balance the trade-off between bilingual evaluation understudy (BLEU), system output against references and against the input sentence (SARI), and Flesch reading ease scores for existing state-of-the art models.

This paper explores how intent classification can be improved by representing the class labels not as a discrete set of symbols but as a space where the word graphs associated to each class are mapped using typical graph embedding techniques.The approach, inspired by a previous algorithm used for an inverse dictionary task, allows the classification algorithm to take in account inter-class similarities provided by the repeated occurrence of some words in the training examples of the different classes.The classification is carried out by mapping text embeddings to the word graph embeddings of the classes.Focusing solely on improving the representation of the class label set, we show in experiments conducted in both private and public intent classification datasets, that better detection of out-of-scope examples (OOS) is achieved and, as a consequence, that the overall accuracy of intent classification is also improved.In particular, using the recently-released Larson dataset, an error of about 9.9% has been achieved for OOS detection, beating the previous state-of-the-art result by more than 31 percentage points.

Illegal tapping of fuel pipelines has recently become one of the most relevant safety problems faced by the industry. Hundreds of illegal interventions have been reported around the world, causing a significant number of deaths, relevant impacts on the environment, and capital loss. Therefore, it is important to develop systems that are able to detect such scenarios at an early stage, enabling a fast counteract. To this end, machine learning algorithms can train models on available data for detecting future issues. Most recently, ensemble learning and dynamic classifier selection (DCS) techniques have been achieving promising results in supervised learning tasks. Such models are usually trained based on a single criterion. However, it is desirable to take into account both the number of false positives (FP) and false negatives (FN) for the illegal tapping detection task, since they are conflicting and both lead to financial losses and/or accidents. Therefore, this work proposes a novel DCS technique based on multiple criteria, namely overall local class-specific accuracy (OLCA), which employs multi-criteria decision making for dynamically selecting the best classifier for a new sample given the local true positive and negative ratios. A numerical experiment is conducted for assessing the generalization performance of the proposed method in an oil pipeline, with the goal of detecting illegal taping using pressure transient signals. Results show that OLCA is able to reduce the number of both FP and FN when dynamically selecting the classifiers of a baseline Random Forest ensemble.

In the field of machine learning, the application of ensemble methods is one of the most successful techniques in order to achieve a good performance in classification tasks. The combination of multiple classifiers is able to achieve better results than a single model, and much effort has been put into applying multi-objective optimisation for improving results with diverse ensemble generation and classifier aggregation. Most recently, dynamic classifier selection and weighting has acquired relevance in the field of multiple-classifier systems. However, to the authors knowledge, there has not yet been a comparison study of Pareto based techniques and dynamic ensemble methods. Thus, this paper proposes a comparison of two ensemble member generation techniques (Pareto-based diverse ensemble generation and bootstrap aggregating) and five aggregation methods (selection of the best classifier, majority voting with all members, majority voting with members selected with multi-objective optimisation, dynamic classifier selection and dynamic classifier weighting), performed on six binary classification benchmark data sets. Results indicate that the combination of bootstrap aggregating and majority voting with multi-objective ensemble member selection achieves the best performance.

Short-term forecasting plays an important role in the economic area. Several studies have been carried, where models with good forecast capacity, focusing on accuracy or stability, were built. Modeling only one of these characteristics without the other can lead to a model with lower generalization capacity. To deal with such situation, this study proposes an ensemble model (EM) to forecast one, two and three months ahead the 60 kg corn bag prices received by producers in the state of Parana (Brazil). Additionally, feature extraction by means of principal component analysis is employed. The EM is built using machine learning models as base (weak) learners (BL) combined by weighted sum. The adopted BL are: Extremely randomized trees, partial least squares, k-nearest neighbors, neural network, bagging and multivariate adaptive regression splines. The weights are chosen through multi-objective optimization, where bias and variance are minimized. The multi-objective differential evolution with spherical pruning algorithm is adopted, while physical programming is used in order to obtain the preferred set of weights. The model built is appointed as multi-objective ensemble model (MOEM). The performance of the model is evaluated using mean absolute percentage error, mean squared error and root mean squared error. Additionally, the Diebold-Mariano test is used to evaluate the reduction on forecasting errors. In general lines, the results show that forecasting using MOEM with two, three or four BL is more stable and accurate than forecasting with single BL. Therefore, this approach is recommended to make short-term forecast of corn prices, which leads to a more assertive decision making.

Real-world classification problems generally deal with imbalanced data, where one class represents the majority of the data set. The present work deals with event detection on a drinking-water quality time series, where the presence of a quality event is the minority class. In order to solve such problems, supervised learning algorithms are recommended. Researchers have also used multi-objective optimization (MOO) in order to generate diverse models to build ensembles of classifiers. Although MOO has been used for ensemble member generation, there is a lack on it's application for member selection, which is usually done by selecting a specific subset from the resulting models, or by using meta-algorithms, such as boosting. The proposed work comprises the application of MOO design in the whole process of ensemble generation. To do so, one multi-objective problem (MOP) is defined for the creation of a set of non-dominated solutions with Pareto-optimal support vector machines (SVM). After that, a second MOP is defined for the selection of such SVMs as members of an ensemble. Such methodology is compared to other member selection methods, such as: the single best classifier, an ensemble composed of the full set of non-dominated solutions, and the selection of a specific subset from the Pareto front. Results show that the proposed method is suitable for the creation of ensembles, achieving the highest classification scores.

Control of industrial plants is an important engineering field of study, where the optimization of such processes can lead to better product quality and higher profit. The present work deals with the optimization of an industrial gasifier, proposed as a benchmark challenge by ALSTOM Power Technologies in 2002. Researchers from around the globe proposed different methods for optimizing such system, using linear and nonlinear system identification techniques, different controller schemes, such as proportional-integral-derivative and model predictive controllers, and search methods for controller tuning. The authors care for the study of a methodology that can generalize the controller optimization process of real industrial systems, where linear system identification can lead to simpler systems, which are used to design more robust controllers. The proposed work deals with linear system identification and multiobjective optimization for the gasifier's controller design. To do so, the authors divide the problem in two steps, the linear system identification, where step functions are fed to the gasifier in order to detect the output response of the system, and the design of a proportional-integral-derivative controller, where spherical pruned multi-objective differential evolution algorithm is used in order to optimize the controller gains. The proposed method, unfortunately, does not provide an optimal controller, which can be explained by the complexity of the benchmark system. With such results, the authors believe a better multi-objective problem definition is necessary. Also, it is concluded that such benchmark presents a real challenge for future controller design methndologies.