Time series forecasting, particularly in the domain of stock prices, is a significant challenge but benefits from the availability of openly accessible data. Our work focuses mainly on (although not limited to) univariate time series forecasting of monthly or daily stock prices, predicting one step ahead. We developed an innovative pipeline that combines fuzzification with Automated Machine Learning, achieving improved forecasting performance. Unlike previous literature, we revise the binomial Fuzzy Time Series and machine learning algorithm , including a classification task (formally motivating it), and involving unused features of fuzzy sets . Thanks to the type of aggregation of the fuzzified data, the approach has the potential to preserve interpretability, unlike most machine learning based approaches . Using several financial datasets, in addition to preliminary experiments on chaotic time series, we found evidence of significantly improved performance in most cases. This study contributes to further understanding of the intersection between fuzzy logic and Automated Machine Learning, particularly in the context of time series forecasting, offering a promising direction for future research.

Federated learning (FL) enables decentralized model training without centralizing raw data, yet achieving interpretability under such constraints remains challenging. We propose AutoCoRe-FL, a framework for interpretable FL that eliminates the need for predefined or manually labeled concepts. In AutoCoRe-FL, each client automatically extracts high-level visual concepts-clusters of semantically coherent image regions that correspond to human-understandable properties-using local segmentation, self-supervised representation learning, and clustering. These concepts are used to encode data as interpretable vectors, from which clients train symbolic models that generate rule-based explanations. The server then aggregates these rules through an iterative, communication-efficient process to build a global, coherent, and transparent model. Experiments on benchmark datasets demonstrate that AutoCoRe-FL produces accurate symbolic explanations while achieving competitive predictive performance. Notably, it outperforms LR-XFL-the current state-of-the-art interpretable FL baseline that relies on predefined concept supervision-in both rule quality and classification accuracy.

In an era where image manipulation tools are widely accessible, detecting digital image forgery has become increasingly challenging. Image forgery detection is a critical concern in cybersecurity, particularly within IoT-based applications and digital forensics. This paper proposes an automated approach for image forgery detection, integrating Fuzzy C-Means clustering with TPOT, an automated machine learning framework, to optimize the detection pipeline. The approach is evaluated on four publicly available datasets: FIDAC, FID, CoMoFoD, and CASIA_V2. Experimental results demonstrate that the proposed approach outperforms conventional CNN and VGG models, achieving 96.83% accuracy on CASIA_V2 and 95.33% on CoMoFoD. These findings highlight the potential of AutoML-based solutions to enhance forgery detection.

With the booming demand for machine learning applications, it has been recognized that the number of knowledgeable data scientists can not scale with the growing data volumes and application needs in our digital world. In response to this demand, several automated machine learning (AutoML) frameworks have been developed to fill the gap of human expertise by automating the process of building machine learning pipelines. Each framework comes with different heuristics-based design decisions. In this study, we present a comprehensive evaluation and comparison of the performance characteristics of six popular AutoML frameworks, namely, AutoWeka, AutoSKlearn, TPOT, Recipe, ATM and SmartML across 100 data sets from established AutoML benchmark suites. Our experimental evaluation considers different aspects for its comparison, including the performance impact of several design decisions, including time budget, size of search space, meta-learning, and ensemble construction. The results of our study reveal various interesting insights that can significantly guide and impact the design of AutoML frameworks.

Automated Machine Learning (AutoML) has been successful when the learning task is assumed to be static. However, it remains unclear whether AutoML methods can efficiently create online pipelines in dynamic environments. The current online AutoML frameworks primarily focus on supervised learning. However, unsupervised learning, particularly clustering, also requires AutoML solutions, especially with the ambiguity associated with evaluating clustering results. In this paper, we introduce OnlineAutoClust, a framework for online automated clustering for algorithm selection and hyperparameter tuning. OnlineAutoClust combines the inherent adaptation capabilities of online learners with automated pipeline optimization using Bayesian optimization. OnlineAutoClust develops a collaborative mechanism based on clustering ensemble to combine optimized pipelines based on different internal cluster validity indices. The proposed framework is based on River library and utilizes five clustering algorithms. Empirical evaluation on several real and synthetic data streams with varying types of concept drift demonstrates the effectiveness of the proposed approach compared to existing methods

Machine learning models are incorporated in different fields and disciplines, some of which require high accountability and transparency, for example, the healthcare sector. A widely used category of explanation techniques attempts to explain models' predictions by quantifying the importance score of each input feature. However, summarizing such scores to provide human-interpretable explanations is challenging. Another category of explanation techniques focuses on learning a domain representation in terms of high-level human-understandable concepts and then utilizing them to explain predictions. These explanations are hampered by how concepts are constructed, which is not intrinsically interpretable. To this end, we propose Concept-based Local Explanations with Feedback (CLEF), a novel local model agnostic explanation framework for learning a set of high-level transparent concept definitions in high-dimensional tabular data that uses clinician-labeled concepts rather than raw features.

Feature engineering is a crucial step in building well-performing machine learning pipelines. However, manually constructing highly predictive features is time-consuming and requires domain knowledge. Although the research area of automated feature engineering has attracted much interest lately, both in academia and industry, the scalability and efficiency of the existing systems and tools are still practically unsatisfactory. This paper presents a scalable and interpretable automated feature engineering framework, BigFeat, that optimizes input features’ quality to maximize the predictive performance according to a user-defined metric. BigFeat employs a dynamic feature generation and selection mechanism that constructs a set of expressive features that improve the prediction performance while retaining interpretability. Extensive experiments are conducted, and the results show that BigFeat provides superior performance compared to the state-of-the-art automated feature engineering framework, AutoFeat, on a wide range of datasets. We show that BigFeat significantly improves the F1-Score of 8 classifiers by 4.59%, on average. In addition, the performance improvement achieved by integrating BigFeat into different AutoML frameworks is higher than that achieved by integrating AutoFeat into the same frameworks. Besides, the scalability of BigFeat is confirmed by its linear complexity, parallel design, and execution time which is, on average, 22x faster than AutoFeat.

Machine learning models are incorporated in different fields and disciplines in which some of them require a high level of accountability and transparency, for example, the healthcare sector. With the General Data Protection Regulation (GDPR), the importance for plausibility and verifiability of the predictions made by machine learning models has become essential. A widely used category of explanation techniques attempts to explain models’ predictions by quantifying the importance score of each input feature. However, summarizing such scores to provide human-interpretable explanations is challenging. Another category of explanation techniques focuses on learning a domain representation in terms of high-level human-understandable concepts and then utilizing them to explain predictions. These explanations are hampered by how concepts are constructed, which is not intrinsically interpretable. To this end, we propose Concept-based Local Explanations with Feedback (CLEF), a novel local model agnostic explanation framework for learning a set of high-level transparent concept definitions in high-dimensional tabular data that uses clinician-labeled concepts rather than raw features. CLEF maps the raw input features to high-level intuitive concepts and then decompose the evidence of prediction of the instance being explained into concepts. In addition, the proposed framework generates counterfactual explanations, suggesting the minimum changes in the instance’s concept based explanation that will lead to a different prediction. We demonstrate with simulated user feedback on predicting the risk of mortality. Such direct feedback is more effective than other techniques, that rely on hand-labelled or automatically extracted concepts, in learning concepts that align with ground truth concept definitions.

Novel technologies in automated machine learning ease the complexity of algorithm selection and hyper-parameter optimization. However, these are usually restricted to supervised learning tasks such as classification and regression, while unsupervised learning remains a largely unexplored problem. In this paper, we offer a solution for automating machine learning specifically for the case of unsupervised learning with clustering, in a domain-agnostic manner. This is achieved through a combination of state-of-the-art processes based on meta-learning for algorithm and evaluation criteria selection, and evolutionary algorithm for hyper-parameter tuning. We introduce a robust and scalable interactive tool, named cSmartML, built on scikit-learn with 8 clustering algorithms. In order to capture more than a single measure of goodness of the output clustering solution, cSmartML optimizes multiple objective functions. A pareto-approach evaluates each objective simultaneously for each clustering solution. On each of the 27 real and synthetic benchmark datasets, we show that the performance of cSmartML is often much better than using standard selection and hyper-parameter optimization methods. In addition, experimentation reveals that cSmartML takes advantage of the defined objective functions on multi-objective functions framework.

Abstract Although complex machine learning models (eg, random forest, neural networks) are commonly outperforming the traditional and simple interpretable models (eg, linear regression, decision tree), in the healthcare domain, clinicians find it hard to understand and trust these complex models due to the lack of intuition and explanation of their predictions. With the new general data protection regulation (GDPR), the importance for plausibility and verifiability of the predictions made by machine learning models has become essential. Hence, interpretability techniques for machine learning models are an area focus of research. In general, the main aim of these interpretability techniques is to shed light and provide insights into the prediction process of the machine learning models and to be able to explain how the results from the prediction was generated. A major problem in this context is that both the quality of the interpretability techniques and trust of the machine learning model predictions are challenging to measure. In this article, we propose four fundamental quantitative measures for assessing the quality of interpretability techniques— similarity , bias detection , execution time , and trust . We present a comprehensive experimental evaluation of six recent and popular local model agnostic interpretability techniques, namely, LIME , SHAP , Anchors , LORE , ILIME “ and MAPLE on different types of real‐world healthcare data. Building on previous work, our experimental evaluation covers different aspects for its comparison including identity , stability , separability , similarity , execution time , bias detection , and trust . The results of our experiments show that MAPLE achieves the highest performance for the identity across all data sets included in this study, while LIME achieves the lowest performance for the identity metric. LIME achieves the highest performance for the separability metric across all data sets. On average, SHAP has the smallest average time to output explanation across all data sets included in this study. For detecting the bias, SHAP and MAPLE enable the participants to better detect the bias. For the trust metric, Anchors achieves the highest performance on all data sets included in this work.

Nowadays, machine learning is playing a crucial role in harnessing the value of massive data amount currently produced every day. The process of building a high-quality machine learning model is an iterative, complex and time-consuming process that requires solid knowledge about the various machine learning algorithms in addition to having a good experience with effectively tuning their hyper-parameters. With the booming demand for machine learning applications, it has been recognized that the number of knowledgeable data scientists can not scale with the growing data volumes and application needs in our digital world. Therefore, recently, several automated machine learning (AutoML) frameworks have been developed by automating the process of Combined Algorithm Selection and Hyper-parameter tuning (CASH). However, a main limitation of these frameworks is that they have been built on top of centralized machine learning libraries (e.g. scikit-learn) that can only work on a single node and thus they are not scalable to process and handle large data volumes. To tackle this challenge, we demonstrate D-SmartML, a distributed AutoML framework on top of Apache Spark, a distributed data processing framework. Our framework is equipped with a meta learning mechanism for automated algorithm selection and supports three different automated hyper-parameter tuning techniques: distributed grid search, distributed random search and distributed hyperband optimization. We will demonstrate the scalability of our framework on handling large datasets. In addition, we will show how our framework outperforms the-state-of-the-art framework for distributed AutoML optimization, TransmogrifAI.

Although complex machine learning models (e.g., Random Forest, Neural Networks) are commonly outperforming the traditional simple interpretable models (e.g., Linear Regression, Decision Tree), in the healthcare domain, clinicians find it hard to understand and trust these complex models due to the lack of intuition and explanation of their predictions. With the new General Data Protection Regulation (GDPR), the importance for plausibility and verifiability of the predictions made by machine learning models has become essential. To tackle this challenge, recently, several machine learning interpretability techniques have been developed and introduced. In general, the main aim of these interpretability techniques is to shed light and provide insights into the predictions process of the machine learning models and explain how the model predictions have resulted. However, in practice, assessing the quality of the explanations provided by the various interpretability techniques is still questionable. In this paper, we present a comprehensive experimental evaluation of three recent and popular local model agnostic interpretability techniques, namely, LIME, SHAP and Anchors on different types of real-world healthcare data. Our experimental evaluation covers different aspects for its comparison including identity, stability, separability, similarity, execution time and bias detection. The results of our experiments show that LIME achieves the lowest performance for the identity metric and the highest performance for the separability metric across all datasets included in this study. On average, SHAP has the smallest average time to output explanation across all datasets included in this study. For detecting the bias, SHAP enables the participants to better detect the bias.