A crucial task in predictive maintenance is estimating the remaining useful life of physical systems. In the last decade, deep learning has improved considerably upon traditional model-based and statistical approaches in terms of predictive performance. However, in order to optimally plan maintenance operations, it is also important to quantify the uncertainty inherent in the predictions. This issue can be addressed by turning standard frequentist neural networks into Bayesian neural networks, which are naturally capable of providing confidence intervals around the estimates. Several methods exist for training those models. Researchers have focused mostly on parametric variational inference and sampling-based techniques, which notoriously suffer from limited approximation power and large computational burden, respectively. In this work, we use Stein variational gradient descent, a recently proposed algorithm for approximating intractable distributions that overcomes the drawbacks of the aforementioned techniques. In particular, we show through experimental studies on both simulated run-to-failure turbofan engine degradation data and real industrial battery degradation data that Bayesian deep learning models trained via Stein variational gradient descent consistently outperform with respect to convergence speed and predictive performance both the same models trained via parametric variational inference and their frequentist counterparts trained via backpropagation. Furthermore, we propose a method to enhance performance based on the uncertainty information provided by the Bayesian models.

In the semiconductor sector, due to high demand but also strong and increasing competition, time to market and quality are key factors in securing significant market share in various application areas. Thanks to the success of deep learning methods in recent years in the computer vision domain, Industry 4.0 and 5.0 applications, such as defect classification, have achieved remarkable success. In particular, Domain Adaptation (DA) has proven highly effective since it focuses on using the knowledge learned on a (source) domain to adapt and perform effectively on a different but related (target) domain. By improving robustness and scalability, DA minimizes the need for extensive manual re-labeling or retraining of models. This not only reduces computational and resource costs but also allows human experts to focus on high-value tasks. Therefore, we tested the efficacy of DA techniques in semi-supervised and unsupervised settings within the context of the semiconductor field. Moreover, we propose the DBACS approach, a CycleGAN-inspired model enhanced with additional loss terms to improve performance. All the approaches are studied and validated on real-world Electron Microscope images, considering the unsupervised and semi-supervised settings, proving the usefulness of our method in advancing DA techniques for the semiconductor field.

In the rapidly evolving landscape of Industry 4.0 and the forthcoming Industry 5.0, the integration of intelligent systems into industrial automation has become a cornerstone for achieving efficiency, adaptability, and sustainability. In particular, deep learning (DL) has been central to this transformation, by empowering machines to extract meaningful patterns from complex, high-dimensional data, DL has demonstrated remarkable success in tackling industrial challenges such as anomaly detection, predictive maintenance, quality control, and process optimization However, as industrial systems become increasingly interconnected and dynamic, classic supervised learning paradigms often fail to meet the practical demands of real-world environments and operate under assumptions that are frequently violated in real-world industrial deployments.

Imposing input-output constraints in multi-layer perceptrons (MLPs) plays a pivotal role in many real world applications. Monotonicity in particular is a common requirement in applications that need transparent and robust machine learning models. Conventional techniques for imposing monotonicity in MLPs by construction involve the use of non-negative weight constraints and bounded activation functions, which poses well known optimization challenges. In this work, we generalize previous theoretical results, showing that MLPs with non-negative weight constraint and activations that saturate on alternating sides are universal approximators for monotonic functions. Additionally, we show an equivalence between saturation side in the activations and sign of the weight constraint. This connection allows us to prove that MLPs with convex monotone activations and non-positive constrained weights also qualify as universal approximators, in contrast to their non-negative constrained counterparts. This results provide theoretical grounding to the empirical effectiveness observed in previous works, while leading to possible architectural simplification. Moreover, to further alleviate the optimization difficulties, we propose an alternative formulation that allows the network to adjust its activations according to the sign of the weights. This eliminates the requirement for weight reparameterization, easing initialization and improving training stability. Experimental evaluation reinforce the validity of the theoretical results, showing that our novel approach compares favorably to traditional monotonic architectures.

Fair classification is a critical challenge that has gained increasing importance due to international regulations and its growing use in high-stakes decision-making settings. Existing methods often rely on adversarial learning or distribution matching across sensitive groups; however, adversarial learning can be unstable, and distribution matching can be computationally intensive. To address these limitations, we propose a novel approach based on the characteristic function distance. Our method ensures that the learned representation contains minimal sensitive information while maintaining high effectiveness for downstream tasks. By utilizing characteristic functions, we achieve a more stable and efficient solution compared to traditional methods. Additionally, we introduce a simple relaxation of the objective function that guarantees fairness in common classification models with no performance degradation. Experimental results on benchmark datasets demonstrate that our approach consistently matches or achieves better fairness and predictive accuracy than existing methods. Moreover, our method maintains robustness and computational efficiency, making it a practical solution for real-world applications.

Despite the critical importance of the medical domain in Deep Learning, most of the research in this area solely focuses on training models in static environments. It is only in recent years that research has begun to address dynamic environments and tackle the Catastrophic Forgetting problem through Continual Learning (CL) techniques. Previous studies have primarily focused on scenarios such as Domain Incremental Learning and Class Incremental Learning, which do not fully capture the complexity of real-world applications. Therefore, in this work, we propose a novel benchmark combining the challenges of new class arrivals and domain shifts in a single framework, by considering the New Instances and New Classes (NIC) scenario. This benchmark aims to model a realistic CL setting for the multi-label classification problem in medical imaging. Additionally, it encompasses a greater number of tasks compared to previously tested scenarios. Specifically, our benchmark consists of two datasets (NIH and CXP), nineteen classes, and seven tasks. To solve common challenges (e.g., the task inference problem) found in the CIL and NIC scenarios, we propose a novel approach called Replay Consolidation with Label Propagation (RCLP). Our method surpasses existing approaches, exhibiting superior performance with minimal forgetting.

Behavior-based Driver Identification is an emerging technology that recognizes drivers based on their unique driving behaviors, offering important applications such as vehicle theft prevention and personalized driving experiences. However, most studies fail to account for the real-world challenges of deploying Deep Learning models within vehicles. These challenges include operating under limited computational resources, adapting to new drivers, and changes in driving behavior over time. The objective of this study is to evaluate if Continual Learning (CL) is well-suited to address these challenges, as it enables models to retain previously learned knowledge while continually adapting with minimal computational overhead and resource requirements. We tested several CL techniques across three scenarios of increasing complexity based on a well-known dataset for the Driver Identification problem. This work provides an important step forward in scalable driver identification solutions, demonstrating that CL approaches, such as Dark Experience Replay (DER), can obtain strong performance with only an 11% reduction in accuracy compared to the static scenario. Furthermore, to enhance the performance, we propose two new methods, Smooth Experience Replay (SmooER) and Smooth Dark Experience Replay (SmooDER), that leverage the temporal continuity of driver identity over time to enhance classification accuracy. Our novel method, SmooDER, achieves optimal results with only a 2% accuracy reduction compared to the 11% of the DER approach. In conclusion, this study proves the feasibility of CL approaches to address the challenges of Driver Identification in dynamic environments, making them suitable for deployment on cloud infrastructure or directly within vehicles. • We investigate Driver Identification in a realistic setting, adapting to new drivers. • We propose three Continual Learning scenarios with progressive real-world alignment. • We propose SmooDER and SmooER, leveraging driver continuity to boost performance. • We validate the effectiveness of these techniques using the OCSLab dataset.

Undesirable biases encoded in the data are key drivers of algorithmic discrimination. Their importance is widely recognized in the algorithmic fairness literature, as well as legislation and standards on anti-discrimination in AI. Despite this recognition, data biases remain understudied, hindering the development of computational best practices for their detection and mitigation. In this work, we present three common data biases and study their individual and joint effect on algorithmic discrimination across a variety of datasets, models, and fairness measures. We find that underrepresentation of vulnerable populations in training sets is less conducive to discrimination than conventionally affirmed, while combinations of proxies and label bias can be far more critical. Consequently, we develop dedicated mechanisms to detect specific types of bias, and combine them into a preliminary construct we refer to as the Data Bias Profile (DBP) . This initial formulation serves as a proof of concept for how different bias signals can be systematically documented. Through a case study with popular fairness datasets, we demonstrate the effectiveness of the DBP in predicting the risk of discriminatory outcomes and the utility of fairness-enhancing interventions. Overall, this article bridges algorithmic fairness research and anti-discrimination policy through a data-centric lens.

Deep Reinforcement Learning (DRL) has emerged as a powerful paradigm for learning complex policies directly from high-dimensional input spaces, enabling advances across a variety of domains. Modern DRL algorithms often rely on dual-network Q-learning architectures to approximate optimal policies to overcome overestimation bias. Recent research has introduced approaches leveraging multiple Q-functions to further mitigate overestimation effects and enhance policy reliability. However, there is a growing emphasis on deploying DRL in edge scenarios, where privacy concerns and stringent hardware constraints necessitate highly efficient algorithms. In such environments, the computational and memory efficiency of learning methods is of critical importance. In this context, we propose Edge Delayed Deep Deterministic Policy Gradient (EdgeD3), a novel reinforcement learning algorithm specifically designed for edge computing settings. EdgeD3 offers significant reductions in GPU time (by 25%) and computational and memory usage (by 30%), while consistently achieving or surpassing the performance of state-of-the-art algorithms across multiple benchmarks and in real-world tasks.

Efficient decision-making is paramount in manufacturing industries, particularly in sectors like semiconductor manufacturing, which operate within high-demand environments. The semiconductor manufacturing domain, driven by the pervasive utilization of electronics in computing and sensing devices, confronts escalating challenges related to quality control and productivity optimization. This work centers on predicting Overall Equipment Efficiency (OEE), a pivotal metric for pinpointing production efficiency hurdles and refining decision-making processes. Despite its widespread adoption across various industrial domains, there exists a dearth of literature concerning OEE prediction methodologies, with no literature in the context of semiconductor manufacturing. In this work, we propose Deep Learning-based Sequential Learning approaches for OEE estimations. Specifically, we employ the CEEMDAN-GRU model, a deep learning architecture that amalgamates modeling techniques with signal filtering, marking the first instance of its application in OEE prediction. We assess the efficacy of our approach leveraging real-world data sourced from a semiconductor manufacturing facility.

While Machine Learning has become crucial for Industry 4.0, its opaque nature hinders trust and impedes the transformation of valuable insights into actionable decision, a challenge exacerbated in the evolving Industry 5.0 with its human-centric focus. This paper addresses this need by testing the applicability of AcME-AD in industrial settings. This recently developed framework facilitates fast and user-friendly explanations for anomaly detection. AcME-AD is modelagnostic, offering flexibility, and prioritizes real-time efficiency. Thus, it seems suitable for seamless integration with industrial Decision Support Systems. We present the first industrial application of AcME-AD, showcasing its effectiveness through experiments. These tests demonstrate AcME-AD’s potential as a valuable tool for explainable AD and feature-based root cause analysis within industrial environments, paving the way for trustworthy and actionable insights in the age of Industry 5.0.

With the rise of Machine Learning (ML) and Artificial Intelligence (AI), the semiconductor industry is undergoing a revolution in how it approaches manufacturing. The SMART-IC project (DATE'24 MPP category: initial stage) works in this direction, by proposing an AI-enabled framework to support the smart monitoring and optimization of the semiconductor manufacturing process. An AI-powered engine examines sensor data recording physical parameters during production (like gas flow, temperature, voltage, etc.) as well as test data, with different goals: (1) the identification of anomalies in the production chain, either offline from collected data-traces or online from a continuous stream of sensed data; (2) the forecasting of new data of the future production; and (3) the automatic generation of synthetic traces, to strengthen the data-based algorithms. All such tasks provide valuable information to an advanced Manufacturing Execution System (MES), which reacts by optimizing the production process and management of the equipment maintenance policies. SMART-IC is a 300k€ academic project funded by the Italian Ministry of University and supported by STMicroelectronics and Technoprobe with industrial expertise and real-world applications. This paper shares the view of SMART-IC on the future of semiconductor manufacturing, the preliminary efforts, and the future results that will be reached by the end of the project, in 2025.

Anomaly Detection (AD) focuses on identifying unusual patterns in complex datasets and systems. While Machine Learning and Decision Support Systems (DSS) are effective for this, simply detecting anomalies often falls short in real-world scenarios, especially in engineering contexts where diagnostics and maintenance are essential. Users need clear explanations behind anomaly predictions to understand the root causes and trust the model. The unsupervised nature of AD complicates the development of interpretable tools. To address this, we propose the Extended Isolation Forest Feature Importance (ExIFFI), a new approach that explains the predictions of the Extended Isolation Forest (EIF), applicable to all Isolation Forest models that split using hyperplanes. ExIFFI provides both global and local explanations by analyzing feature importance. Additionally, we introduce Enhanced Extended Isolation Forest ( EIF + ), an improved version of EIF, designed to better detect unseen anomalies by modifying the splitting strategy of hyperplanes. We compare various unsupervised AD methods across five synthetic and eleven real-world datasets using the Average Precision metric. EIF + consistently outperforms EIF in all scenarios, demonstrating superior generalization. To validate the interpretability, we propose a new metric — A U C F S (Area Under the Curve of Feature Selection) — which uses feature selection as a performance indicator. ExIFFI proves more effective than other unsupervised interpretation methods, excelling in 8 out of 11 real-world datasets and correctly identifying anomalous features in synthetic datasets. Finally, we provide open-source code to encourage further research and reproducibility. • Introduced ExIFFI, an interpretability tool for standard and Extended Isolation Forests. • Developed EIF + , an advanced model enhancing generalization in anomaly detection. • Demonstrated superior performance of ExIFFI and EIF + on synthetic and real-world data. • Contributed open-source code to support research and reproducibility in machine learning. • Applied a novel feature importance metric for unsupervised anomaly detection interpretability.

In this paper, we present the project “VIR2EM: VIrtualization and Remotization for Resilient and Efficient Manufacturing” by providing details on its research themes and its scientific and technological output. The project, centered on virtualization and remotization in the industrial sector, was promoted by Regione Veneto in Italy, and it has seen the participation and collaboration of 3 universities, 1 public research entity, and 10 companies composed of end users of digital solutions and high knowledge-intensive service providers. The project aims to develop and use tools for the virtualization of processes, systems, resources, and remoting of operations in order to: (1) maximize the efficiency of manufacturing systems under normal operating conditions; (2) maintain operations in case of emergency situations; (3) facilitate the restart of operations downstream of emergency situations by ensuring flexibility and predictive capability. Each theoretical proposal has been validated in distinct industrial facilities by constructing ten different prototypes.

Hydroelectric is currently the most prominent among the sources of green energy, but, differently from the other sources, it has very strict requirements in terms of security that are taken into account with extremely robust constraints both at design and operations control times. In this paper, we evaluated the effectiveness of anomaly detection and explainability algorithms to supplement Decision Support System insights in Predictive Maintenance and Root Cause Analysis for hydroelectric power plants. The objective is to reduce operational costs and increase reliability in the plant, making hydroelectric technology more appealing to investors and promoting the transition to renewable energy. Specifically, the performance of several anomaly detection models was compared on real-world data with respect to the needs of the expert of the domain, that is the final user of the DSS, to work as an additional feature to speed up predictive maintenance. Additionally, the impact of SHapley Additive exPlanations values on helping the user understand the anomaly causes was investigated. Our findings are that the most performing algorithm was Auto-Encoder since it was able to find all recorded anomalies and even propose additional ones later confirmed by domain experts. The application of SHAP values was found to effectively guide the user toward the features related to the anomaly, although its application on streaming data was slow.

Predictive Maintenance (PdM) aims to detect forth-coming failures in machinery to reduce costs associated with defective products and equipment inactivity. Remaining Useful Life (RUL) estimation is the most common approach in PdM: in this formalization, forecast or regression models aim at predicting the time/process iterations left before machinery loses its operation ability or a failure happens. In the RUL literature, Deep Learning (DL) algorithms are typically the preferred choice because they achieve high performance and can automatically handle the feature extraction phase. Usually, developed DL architectures are application or equipment specific; thus, there is no clear way to select, design, or implement such architectures. However, the research usually does not justify the choice of one architecture over another that may potentially work for the same problem. In addition, many of the reviewed papers do not investigate the computational complexity of these techniques, which is a critical aspect of real-time applications. In this work, we compare the most widely used deep learning architectures for performing RUL estimation in four datasets: two public datasets known in the PdM research community and two confidential industrial datasets. Moreover, we release a library called CeRULEo, to support the research within this field, speeding up the development of RUL models and providing a complete pre-processing pipeline for dataset handling.

Fairness and equity have become central to ranking problems in information access systems, such as search engines, recommender systems, or marketplaces. To date, several types of fair ranking measures have been proposed, including diversity, exposure, and pairwise fairness measures. Out of those, pairwise fairness is a family of metrics whose normative grounding has not been clearly explicated, leading to uncertainty with respect to the construct that is being measured and how it relates to stakeholders' desiderata.

Continual Learning aims to learn from a stream of tasks, being able to remember at the same time both new and old tasks. We propose a scenario that holds immense appeal for various real-world applications, where a model adapts to handle a stream of machines with distribution shifts Tests on real packaging data proved the feasibility of Continual Learning for addressing such problems. Our study uncovers the limitations of previous algorithms in the Domain Incremental Learning. Our research presents a novel approach for tackling multi-label tasks in Continual Learning, achieving superior performance compared to existing approaches found in the literature. Our method not only achieves optimal performance but also has logarithmic complexity, significantly reducing computation times.

Various unsupervised greedy selection methods have been proposed as computationally tractable approximations to the NP-hard subset selection problem. These methods rely on sequentially selecting the variables that best improve performance with respect to a selection criterion. Theoretical results exist that provide performance bounds and enable ‘lazy greedy’ efficient implementations for selection criteria that satisfy a diminishing returns property known as submodularity. Recently, the authors introduced Forward Selection Component Analysis (FSCA) which uses variance explained as its selection criterion. While variance explained is not a submodular criterion, FSCA has been shown to be highly effective for applications such as measurement plan optimization. Motivated by the desire to achieve a more computationally efficient and scalable algorithm implementation, in this paper a ‘lazy’ implementation of FSCA (L-FSCA) is proposed, which, although not equivalent to FSCA due to the absence of submodularity, has the potential to yield comparable performance while being up to an order of magnitude faster to compute. The efficacy of L-FSCA is demonstrated by performing a systematic comparison with FSCA and five other unsupervised variable selection methods from the literature using simulated and real-world case studies. Experimental results confirm that L-FSCA yields almost identical performance to FSCA while reducing computation time by between 22% and 94% for the case studies considered.

Abstract Data-driven algorithms are studied and deployed in diverse domains to support critical decisions, directly impacting people’s well-being. As a result, a growing community of researchers has been investigating the equity of existing algorithms and proposing novel ones, advancing the understanding of risks and opportunities of automated decision-making for historically disadvantaged populations. Progress in fair machine learning and equitable algorithm design hinges on data, which can be appropriately used only if adequately documented. Unfortunately, the algorithmic fairness community, as a whole, suffers from a collective data documentation debt caused by a lack of information on specific resources (opacity) and scatteredness of available information (sparsity). In this work, we target this data documentation debt by surveying over two hundred datasets employed in algorithmic fairness research, and producing standardized and searchable documentation for each of them. Moreover we rigorously identify the three most popular fairness datasets, namely Adult, COMPAS, and German Credit, for which we compile in-depth documentation. This unifying documentation effort supports multiple contributions. Firstly, we summarize the merits and limitations of Adult, COMPAS, and German Credit, adding to and unifying recent scholarship, calling into question their suitability as general-purpose fairness benchmarks. Secondly, we document hundreds of available alternatives, annotating their domain and supported fairness tasks, along with additional properties of interest for fairness practitioners and researchers, including their format, cardinality, and the sensitive attributes they encode. We summarize this information, zooming in on the tasks, domains, and roles of these resources. Finally, we analyze these datasets from the perspective of five important data curation topics: anonymization, consent, inclusivity, labeling of sensitive attributes, and transparency. We discuss different approaches and levels of attention to these topics, making them tangible, and distill them into a set of best practices for the curation of novel resources.

Abstract LEarning TO Rank (LETOR) algorithms are usually trained on annotated corpora where a single relevance label is assigned to each available document‐topic pair. Within the Cranfield framework, relevance labels result from merging either multiple expertly curated or crowdsourced human assessments. In this paper, we explore how to train LETOR models with relevance judgments distributions (either real or synthetically generated) assigned to document‐topic pairs instead of single‐valued relevance labels. We propose five new probabilistic loss functions to deal with the higher expressive power provided by relevance judgments distributions and show how they can be applied both to neural and gradient boosting machine (GBM) architectures. Moreover, we show how training a LETOR model on a sampled version of the relevance judgments from certain probability distributions can improve its performance when relying either on traditional or probabilistic loss functions. Finally, we validate our hypothesis on real‐world crowdsourced relevance judgments distributions. Overall, we observe that relying on relevance judgments distributions to train different LETOR models can boost their performance and even outperform strong baselines such as LambdaMART on several test collections.

In manufacturing industries, it is of fundamental importance to detect anomalies in production in order to meet the required quality goals and to limit the number of defective products that are accidentally delivered to the customers. Nevertheless, monitoring systems currently employed in production are typically very simple and rely on a set of univariate control charts that fail to capture the multivariate and complex nature of real-world industrial systems. In such context, Machine Learning (ML)-based approaches for Anomaly Detection (AD) have proven to be extremely effective in increasing anomalies detectability and, in general, in enhancing monitoring procedures. However, industrial data are typically very complex and not suitable to be fed directly to classical ML-based AD tools making feature extraction procedures a necessary step that unfortunately may lead to information loss and low scalability. Deep Learning, has proven very effective at learning useful representations of complex data in an automatic way. In this paper, we propose an AD pipeline that makes use of convolutional autoencoders to extract useful features from two-dimensional, non-image, data. We test our approach on real world Optical Emission Spectroscopy data that are typical of semiconductor manufacturing and we achieve improved performance over classical monitoring methods.Note to Practitioners—Advanced monitoring is one of the most important task in the context of Industry 4.0. Some of the main issues in developing Machine Learning-based solutions in industrial environment are: (i) the lack of reliable tagged data; (ii) the complexity of data structures present in real-world scenarios. In this paper we investigate unsupervised anomaly detection for 2-dimensional data in manufacturing environment: we provide an approach that exploit Deep Learning-based architecture for handling the data at hand. We show the effectiveness of the proposed approach in a real world case study related to optical emission spectroscopy data in semiconductor manufacturing process providing satisfactory classification accuracy.

Predictive Maintenance technologies are particularly appealing for Industrial Equipment producers, as they pave the way to the selling of high added-value services and customized maintenance plans. However, standard Predictive Maintenance approaches assume the availability of sensor measurements, and the costs associated with adding sensors or remotely accessing sensor readings may discourage the development of such technologies. In this context, Alarm Forecasting can be very useful as it represents a low-cost alternative or helpful support to sensor-based Predictive Maintenance. In this work, we propose a new formulation for the Alarm Forecasting problem, framed as a multi-label classification task. We present a novel deep learning-based approach called FORMULA (alarm FORecasting in MUlti-LAbel setting). FORMULA leverages Transformer, a popular Neural Network architecture in the field of Natural Language Processing. To cope with alarm imbalance, we draw inspiration from Segmentation and Object Detection. Thus, FORMULA is trained by minimizing the Weighted Focal Loss, which turns out to be very effective in predicting rare alarms. These alarms, even if they are difficult to predict by nature, often are business-critical. We assess the proposed approach on a representative real-world problem from the packaging industry. In particular, we show that it outperforms not only classic multilabel techniques but also models based on recurrent neural networks. As regards the latter, the proposed approach also exhibits a lower computational burden, both in terms of training time and model size. To foster research in the field and reproducibility, we also publicly share the alarm logs dataset and the code used to perform the experiments.Note to Practitioners—This paper was motivated by the problem of monitoring equipment in the scenario of dairy products packaging, under the mild assumption that logs of the alarm generated by the packaging machines are available. This paper proposes an alarm forecasting algorithm. Its goal is to predict if any alarm will occur in the future, based only on past alarm logs. The limits of the considered future window can be defined arbitrarily, so there is enough time to perform corrective actions. Thus, the proposed approach aims to prevent unexpected downtime that would not only hinder productivity but also imply significant material waste. The proposed approach leverages methodologies from Natural Language Processing and Object Detection to deal with rare alarms that are often very informative in the industrial scenario. Besides, both the code and the real-world industrial datasets used to evaluate the methodology are available publicly. Currently, the proposed approach only uses alarm logs. Especially in the context of Industry 4.0, where many sensory data may be available, this is a limitation. Thus, the described approach might be extended by integrating alarm logs with sensory data. This integration is expected to improve the estimation of equipment health state. The results described in this paper may find application not only in the manufacturing sector but also in different areas such as Cyber Security, where log files keep activity records of each process performed.

Recent results show that features of adversarially trained networks for classification, in addition to being robust, enable desirable properties such as invertibility. The latter property may seem counter-intuitive as it is widely accepted by the community that classification models should only capture the minimal information (features) required for the task. Motivated by this discrepancy, we investigate the dual relationship between Adversarial Training and Information Theory. We show that the Adversarial Training can improve linear transferability to new tasks, from which arises a new trade-off between transferability of representations and accuracy on the source task. We validate our results employing robust networks trained on CIFAR-10, CIFAR-100 and ImageNet on several datasets. Moreover, we show that Adversarial Training reduces Fisher information of representations about the input and of the weights about the task, and we provide a theoretical argument which explains the invertibility of deterministic networks without violating the principle of minimality. Finally, we leverage our theoretical insights to remarkably improve the quality of reconstructed images through inversion.

We conduct an audit of pricing algorithms employed by companies in the Italian car insurance industry, primarily by gathering quotes through a popular comparison website. While acknowledging the complexity of the industry, we find evidence of several problematic practices. We show that birthplace and gender have a direct and sizeable impact on the prices quoted to drivers, despite national and international regulations against their use. Birthplace, in particular, is used quite frequently to the disadvantage of foreign-born drivers and drivers born in certain Italian cities. In extreme cases, a driver born in Laos may be charged 1,000 more than a driver born in Milan, all else being equal. For a subset of our sample, we collect quotes directly on a company website, where the direct influence of gender and birthplace is confirmed. Finally, we find that drivers with riskier profiles tend to see fewer quotes in the aggregator result pages, substantiating concerns of differential treatment raised in the past by Italian insurance regulators.

Abstract LEarning TO Rank (LETOR) is a research area in the field of Information Retrieval (IR) where machine learning models are employed to rank a set of items. In the past few years, neural LETOR approaches have become a competitive alternative to traditional ones like LambdaMART. However, neural architectures performance grew proportionally to their complexity and size. This can be an obstacle for their adoption in large-scale search systems where a model size impacts latency and update time. For this reason, we propose an architecture-agnostic approach based on a neural LETOR model to reduce the size of its input by up to 60% without affecting the system performance. This approach also allows to reduce a LETOR model complexity and, therefore, its training and inference time up to 50%.

This paper introduces two simple techniques to improve off-policy Reinforcement Learning (RL) algorithms. First, we formulate off-policy RL as a stochastic proximal point iteration. The target network plays the role of the variable of optimization and the value network computes the proximal operator. Second, we exploits the two value functions commonly employed in state-of-the-art off-policy algorithms to provide an improved action value estimate through bootstrapping with limited increase of computational resources. Further, we demonstrate significant performance improvement over state-of-the-art algorithms on standard continuous-control RL benchmarks.

Metrology, which plays an important role in ensuring production quality in modern manufacturing industries, incurs substantial costs both in terms of the infrastructure required and the time needed to perform measurements. In particular, in the semiconductor manufacturing industry, measuring fundamental quantities on different sites of a wafer surface is associated with increased production time. To increase metrology efficiency, a typical strategy is to limit the number of sites measured and to exploit statistical models (soft sensing) to reconstruct the wafer profile. Moreover, for quality reasons, spatial dynamic sampling strategies may be employed to ensure that all regions of a wafer surface are checked periodically during production. In this paper, we propose a new sampling strategy, called induced start dynamic sampling (ISDS), which adapts greedy feature selection algorithms to the spatial dynamic sampling problem, such that the number of measured sites at each process run is minimized while achieving good wafer profile reconstruction accuracy and process visibility. The superiority of the proposed strategy with respect to the state of the art is demonstrated using both simulated data and an industrial chemical vapor deposition case study.

In modern manufacturing scenarios, detecting anomalies in production systems is pivotal to keep high-quality standards and reduce costs. Even in the Industry 4.0 context, real-world monitoring systems are often simple and based on the use of multiple univariate control charts. Data-driven technologies offer a whole range of tools to perform multivariate data analysis that allow to implement more effective monitoring procedures. However, when dealing with complex data, common data-driven methods cannot be directly used, and a feature extraction phase must be employed. Feature extraction is a particularly critical operation, especially in anomaly detection tasks, and it is generally associated with information loss and low scalability. In this paper we consider the task of Anomaly Detection with two-dimensional, image-like input data, by adopting a Deep Learning-based monitoring procedure, that makes use of convolutional autoencoders. The procedure is tested on real Optical Emission Spectroscopy data, typical of semiconductor manufacturing. The results show that the proposed approach outperforms classical feature extraction procedures.

In the past recent years, Machine Learning methodologies have been applied in countless application areas. In particular, they play a key role in enabling Industry 4.0. However, one of the main obstacles to the diffusion of Machine Learning-based applications is related to the lack of interpretability of most of these methods. In this work, we propose an approach for defining a `feature importance' in Anomaly Detection problems. Anomaly Detection is an important Machine Learning task that has an enormous applicability in industrial scenarios. Indeed, it is extremely relevant for the purpose of quality monitoring. Moreover, it is often the first step towards the design of a Machine Learning-based smart monitoring solution because Anomaly Detection can be implemented without the need of labelled data. The proposed feature importance evaluation approach is designed for Isolation Forest, one of the most commonly used algorithm for Anomaly Detection. The efficacy of the proposed method is tested on synthetic and real industrial datasets.

Hand motion tracking traditionally requires highly complex and expensive systems in terms of energy and computational demands. A low-power, low-cost system could lead to a revolution in this field as it would not require complex hardware while representing an infrastructure-less ultra-miniature (∼ 100μm — [1]) solution. The present paper exploits the Multiple Point Tracking algorithm developed at the Tyndall National Institute as the basic algorithm to perform a series of gesture recognition tasks. The hardware relies upon the combination of a stereoscopic vision of two novel Lensless Smart Sensors (LSS) combined with IR filters and five hand-held LEDs to track. Tracking common gestures generates a six-gestures dataset, which is then employed to train three Machine Learning models: k-Nearest Neighbors, Support Vector Machine and Random Forest. An offline analysis highlights how different LEDs' positions on the hand affect the classification accuracy. The comparison shows how the Random Forest outperforms the other two models with a classification accuracy of 90–91 %.

In semiconductor manufacturing, metrology is generally a high cost nonvalue-added operation that significantly impacts on cycle time. As such, reducing wafer metrology continues to be a major target in semiconductor manufacturing efficiency initiatives. A novel data-driven spatial dynamic sampling methodology is presented that minimizes the number of sites that need to be measured across a wafer surface while maintaining an acceptable level of wafer profile reconstruction accuracy. The methodology is based on analyzing historical metrology data using forward selection component analysis (FSCA) to determine, from a set of candidate wafer sites, the minimum set of sites that need to be monitored in order to reconstruct the full wafer profile using statistical regression techniques. Dynamic sampling is then implemented by clustering unmeasured sites in accordance with their similarity to the FSCA selected sites and temporally selecting a different sample from each cluster. In this way, the risk of not detecting previously unseen process behavior is mitigated. We demonstrate the efficacy of the proposed methodology using both simulation studies and metrology data from a semiconductor manufacturing process.

In this paper an approach to deal with Predictive Maintenance (PdM) problems with time-series data is discussed. PdM is a important approach to tackle maintenance and it is gaining an increasing attention in advanced manufacturing to minimize scrap materials, downtime, and associated costs. PdM approaches are generally based on Machine Learning tools that require the availability of historical process and maintenance data. Given the exponential growth in data logging in modern equipment, time series dataset are increasingly available in PdM applications. To exploit time series data for PdM, a functional learning methodology, namely Supervised Aggregative Feature Extraction (SAFE), is here employed on a semiconductor manufacturing maintenance problem.

In many applications, and especially those where batch processes are involved, a target scalar output of interest is often dependent on one or more time series of data. With the exponential growth in data logging in modern industries, such time series are increasingly available for statistical modeling in soft sensing applications. In order to exploit time-series data for predictive modeling, it is necessary to summarize the information they contain as a set of features to use as model regressors. Typically this is done in an unsupervised fashion using simple techniques such as computing statistical moments, principal components or wavelet decompositions, often leading to significant information loss, and hence suboptimal predictive models. In this paper, a functional learning paradigm is exploited in a supervised fashion to derive continuous smooth estimates of time-series data (yielding aggregated local information), while simultaneously estimating a continuous shape function yielding optimal predictions. The proposed supervised aggregative feature extraction (SAFE) methodology can be extended to support nonlinear predictive models by embedding the functional learning framework in a reproducing kernel Hilbert spaces (RKHSs) setting. SAFE has a number of attractive features including closed-form solution and the ability to explicitly incorporate first- and second-order derivative information. Using simulation studies and a practical semiconductor manufacturing case study, we highlight the strengths of the new methodology with respect to standard unsupervised feature extraction approaches.

In this paper, a machine learning (ML) approach is presented that exploits accelerometers data to deal with gesture recognition (GR) problems. The proposed methodology aims at providing high accuracy classification for home automation systems, which are generally user independent, device independent, and device orientation independent, an heterogeneous scenario that has not been fully investigated in previous GR literature. The approach illustrated in this paper is composed of three main steps: event identification; feature extraction; and ML-based classification. The elements of the novelty of the proposed approach are 1) a preprocessing phase based on principal component analysis to increase the performance in real-world scenario conditions and 2) the development of parsimonious novel classification techniques based on sparse Bayesian learning. This methodology is tested on two datasets of four gesture classes (horizontal, vertical, circles, and eight-shaped movements) and on a further dataset with eight classes. In order to authentically describe a real-world home automation environment, the gesture movements are collected from more than 30 people who freely perform any gesture. It results in a dictionary of 12 and 20 different movements, respectively, in the case of the four-class and the eight-class databases.

In this paper, a multiple classifier machine learning (ML) methodology for predictive maintenance (PdM) is presented. PdM is a prominent strategy for dealing with maintenance issues given the increasing need to minimize downtime and associated costs. One of the challenges with PdM is generating the so-called “health factors,” or quantitative indicators, of the status of a system associated with a given maintenance issue, and determining their relationship to operating costs and failure risk. The proposed PdM methodology allows dynamical decision rules to be adopted for maintenance management, and can be used with high-dimensional and censored data problems. This is achieved by training multiple classification modules with different prediction horizons to provide different performance tradeoffs in terms of frequency of unexpected breaks and unexploited lifetime, and then employing this information in an operating cost-based maintenance decision system to minimize expected costs. The effectiveness of the methodology is demonstrated using a simulated example and a benchmark semiconductor manufacturing maintenance problem.

Smart management of maintenances has become fundamental in manufacturing environments in order to decrease downtime and costs associated with failures. Predictive Maintenance (PdM) systems based on Machine Learning (ML) techniques have the possibility with low added costs of drastically decrease failures-related expenses; given the increase of availability of data and capabilities of ML tools, PdM systems are becoming really popular, especially in semiconductor manufacturing. A PdM module based on Classification methods is presented here for the prediction of integral type faults that are related to machine usage and stress of equipment parts. The module has been applied to an important class of semiconductor processes, ion-implantation, for the prediction of ion-source tungsten filament breaks. The PdM has been tested on a real production dataset.

In semiconductor manufacturing plants, monitoring of all wafers is fundamental in order to maintain good yield and high quality standards. However, this is a costly approach and in practice only few wafers in a lot are actually monitored. With a Virtual Metrology (VM) system it is possible to partly overcome the lack of physical metrology. In a VM scheme, tool data are used to predict, for every wafer, metrology measurements. In this paper, we present a VM system for a Chemical Vapor Deposition (CVD) process. Various data mining techniques are proposed. Due to the huge fragmentation of data derived from CVD's mixed production, several kind of data clustering have been adopted. The proposed models have been tested on real productive industrial data sets.