This paper presents one of the first contributions in the context of Multimodal Predictive Process Monitoring (MM-PPM) . In recent years, Predictive Process Monitoring (PPM) has evolved at the intersection of process mining, machine learning, and data science, as organizations seek to anticipate the future course of ongoing processes. Traditional PPM mainly relies on structured event log data, but many real-world scenarios generate richer information, including text, images, audio, and video. MM-PPM promises to start addressing this rich data scenario by integrating complementary knowledge from heterogeneous modalities through modality-specific representations and information fusion techniques. The growing digitization of healthcare systems, combined with advances in Artificial Intelligence (AI), has accelerated AI-based PPM for analyzing sequences of clinical events, supporting decision-making, enabling personalized care, and improving clinical facility management. Given these characteristics, clinical pathways represent an ideal domain for experimenting with MM-PPM, as they may naturally involve diverse modalities such as structured records, free-text notes, or medical images. To handle multimodal information available with clinical pathways, we introduce MEDUSA , an MM-PPM approach for outcome prediction, which jointly processes medical image information coupled with the storytelling of structural records and text notes collected during the clinical pathway of a patient until the acquisition of the considered image. The evaluation of MEDUSA is done in a COVID-19 case study, to assess the performance of the proposed approach and explain how specific information within each modality influences the decisions of the predictive model.

Process discovery aims to automatically discover a process model to explain the behavior of event traces recorded in an event log during the execution of the activities of an underlying process. Several powerful process discovery algorithms are already formulated in process mining to identify regular control-flow structures in event logs and strike different trade-offs between the accuracy in capturing the behavior recorded in the event $\log$ and the complexity of the derived process model. This is commonly done under the assumption that log event traces are distributed according to the Pareto principle with a large portion of event traces held by a small fraction of top-frequent variants. However, the Pareto principle is not always satisfied in several real-life, complex processes. For example, the majority of log event traces produced in various healthcare or gaming processes is often spanned on a high number of top-frequent varianttraces. Various techniques (e.g. event filtering, trace extraction and trace abstraction) are already formulated in process mining to cope traditional process discovery algorithms also with event logs that do not conform the Pareto principle. Following this line of research, we explore the performance of a trace extraction method introduced to support the quest for Pareto-like event trace distribution during process discovery. The trace extraction is done resorting to a deep embedding representation of event traces, which sees traces at an abstraction level that removes noise and anomalous trace excerpt by enabling the discovery of simpler process models with higher accuracy. The deep embedding is used in combination with clustering. Experiments with several benchmark event logs show the effectiveness of the two proposed methods also compared to prior methods.

Abstract Artificial Intelligence (AI) is transforming cybersecurity practices thanks to the amazing accuracy performance achieved with several AI-based malware detection systems. However, several recent studies have shown that AI decision models can be vulnerable to adversarial attacks. In malware detection scenarios, adversarial attacks are realistic manipulations of existing malware, which preserve the executable and malicious behaviour but evade the malware detection measures. In this study, we consider Windows Portable Executable (PE) malware, which is currently trending to prominent malware types, and we show that counterfactual explanations can be used to drive the generation of realistic adversarial Windows PE malware to evade AI-based detection. In particular, the proposed method OLIVANDER works in a black-box manner, which is the most restrictive attack option, as the evasion method interacts with the target decision system to evade by merely knowing the model input and output. The evaluation study explores the effectiveness of the proposed evasion method in terms of evasion ability, efficiency of computation, and attack transferability compared to two state-of-the-art evasion methods. In addition, the performed evaluation accounts for performances on commercial anti-malware systems.

Forecasting future states of running process instances is one of the main challenges of Predictive Process Monitoring (PPM). Several deep learning approaches have recently achieved a valuable accuracy performance by addressing this task. On the other hand, with the recent boom of Large Language Models (LLMs) in multiple fields, LLMs have started attracting attention in PPM research also. In this study, we leverage the rich context of textual data to transform information recorded in event logs in smart textual data ready for boosting accurate PPM learning. In detail, we propose LUPIN, a LLM approach to predict the activity suffix of a running process instance. First it encodes historical running process instances in semantic text stories formulated according to narrative templates that account for information recorded in the event log. Then it fine tunes a pre-trained LLM model – medium BERT – on the text stories of historic running instances of a business process, to predict the activity suffix of any future running instance of the same business process. Finally, LUPIN integrates the XAI Integrated Gradient (IG) algorithm to explain how each part of the textual description of a running process instance has an effect on the prediction of its activity completion. The experimental evaluation explores the accuracy performance of LUPIN compared to that of several related methods and draws insights from the explanation retrieved through the IG algorithm.

Abstract Forest tree dieback inventory has a crucial role in improving forest management strategies. This inventory is traditionally performed by forests through laborious and time-consuming human assessment of individual trees. On the other hand, the large amount of Earth satellite data that are publicly available with the Copernicus program and can be processed through advanced deep learning techniques has recently been established as an alternative to field surveys for forest tree dieback tasks. However, to realize its full potential, deep learning requires a deep understanding of satellite data since the data collection and preparation steps are essential as the model development step. In this study, we explore the performance of a data-centric semantic segmentation approach to detect forest tree dieback events due to bark beetle infestation in satellite images. The proposed approach prepares a multisensor data set collected using both the SAR Sentinel-1 sensor and the optical Sentinel-2 sensor and uses this dataset to train a multisensor semantic segmentation model. The evaluation shows the effectiveness of the proposed approach in a real inventory case study that regards non-overlapping forest scenes from the Northeast of France acquired in October 2018. The selected scenes host bark beetle infestation hotspots of different sizes, which originate from the mass reproduction of the bark beetle in the 2018 infestation.

This study describes a new cyber-threat detection method, named PANACEA, that uses Ensemble Deep Learning coupled with Adversarial Training and XAI, to gain accuracy with neural models trained in cybersecurity problems.

Abstract The Italian Public Administration (PA) relies on costly manual analyses to ensure the GDPR compliance of public documents and secure personal data. Despite recent advances in Artificial Intelligence (AI) have benefited many legal fields, the automation of workflows for data protection of public documents is still only marginally affected. The main aim of this work is to design a framework that can be effectively adopted to check whether PA documents written in Italian meet the GDPR requirements. The main outcome of our interdisciplinary research is INTREPID (art ficial i elligence for gdp complianc of ublic adm nistration ocuments), an AI-based framework that can help the Italian PA to ensure GDPR compliance of public documents. INTREPID is realized by tuning some linguistic resources for Italian language processing (i.e. SpaCy and Tint) to the GDPR intelligence. In addition, we set the foundations for a text classification methodology to recognise the public documents published by the Italian PA, which perform data breaches. We show the effectiveness of the framework over a text corpus of public documents that were published online by the Italian PA. We also perform an inter-annotator study and analyse the agreement of the annotation predictions of the proposed methodology with the annotations by domain experts. Finally, we evaluate the accuracy of the proposed text classification model in detecting breaches of security.

Abstract Various applications in remote sensing demand automatic detection of changes in optical satellite images of the same scene acquired over time. This paper investigates how to leverage autoencoders in change vector analysis, in order to better delineate possible changes in a couple of co-registered, optical satellite images. Let us consider both a primary image and a secondary image acquired over time in the same scene. First an autoencoder artificial neural network is trained on the primary image. Then the reconstruction of both images is restored via the trained autoencoder so that the spectral angle distance can be computed pixelwise on the reconstructed data vectors. Finally, a threshold algorithm is used to automatically separate the foreground changed pixels from the unchanged background. The assessment of the proposed method is performed in three couples of benchmark hyperspectral images using different criteria, such as overall accuracy, missed alarms and false alarms. In addition, the method supplies promising results in the analysis of a couple of multispectral images of the burned area in the Majella National Park (Italy).

Predictive process monitoring (PPM) techniques have become a key element in both public and private organizations by enabling crucial operational support of their business processes. Thanks to the availability of large amounts of data, different solutions based on machine and deep learning have been proposed in the literature for the monitoring of process instances. These state-of-the-art approaches leverage accuracy as main objective of the predictive modeling, while they often neglect the interpretability of the model. Recent studies have addressed the problem of interpretability of predictive models leading to the emerging area of Explainable AI (XAI). In an attempt to bring XAI in PPM, in this paper we propose a fully interpretable model for outcome prediction. The proposed method is based on a set of fuzzy rules acquired from event data via the training of a neuro-fuzzy network. This solution provides a good trade-off between accuracy and interpretability of the predictive model. Experimental results on different benchmark event logs are encouraging and motivate the importance to develop explainable models for predictive process analytics.

Abstract Saliency detection mimics the natural visual attention mechanism that identifies an imagery region to be salient when it attracts visual attention more than the background. This image analysis task covers many important applications in several fields such as military science, ocean research, resources exploration, disaster and land-use monitoring tasks. Despite hundreds of models have been proposed for saliency detection in colour images, there is still a large room for improving saliency detection performances in hyperspectral imaging analysis. In the present study, an ensemble learning methodology for saliency detection in hyperspectral imagery datasets is presented. It enhances saliency assignments yielded through a robust colour-based technique with new saliency information extracted by taking advantage of the abundance of spectral information on multiple hyperspectral images. The experiments performed with the proposed methodology provide encouraging results, also compared to several competitors.

Predictive process monitoring has recently become one of the main enablers of data-driven insights in process mining. As an application of predictive analytics, process prediction is mainly concerned with predicting the evolution of running traces based on models extracted from historical event logs. This paper presents a process mining approach, which uses convolutional neural networks to equip the execution scenario of a business process with a means to predict the next activity in a running trace. The basic idea is to convert the temporal data enclosed in the historical event log of a business process into spatial data so as to treat them as images. To this purpose, every trace of the event log is first transformed into the set of its prefix traces (i.e. sequences of events that represent the prefix of a trace). These prefix traces are mapped into 2D image-like data structures. Created spatial data are finally used to train a Convolutional Neural Network, in order to learn a deep learning model capable to predict the next activity (i.e. the activity associated to the event occurring after the last event in the considered prefix trace). This predictive deep model can be employed as a powerful service to support participants in performing business processes since it guarantees a higher utilization by acting proactively in anticipation. Preliminary tests with two benchmark logs are carried out to investigate the viability of the proposed approach.

Managing geophysical data generated by emerging spatiotemporal data sources (e.g. geosensor networks) presents a growing challenge to Geographic Information System science. The presence of correlation poses difficulties with respect to traditional spatial data analysis. This paper describes a novel spatiotemporal analytical scheme that allows us to yield a characterization of correlation in geophysical data along the spatial and temporal dimensions. We resort to a multivariate statistical model, namely CoKriging, in order to derive accurate spatiotemporal interpolation models. These predict unknown data by utilizing not only their own geosensor values at the same time, but also information from near past data. We use a window-based computation methodology that leverages the power of temporal correlation in a spatial modeling phase. This is done by also fitting the computed interpolation model to data which may change over time. In an assessment, using various geophysical data sets, we show that the presented algorithm is often able to deal with both spatial and temporal correlations. This helps to gain accuracy during the interpolation phase, compared to spatial and spatiotemporal competitors. Specifically, we evaluate the efficacy of the interpolation phase by using established machine-learning metrics (i.e. root mean squared error, Akaike information criterion and computation time).

In many real-time applications, such as wireless sensor network monitoring, traffic control or health monitoring systems, it is required to analyze continuous and unbounded geographically distributed streams of data (e.g. temperature or humidity measurements transmitted by sensors of weather stations). Storing and querying geo-referenced stream data poses specific challenges both in time (real-time processing) and in space (limited storage capacity). Summarization algorithms can be used to reduce the amount of data to be permanently stored into a data warehouse without losing information for further subsequent analysis. In this paper we present a framework in which data streams are seen as time-varying realizations of stochastic processes. Signal compression techniques, based on transformed domains, are applied and compared with a geometrical segmentation in terms of compression efficiency and accuracy in the subsequent reconstruction.

The automatic discovery of process models can help to gain insight into various perspectives (e.g., control flow or data perspective) of the process executions traced in an event log. Frequent patterns mining offers a means to build human understandable representations of these process models. This paper describes the application of a multi-relational method of frequent pattern discovery into process mining. Multi-relational data mining is demanded for the variety of activities and actors involved in the process executions traced in an event log which leads to a relational (or structural) representation of the process executions. Peculiarity of this work is in the integration of disjunctive forms into relational patterns discovered from event logs. The introduction of disjunctive forms enables relational patterns to express frequent variants of process models. The effectiveness of using relational patterns with disjunctions to describe process models with variants is assessed on real logs of process executions.