Many Graph Neural Networks (GNNs) in the literature are based on message-passing, which introduces a strong learning bias that may fail to capture critical relational information encoded in the edges of the graph, particularly in tasks where the structural role of edges is as significant as that of nodes, such as in chemical molecular analysis or social network dynamics.We propose a novel architecture inspired by line graph theory that explicitly models edge adjacencies, iteratively transforming a graph into its corresponding line graph.Differently from message-passing, the iterative application of this transformation enables the exchange of information among non-adjacent nodes, allowing for the capture of complex topological dependencies, which standard GNNs overlook.Experiments on standard benchmarks show promising results.

Designing molecules with specific attributes is vital in drug discovery and materials science.Ring structures are key to a molecule's stability, reactivity, and biological interactions, ensuring that designed compounds are both feasible and synthetically viable, thereby increasing their potential for lab production and therapeutic use.Generative diffusion models have become essential tools for in silico molecule generation.However, integrating structural constraints, especially those involving ring structures, remains challenging.This study introduces a method for applying hard ring-related constraints in molecule generation, enhancing synthetic validity and utility, with evaluations on the QM9 dataset.

Machine Learning (ML) has become a central force in Artificial Intelligence, driving major breakthroughs in applications that handle increasingly complex data, from images and text sequences to graph structures. While new architectures such as Transformers and Graph Neural Networks continue to redefine performance benchmarks in various domains, these predominantly data-driven methods often neglect critical domain knowledge, practical constraints, and broader contextual factors. This oversight diminishes their trustworthiness and restricts their impact in real-world settings. In this paper, we discuss the need for a more informed approach to ML for complex data. Specifically, we advocate for solutions that explicitly integrate structural awareness to capture underlying relationships in the data, incorporate key technical requirements to ensure safety and compliance with industry standards, embed environmental considerations to promote sustainability and resource efficiency, adhere to established physical principles, and uphold ethical and societal values. By weaving these dimensions together, informed ML can bridge the gap between purely data-centric methods and the nuanced demands of practical applications. We show how this integrated framework not only strengthens model performance but also ensures that ML solutions remain trustworthy, efficient, and sensitive to human ecological, ethical, and regulatory imperatives. Our discussion underscores the transformative potential of Informed ML to drive innovation across diverse domains, setting a new benchmark for responsible and high-impact ML system design.

Abstract The framework of Backpropagation-Free Graph Neural Networks (BF-GNNs) enables local learning at the neuron level in GNNs. While BF-GNNs can match the performance of their backpropagation-based counterparts, they may develop redundant internal representations that limit further gains. To address this issue, we propose an innovative architecture dubbed Boosting-based Backpropagation-Free GNN (B 3 F-GNN), where each network module contains multiple backpropagation-free neurons trained locally and combined as a classifier. Within each layer, later modules exploit error signals from earlier trained modules to refine predictions by diversifying internal representations. We implement this approach with two complementary boosting strategies: sample reweighting, in the spirit of AdaBoost, and error-guided prototype selection for gating, which concentrates non-linearities where previous modules struggled. The modular design also enables any-time incremental training by adding more modules on demand within resource constraints. An ablation study and in-depth experimental analysis show that both strategies reduce redundancy and increase specialization, leading to statistically significant accuracy improvements over backpropagation-based counterparts on standard node-classification benchmarks.

Functional Magnetic Resonance Imaging (fMRI) provides spatio-temporal maps of brain activity; however, extracting the rich information they contain is challenging. Traditional approaches use only summary statistics, losing details that might be hidden in the complex temporal dynamics. Deep neural networks are emerging as an apt solution in this context, given their ability to handle vast amounts of structured data. In this paper, we consider two widely studied fMRI datasets: the Human Connectome Project for connectome fingerprinting, and ABIDE for autism classification. We aim to understand how handling the temporal and spatial dimensions could influence the performance of the models and their interpretability. Specifically, we compare neural network models with architectural biases toward temporal, spatial, or combined spatio-temporal features. The results of our analysis show that existing methods exploiting the spatial dimension, or spatio-temporal hybrids, are not competitive with simpler ones considering the temporal dimension only, such as LSTM. Additionally, we propose a contrastive learning approach for connectome fingerprinting, enabling robust individual identification without requiring access to all subjects during training. Our findings suggest that explicit graph modeling of the interaction between brain regions introduces complexity without improving performance, thereby challenging current trends.

We consider the setting of online/streaming continual learning. In particular, we analyze the implications of the consolidated continual learning evaluation setting in an online scenario. As the data stream potentially extends infinitely, the evaluation set also grows boundlessly, necessitating a redefinition of model evaluation. In the streaming learning literature, the model is usually evaluated with the test-then-train method that does not require a separate evaluation set at all, or with continuous reevaluation that considers partially delayed labels [1]. In this work, we propose to model the data-generating process as random walks on concept graphs, i.e. graphs that define the possible transitions between the concepts to learn. This offers a precise formalization of the problem of learning on (possibly) infinitely long data streams with concept drifts and recurring concepts. In such a setting, we also discuss alternative possibilities that blend the continual learning setting with the traditional online learning one. We propose a framework that generalizes the commonly considered online continual learning scenario (i.e. corresponds to a specific topology of the concept graph). Finally, we discuss how different topologies of data streams, derived from our framework, could inspire new research directions in continual learning.

Post-hoc methods in Explainable AI (XAI) elucidate black-box models by identifying input features critical to the model's decision-making. Recent advancements in these methods have facilitated the generation of logic-based explanations that capture interactions among input features. However, these techniques often encounter critical limitations, notably the inability to ensure logical consistency and fidelity between generated explanations and the model's actual decision-making processes. Such inconsistencies jeopardize the reliability of explanations particularly in high-risk domains. To address this gap, we introduce a novel, theoretically rigorous approach rooted in category theory. Specifically, we propose the concept of an explaining functor, which preserves logical entailment structurally between the explanations and the decisions of black-box models. By establishing a categorical framework, our method guarantees the coherence and accuracy of extracted explanations, thus overcoming the common pitfalls associated with heuristic-based explanation methods. We demonstrate the practical efficacy of our theoretical contributions through two synthetic benchmarks that highlight significant reductions in contradictory and unfaithful explanations. Our experiments show how our framework can provide mathematically grounded, compositional, and coherent explanations.

Abstract Identifying molecules that exhibit some pre-specified properties is a difficult problem to solve. In the last few years, deep generative models have been used for molecule generation. Deep Graph Variational Autoencoders are among the most powerful machine learning tools with which it is possible to address this problem. However, existing methods struggle to capture the true data distribution and tend to be computationally expensive. In this work, we propose RGCVAE, an efficient and effective Graph Variational Autoencoder based on: (i) an encoding network exploiting a new powerful Relational Graph Isomorphism Network; (ii) a novel probabilistic decoding component. Compared to several State-of-the-Art VAE methods on two widely adopted datasets, RGCVAE shows State-of-the-Art molecule generation performance while being significantly faster to train. The Python code implementing RGCVAE is openly accessible for download at: https://github.com/drigoni/RGCVAE .

Neural networks find applications in many safety-critical systems that raise concerns about their deployment: Are we sure the network will never advise doing anything violating a set of safety constraints? Formal verification has been recently applied to prove whether an existing neural network is certified for some property (i.e., if it satisfies the property for all possible inputs) or not. Formal verification can prove that a network respects the property, but cannot fix a network that does not respect it. In this paper we focus on the automated synthesis of certified neural networks, that is, on how to automatically build a network that is guaranteed to respect some required properties. We exploit a Counter Example Guided Inductive Synthesis (CEGIS) loop that alternates Deep Learning, Formal Verification, and a novel data generation technique that augments the training data to synthesize certified networks in a fully automatic way. An application of a proof-of-concept implementation of the framework shows the feasibility of the approach.

Backpropagation-Free Graph Convolutional Networks (BF-GCN) are backpropagation-free neural models dealing with graph data based on Gated Linear Networks.Each neuron in a BF-GCN is defined as a set of graph convolution filters (weight vectors) and a gating mechanism that, given a node's context, selects the weight vector to use for processing the node's attributes based on its distance from a set of prototypes.Given the higher expressivity BF-GNN's neurons compared to the standard graph convolutional neural networks' ones, they show bigger memory footprint.In this paper, we explore how reducing the size of node contexts through randomization can reduce the memory occupancy of the method, enabling its application to huge datasets.We empirically show how working with very low dimensional contexts does not impact the resulting predictive performances.* We acknowledge the support of the projects: "Future AI Research (FAIR) -Spoke 2 Integrative AI -Symbolic conditioning of Graph Generative Models (SymboliG)" funded by the European Union under the National Recovery and Resilience Plan (NRRP), Mission 4 Component 2 Investment 1.3 -Call for tender No. 341 of March 15, 2022 of Italian Ministry of University and Research -NextGenerationEU, Code PE0000013, Concession Decree No. 1555 of October 11, 2022 CUP C63C22000770006; "iNEST: Interconnected Nord-Est Innovation Ecosystem" funded under the NRRP, Mission 4 Component 2 Investment 1.5 -Call for tender No. 3277 of 30 December 2021 of Italian Ministry of University and Research -NextGenerationEU, Code ECS00000043, Concession Decree No. 1058 of June 23, 2022, CUP C43C22000340006; the PON R&I 2014-2020 project Smart Waste Treatment founded by the FSE REAC-EU; the project "Lifelong Learning on large-scale and structured data"

In the contemporary era of data-driven decision-making, the application of Machine Learning (ML) on complex data (e.g., images, text, sequences, trees, and graphs) has become increasingly pivotal (e.g., Large Language Models and Graph Neural Networks).In this context, there is a gap between purely data-driven models and domain-specific knowledge, requirements, and expertise.In particular, this domain specificity needs to be integrated into the ML models to improve learning generalization, sustainability, trustworthiness, reliability, security, and safety.This additional knowledge can assume different forms, e.g.: software developers require ML to comply with many technical requirements, companies require ML to comply with economic and environmental sustainability, domain experts require ML to be aligned with physical and logical laws, and society requires ML to be aligned with ethical principles.This special session gathers valuable contributions and early findings in the field of Informed ML for Complex Data.Our main objective is to showcase the potential and limitations of new ideas, improvements, or the blending of ML and other research areas in solving real-world problems.

The recent outbreak of COVID-19 has spurred global collaborative research efforts to model and forecast the disease to improve preparation and control. Epidemiological models integrate experimental data and expert opinions to understand infection dynamics and control measures. Classical Machine Learning techniques often face challenges such as high data requirements, lack of interpretability, and difficulty integrating domain knowledge. A potential solution is to leverage Physically-Informed Machine Learning (PIML) models, which enhance models by incorporating known physical properties of viral spread. Additionally, epidemiological datasets are best represented as graphs, facilitating the modelling of interactions between individuals. In this paper, we propose a novel, interpretable graph-based PIML technique called SINDy-Graph to model infectious disease dynamics. Our approach is a Graph Cellular Automata architecture that combines the ability to identify dynamics for discovering the differential equations governing the physical phenomena under study using graphs modelling relationships between nodes (individuals). The experimental results demonstrate that integrating domain knowledge ensures better physical plausibility. In addition, our proposed model is easier to train and achieves a lower generalisation error compared to other baseline methods.

Recommender systems are traditionally optimized to facilitate content discovery for consumers by ranking items based on predicted relevance. As such, these systems often do not consider the varying profitability of items for service providers. Since the purpose of recommender systems is usually to create value for both consumers and providers, we hypothesize that integrating profit awareness into recommender systems, considering both consumer relevance and provider profitability, can enhance recommendation outcomes from a provider point of view. In this study, we design and evaluate novel modeling approaches for different families of collaborative filtering algorithms to overcome the existing limitations of conventional reranking methods. Specifically, we show how to embed the business value perspective directly into the loss functions during model training. Through empirical evaluations on three datasets, we show that our proposed models effectively generate recommendations that balance profitability and relevance. Overall, our analyses indicate that these models offer a promising alternative to traditional reranking approaches, particularly because they exhibit improved efficiency during prediction.

In this paper, we investigate neural models based on graph random features for classification tasks. First, we aim to understand when over parameterization, namely generating more features than the ones necessary to interpolate, may be beneficial for the generalization abilities of the resulting models. We employ two measures: one from the algorithmic stability framework and another one based on information theory. We provide empirical evidence from several commonly adopted graph datasets showing that the considered measures, even without considering task labels, can be effective for this purpose. Additionally, we investigate whether these measures can aid in the process of hyperparameters selection. The results of our empirical analysis show that the considered measures have good correlations with the estimated generalization performance of the models with different hyperparameter configurations. Moreover, they can be used to identify good hyperparameters, achieving results comparable to the ones obtained with a classic grid search.

The increasing complexity and amount of data available in modern applications strongly demand Trustworthy Learning algorithms that can be fed directly with complex and large graphs data. In fact, on one hand, machine learning models must meet high technical standards (e.g., high accuracy with limited computational requirements), but, at the same time, they must be sure not to discriminate against subgroups of the population (e.g., based on gender or ethnicity). Graph Neural Networks (GNNs) are currently the most effective solution to meet the technical requirements, even if it has been demonstrated that they inherit and amplify the biases contained in the data as a reflection of societal inequities. In fact, when dealing with graph data, these biases can be hidden not only in the node attributes but also in the connections between entities. Several Fair GNNs have been proposed in the literature, with uNIfying Fairness and stabiliTY (NIFTY) (Agarwal et al., 2021) being one of the most effective. In this paper, we will empower NIFTY’s fairness with two new strategies. The first one is a Biased Edge Dropout, namely, we drop graph edges to balance homophilous and heterophilous sensitive connections, mitigating the bias induced by subgroup node cardinality. The second one is Attributes Preprocessing, which is the process of learning a fair transformation of the original node attributes. The effectiveness of our proposal will be tested on a series of datasets with increasingly challenging scenarios. These scenarios will deal with different levels of knowledge about the entire graph, i.e., how many portions of the graph are known and which sub-portion is labelled at the training and forward phases.

Abstract We propose a framework for the definition of neural models for graphs that do not rely on backpropagation for training, thus making learning more biologically plausible and amenable to parallel implementation. Our proposed framework is inspired by Gated Linear Networks and allows the adoption of multiple graph convolutions. Specifically, each neuron is defined as a set of graph convolution filters (weight vectors) and a gating mechanism that, given a node and its topological context, generates the weight vector to use for processing the node’s attributes. Two different graph processing schemes are studied, i.e., a message-passing aggregation scheme where the gating mechanism is embedded directly into the graph convolution, and a multi-resolution one where neighboring nodes at different topological distances are jointly processed by a single graph convolution layer. We also compare the effectiveness of different alternatives for defining the context function of a node, i.e., based on hyperplanes or on prototypes, and using a soft or hard-gating mechanism. We propose a unified theoretical framework allowing us to theoretically characterize the proposed models’ expressiveness. We experimentally evaluate our backpropagation-free graph convolutional neural models on commonly adopted node classification datasets and show competitive performances compared to the backpropagation-based counterparts.

Many neural networks for graphs are based on the graph convolution (GC) operator, proposed more than a decade ago. Since then, many alternative definitions have been proposed, which tend to add complexity (and nonlinearity) to the model. Recently, however, a simplified GC operator, dubbed simple graph convolution (SGC), which aims to remove nonlinearities was proposed. Motivated by the good results reached by this simpler model, in this article we propose, analyze, and compare simple graph convolution operators of increasing complexity that rely on linear transformations or controlled nonlinearities, and that can be implemented in single-layer graph convolutional networks (GCNs). Their computational expressiveness is characterized as well. We show that the predictive performance of the proposed GC operators is competitive with the ones of other widely adopted models on the considered node classification benchmark datasets.

In a broad range of real-world machine learning applications, representing examples as graphs is crucial to avoid a loss of information.For this reason, in the last few years, the definition of machine learning methods, particularly neural networks, for graph-structured inputs has been gaining increasing attention.In particular, Deep Graph Networks (DGNs) are nowadays the most commonly adopted models to learn a representation that can be used to address different tasks related to nodes, edges, or even entire graphs.This tutorial paper reviews fundamental concepts and open challenges of graph representation learning and summarizes the contributions that have been accepted for publication to the ESANN 2023 special session on the topic.

In this paper, we investigate neural models based on graph random features.In particular, we aim to understand when over-parameterization, namely generating more features than the ones necessary to interpolate, may be beneficial for the generalization of the resulting models.Exploiting the algorithmic stability framework and based on empirical evidences from several commonly adopted graph datasets, we will shed some light on this issue.

Predictive Process Analytics is becoming an essential aid for organizations, providing online operational support of their processes. However, process stakeholders need to be provided with an explanation of the reasons why a given process execution is predicted to behave in a certain way. Otherwise, they will be unlikely to trust the predictive monitoring technology and, hence, adopt it. This paper proposes a predictive analytics framework that is also equipped with explanation capabilities based on the game theory of Shapley Values . The framework has been implemented in the IBM Process Mining suite and commercialized for business users. The framework has been tested on real-life event data to assess the quality of the predictions and the corresponding evaluations. In particular, a user evaluation has been performed in order to understand if the explanations provided by the system were intelligible to process stakeholders.

Research on recommender systems (RSs) has traditionally focused on the design of systems capable of suggesting items of interest for users. However, often the most important expectation for RSs used in commercial applications is to improve the business performance of the organization. For this reason, alongside the growth of e-business, we have witnessed growing interest in value-aware RSs that, unlike traditional RSs, are designed to optimize the economic value of recommendations by considering the objectives of multiple stakeholders. In this paper, we provide a systematic literature review, following the PRISMA guidelines, specialized in value-aware RSs. We explore key commercial applications, main algorithms, value categories typically optimized, and the most commonly used datasets. Furthermore, we note limitations of the state-of-the-art approaches and identify future research directions.

Recommender systems, in real-world circumstances, tend to limit user exposure to certain topics and to overexpose them to others to maximize performance. However, repeated exposure to biased content could lead to the so-called echo chamber phenomenon: especially in social network environments, people encounter only information that reflects their previous beliefs and opinions, reinforcing them. This phenomenon could have worrying consequences for society, including the spread of aggressive, unhealthy, or risky behaviors. Some persons can be more affected than others by echo-chambers. We define as sensitive the users whose behavior could be influenced by the over- or under-exposure to certain items due to the echo-chamber effect, and as influential the items that could influence the behavior of such users. In this paper, we address the problem of recommending influential items to sensitive users. We formalize the problem and propose three techniques that can be used to diversify the distributions of influential items in order to positively affect sensitive users’ behavior. Recommendations that meet this diversity criterion could potentially avoid dangerous societal consequences and simultaneously promote healthier lifestyles. We tested the proposed techniques in a real-world dataset by considering two different case studies that involved potentially aggressive and potentially depressed users. All techniques have been proven to be effective and allow high performance to be maintained while diversifying recommendations.

The flourishing field of deep learning for graphs relies on the layered computation of representations from graph-structured input data.Message passing is the most common strategy for such processing of graphs, based on an efficient information exchange among the connected nodes via a local and iterative procedure.Representations learned in this way can be used to address different tasks related to nodes, edges, or even entire graphs.This tutorial paper reviews fundamental concepts and open challenges of deep learning for graphs and summarizes the contributions that have been accepted for publication to the ESANN 2022 special session on the topic.

Graph Neural Networks (GNNs) are nowadays widely used in many real-world applications.Nonetheless, the data relationships can be a source of biases based on sensitive attributes (e.g., gender or ethnicity).Several methods have been proposed to learn fair graph node representations.In this work we extend NIFTY, an approach that exploits additional terms in the loss function based on perturbing the input data to enforce the fairness of the GNNs.In particular, we exploit a biased perturbation of the adjacency matrix of the graph able to reduce the edge homophily.We show the effectiveness of our approach in four real-world graph datasets.

We propose a class of neural models for graphs that do not rely on backpropagation for training, thus making learning more biologically plausible and amenable to parallel implementation in hardware. The base component of our architecture is a generalization of Gated Linear Networks which allows the adoption of multiple graph convolutions. Specifically, each neuron is defined as a set of graph convolution filters (weight vectors) and a gating mechanism that, given a node and its topological context, selects the weight vector to use for processing the node’s attributes. Two different graph processing schemes are studied, i.e., a message-passing aggregation scheme where the gating mechanism is embedded directly into the graph convolution, and a multi-resolution one where neighbouring nodes at different topological distances are jointly processed by a single graph convolution layer. We also compare the effectiveness of different alternatives for defining the context function of a node, i.e., based on hyper-planes or on prototypes. A theoretical result on the expressiveness of the proposed models is also reported. We experimented our backpropagation-free graph convolutional neural architectures on commonly adopted node classification datasets, and show competitive performances compared to the backpropagation-based counterparts.

The identification of emotions conveyed by faces as genuine or not is a topic under-explored. While some controversial insights are available for happiness (where genuine happiness is supposed to create crow's feet around the eyes), nothing is known regarding other emotions. This topic is important as human beings are known to perform around the chance level to identify genuine emotions. It is thus pivotal to identify the relevant features for the correct identification of a facial emotion as genuine. To this aim, we capitalized on explainable artificial intelligence (XAI), characterized by a superior ability to detect subtle patterns in the data. Two different XAI models were created and applied to a dataset including 50 participants displaying both genuine and not genuine emotions. Results revealed that ML models achieved high accuracies in genuineness discrimination (78 % of average) while being robust and interpretable. The robustness of the results, ensured by their stability across different models and experimental conditions, is critical to improving the generalizability of the results. Our XAI algorithms provided interpretable results as they correctly identified facial muscle movements that are critical for the classification of emotions as genuine or fake.

Handwriting verification is an important task in the legal framework where the paternity of handwritten wills or con-tracts needs to be ascertained. Due to the lack of automated tools, the court currently relies exclusively on handwriting experts, who often show a wide degree of uncertainty in their responses. Machine learning models, and in particular artificial Neural Networks (NNs) might be a valuable aid to experts by providing an objective and automated instrument for handwriting analysis, especially when expert witnesses are undecided. However, at the state of the art there is a scarcity of datasets which are suitable for training NNs for the handwriting verification task, preventing the development of models accurate enough to be introduced into forensic practice. In this paper, a dataset of 3320 genuine and 3320 forged handwritten samples from 166 subjects is presented, considering two scenarios: copy (imitation of handwriting maintaining the same text, 3320 samples) and spontaneous production (imitation of handwriting generating a new text, 3320 samples). We provide baseline results for deep siamese convolutional neural networks, that are deep learning models widely adopted in similar tasks. In the proposed dataset, such NNs were able to reach accuracies over 75% in identifying forged samples in the copy scenario and of almost 82% in the spontaneous production scenario. Finally, a sample of 550 humans were tested in the same classification task. The experiments show that NNs perform significantly better than humans in the spontaneous production scenario, which is more complex than the copy one. We publicly release our dataset to encourage the future development of advanced Deep Learning models for the forged handwriting verification task.

In this paper, we consider the recently proposed family of continual learning models, called Gated Linear Networks (GLNs), and study two crucial aspects impacting on the amount of catastrophic forgetting affecting gated linear networks, namely, data standardization and gating mechanism. Data standardization is particularly challenging in the online/continual learning setting because data from future tasks is not available beforehand. The results obtained using an online standardization method show a considerably higher amount of forgetting compared to an offline -static- standardization. Interestingly, with the latter standardization, we observe that GLNs show almost no forgetting on the considered benchmark datasets. Secondly, for an effective GLNs, it is essential to tailor the hyperparameters of the gating mechanism to the data distribution. In this paper, we propose a gating strategy based on a set of prototypes and the resulting Voronoi tessellation. The experimental assessment shows that the proposed approach is more robust to different data standardizations compared to the original one, based on a halfspace gating mechanism, and shows improved predictive performance.

Learning machines for structured data (e.g., trees) are intrinsically based on their capacity to learn representations by aggregating information from the multi-way relationships emerging from the structure topology. While complex aggregation functions are desirable in this context to increase the expressiveness of the learned representations, the modelling of higher-order interactions among structure constituents is unfeasible, in practice, due to the exponential number of parameters required. Therefore, the common approach is to define models which rely only on first-order interactions among structure constituents.In this work, we leverage tensors theory to define a framework for learning in structured domains. Such a framework is built on the observation that more expressive models require a tensor parameterisation. This observation is the stepping stone for the application of tensor decompositions in the context of recursive models. From this point of view, the advantage of using tensor decompositions is twofold since it allows limiting the number of model parameters while injecting inductive biases that do not ignore higher-order interactions.We apply the proposed framework on probabilistic and neural models for structured data, defining different models which leverage tensor decompositions. The experimental validation clearly shows the advantage of these models compared to first-order and full-tensorial models.

Abstract Graph property prediction is becoming more and more popular due to the increasing availability of scientific and social data naturally represented in a graph form. Because of that, many researchers are focusing on the development of improved graph neural network models. One of the main components of a graph neural network is the aggregation operator, needed to generate a graph-level representation from a set of node-level embeddings. The aggregation operator is critical since it should, in principle, provide a representation of the graph that is isomorphism invariant, i.e. the graph representation should be a function of graph nodes treated as a set. DeepSets (in: Advances in neural information processing systems, pp 3391–3401, 2017) provides a framework to construct a set-aggregation operator with universal approximation properties. In this paper, we propose a DeepSets aggregation operator, based on Self-Organizing Maps (SOM), to transform a set of node-level representations into a single graph-level one. The adoption of SOMs allows to compute node representations that embed the information about their mutual similarity. Experimental results on several real-world datasets show that our proposed approach achieves improved predictive performance compared to the commonly adopted sum aggregation and many state-of-the-art graph neural network architectures in the literature.

The papers in this special section focus on the interaction with artificial intelligence (AI) systems using human-centered applications. AI methods are being applied to numerous areas, including medicine, security, transportation, industry, smart homes and cities, business, social sciences, and psychology. AI is currently a part of our daily lives. People interact continuously with AI: it is inside houses, computers, mobile phones, and applications. AI can make predictions and give suggestions for movies, songs, or future purchases based on our previous choices. It affects the society and economy. People are fascinated by AI in the ways it improves and facilitates human life (improving health care and discharging workers from heavy or dangerous jobs). People are also concerned with AI’s implementation risks, such as ethical, security, and privacy issues. There are also concerns that AI machines may replace humans in various activities. AI researchers and practitioners have been facing these issues and further research is needed to design technical and regulatory applicable solutions. This special issue (SI) investigates a broad range of issues deriving from human interaction with AI. We encouraged interdisciplinary and multidisciplinary contributions toward understanding how AI could improve human life in various fields.

Graph neural networks are receiving increasing attention as state-of-the-art methods to process graph-structured data. However, similar to other neural networks, they tend to suffer from a high computational cost to perform training. Reservoir computing (RC) is an effective way to define neural networks that are very efficient to train, often obtaining comparable predictive performance with respect to the fully trained counterparts. Different proposals of reservoir graph neural networks have been proposed in the literature. However, their predictive performances are still slightly below the ones of fully trained graph neural networks on many benchmark datasets, arguably because of the oversmoothing problem that arises when iterating over the graph structure in the reservoir computation. In this work, we aim to reduce this gap defining a multiresolution reservoir graph neural network (MRGNN) inspired by graph spectral filtering. Instead of iterating on the nonlinearity in the reservoir and using a shallow readout function, we aim to generate an explicit k -hop unsupervised graph representation amenable for further, possibly nonlinear, processing. Experiments on several datasets from various application areas show that our approach is extremely fast and it achieves in most of the cases comparable or even higher results with respect to state-of-the-art approaches.

Machine Learning (ML) achievements enabled automatic extraction of actionable information from data in a wide range of decisionmaking scenarios.This demands for improving both ML technical aspects (e.g., design and automation) and human-related metrics (e.g., fairness, robustness, privacy, and explainability), with performance guarantees at both levels.The aforementioned scenario posed three main challenges: (i) Learning from Complex Data (i.e., sequence, tree, and graph data), (ii) Learning Trustworthily, and (iii) Learning Automatically with Guarantees.The focus of this special session is on addressing one or more of these challenges with the final goal of Learning Trustworthily, Automatically, and with Guarantees from Complex Data.

Most Graph Convolutions (GCs) proposed in the Graph Neural Networks (GNNs) literature share the principle of computing topologically enriched node representations based on the ones of their neighbors.In this paper, we propose a novel GNN named Tangent Graph Convolutional Network (TGCN) that, in addition to the traditional GC approach, exploits a novel GC that computes node embeddings based on the differences between the attributes of a vertex and the attributes of its neighbors.This allows the GC to characterize each node's neighbor by computing its tangent space representation with respect to the considered vertex.* This research was supported by the Department of Mathematics, University of Padua with the SID/BIRD 2020 project "Deep Graph Memory Networks" and with the provision of the necessary HPC resources.

Trustworthiness, and in particular Algorithmic Fairness, is emerging as one of the most trending topics in Machine Learning (ML). In fact, ML is now ubiquitous in decision making scenarios, highlighting the necessity of discovering and correcting unfair treatments of (historically discriminated) subgroups in the population (e.g., based on gender, ethnicity, political and sexual orientation). This necessity is even more compelling and challenging when unexplainable black-box Deep Neural Networks (DNN) are exploited. An emblematic example of this necessity is provided by the detected unfair behavior of the ML-based face recognition systems exploited by law enforcement agencies in the United States. To tackle these issues, we first propose different (un)fairness mitigation regularizers in the training process of DNNs. We then study where these regularizers should be applied to make them as effective as possible. We finally measure, by means of different accuracy and fairness metrics and different visual explanation strategies, the ability of the resulting DNNs in learning the desired task while, simultaneously, behaving fairly. Results on the recent FairFace dataset prove the validity of our approach.

We investigate whether Graph Convolutional Neural Networks (GCNNs) may benefit from incorporating information conveyed by a state-of-the-art graph kernel in the learning process. We propose a GCNN architecture and a training procedure based on multi-task learning, where we provide supervision not only from the graph labels, but also from the kernel to each layer of the network, achieving state-of-the-art performances on many real-world datasets. We conduct an ablation study to analyze the impact on the predictive performances of each part of our proposal, including a simplified version of our multi-task learning formulation that can, in principle, be applied with a broad family of graph embeddings. Finally, we study how to improve the performance of a model considering graphs coming from related datasets into the training procedure in a semi-supervised learning fashion.

Predictive Business Process Monitoring is becoming an essential aid for organizations, providing online operational support of their processes. This paper tackles the fundamental problem of equipping predictive business process monitoring with explanation capabilities, so that not only the what but also the why is reported when predicting generic KPIs like remaining time, or activity execution. We use the game theory of Shapley Values to obtain robust explanations of the predictions. The approach has been implemented and tested on real-life benchmarks, showing for the first time how explanations can be given in the field of predictive business process monitoring.

In recent years, several techniques have been made available to automatically discover declarative process models from event logs. These techniques are useful to provide a comprehensible picture of the process as opposed to full specifications of process behavior provided by procedural modeling languages. Since many modern systems produce "big data" from business process executions, in previous work, a framework for the discovery of LTL-based declarative process models from streaming event data has been proposed. This framework can be used to process events online, as they occur, as a way to deal with large and complex collections of datasets that are impossible to store and process altogether. However, the proposed framework does not take into account data attributes associated with events in the log, which can otherwise provide valuable insights into the rules that govern the process. This paper makes the first proposal to close this gap by presenting a technique for discovering declarative process models from event streams that incorporates both control-flow dependencies and data conditions. Specifically, we use Hoeffding trees to incrementally discover data-aware declarative process models, which are represented as conjunctions of first-order temporal logic expressions. The proposed technique has been validated on a synthetic event log, and on a real-life log of a cancer treatment process.

This work proposes a general formulation for the Object Sorting problem, suitable to describe any non-deterministic environment characterized by friendly and adversarial interference. Such an approach, coupled with a Deep Reinforcement Learning algorithm, allows training policies to solve different sorting tasks without adjusting the architecture or modifying the learning method. Briefly, the environment is subdivided into a clutter, where objects are freely located, and a set of clusters, where objects should be placed according to predefined ordering and classification rules. A 3D grid discretizes such environment: the properties of an object within a cell depict its state. Such attributes include object category and order. A Markov Decision Process formulates the problem: at each time step, the state of the cells fully defines the environment's one. Users can custom-define object classes, ordering priorities, and failure rules. The latter by assigning a non-uniform risk probability to each cell. Performed experiments successfully trained and validated a Deep Reinforcement Learning model to solve several sorting tasks while minimizing the number of moves and failure probability. Obtained results demonstrate the capability of the system to handle non-deterministic events, like failures, and unpredictable external disturbances, like human user interventions.

Several machine learning problems can be naturally defined over graph data. Recently, many researchers have been focusing on the definition of neural networks for graphs. The core idea is to learn a hidden representation for the graph vertices, with a convolutive or recurrent mechanism. When considering discriminative tasks on graphs, such as classification or regression, one critical component to design is the readout function, i.e. the mapping from the set of vertex representations to a fixed-size vector (or the output). Different approaches have been presented in literature, but recent approaches tend to be complex, making the training of the whole network harder. In this paper, we frame the problem in the setting of learning over sets. Adopting recently proposed theorems over functions defined on sets, we propose a simple but powerful formulation for a readout layer that can encode or approximate arbitrarily well any continuous permutation-invariant function over sets. Experimental results on real-world graph datasets show that, compared to other approaches, the proposed readout architecture can improve the predictive performance of Graph Neural Networks while being computationally more efficient.

Learning with limited resources (processing power and memory) on a stream of data is a challenging problem. When dealing with structured data, in particular with graphs, state-of- the-art graph kernels coupled with budget-aware online learning algorithms provide an efficient and effective solution. They map the input graph in a feature space that can be represented explicitly in sparse format. In this paper, we propose a method to enhance existing graph kernels in a streaming scenario with strict memory constraints. Specifically, we combine state-of-the-art online kernel methods with the power and flexibility of the feature representation of Extreme Learning Machines (ELM). Although being in principle a simple idea, our proposal, applied to several real-world datasets, outperformed state-of-the-art algorithms on streams of graphs with respect to predictive performance.

BACKGROUND: The uncovering of genes linked to human diseases is a pressing challenge in molecular biology and precision medicine. This task is often hindered by the large number of candidate genes and by the heterogeneity of the available information. Computational methods for the prioritization of candidate genes can help to cope with these problems. In particular, kernel-based methods are a powerful resource for the integration of heterogeneous biological knowledge, however, their practical implementation is often precluded by their limited scalability. RESULTS: We propose Scuba, a scalable kernel-based method for gene prioritization. It implements a novel multiple kernel learning approach, based on a semi-supervised perspective and on the optimization of the margin distribution. Scuba is optimized to cope with strongly unbalanced settings where known disease genes are few and large scale predictions are required. Importantly, it is able to efficiently deal both with a large amount of candidate genes and with an arbitrary number of data sources. As a direct consequence of scalability, Scuba integrates also a new efficient strategy to select optimal kernel parameters for each data source. We performed cross-validation experiments and simulated a realistic usage setting, showing that Scuba outperforms a wide range of state-of-the-art methods. CONCLUSIONS: Scuba achieves state-of-the-art performance and has enhanced scalability compared to existing kernel-based approaches for genomic data. This method can be useful to prioritize candidate genes, particularly when their number is large or when input data is highly heterogeneous. The code is freely available at https://github.com/gzampieri/Scuba .

The availability of graph data with node attributes that can be either discrete or real-valued is constantly increasing. While existing Kernel methods are effective techniques for dealing with graphs having discrete node labels, their adaptation to nondiscrete or continuous node attributes has been limited, mainly for computational issues. Recently, a few kernels especially tailored for this domain, and that trade predictive performance for computational efficiency, have been proposed. In this brief, we propose a graph kernel for complex and continuous nodes' attributes, whose features are tree structures extracted from specific graph visits. The kernel manages to keep the same complexity of the state-of-the-art kernels while implicitly using a larger feature space. We further present an approximated variant of the kernel, which reduces its complexity significantly. Experimental results obtained on six real-world data sets show that the kernel is the best performing one on most of them. Moreover, in most cases, the approximated version reaches comparable performances to the current state-of-the-art kernels in terms of classification accuracy while greatly shortening the running times.

When dealing with kernel methods, one has to decide which kernel and which values for the hyperparameters to use. Resampling techniques can address this issue but these procedures are time-consuming. This problem is particularly challenging when dealing with structured data, in particular with graphs, since several kernels for graph data have been proposed in literature, but no clear relationship among them in terms of learning properties is defined. In these cases, exhaustive search seems to be the only reasonable approach. Recently, the global Rademacher complexity (RC) and local Rademacher complexity (LRC), two powerful measures of the complexity of a hypothesis space, have shown to be suited for studying kernels properties. In particular, the LRC is able to bound the generalization error of an hypothesis chosen in a space by disregarding those ones which will not be taken into account by any learning procedure because of their high error. In this paper, we show a new approach to efficiently bound the RC of the space induced by a kernel, since its exact computation is an NP-Hard problem. Then we show for the first time that RC can be used to estimate the accuracy and expressivity of different graph kernels under different parameter configurations. The authors' claims are supported by experimental results on several real-world graph data sets.

Nowadays, developing effective techniques able to deal with data coming from structured domains is becoming crucial. In this context kernel methods are the state-of-the-art tool widely adopted in real-world applications that involve learning on structured data. Contrarily, when one has to deal with unstructured domains, deep learning methods represent a competitive, or even better, choice. In this paper we propose a new family of kernels for graphs which exploits a deep representation of the information. Our proposal exploits the advantages of the two worlds. From one side we exploit the potentiality of the state-of-the-art graph kernels. From the other side we develop a deep architecture through a series of stacked kernel pre-image estimators trained in an unsupervised fashion via convex optimization. The hidden layers of the proposed framework are trained in a forward manner and this allows us to avoid the greedy layerwise training of classical deep learning. Results on real world graph datasets confirm the quality of the proposal.

MOTIVATION: The importance of RNA protein-coding gene regulation is by now well appreciated. Non-coding RNAs (ncRNAs) are known to regulate gene expression at practically every stage, ranging from chromatin packaging to mRNA translation. However the functional characterization of specific instances remains a challenging task in genome scale settings. For this reason, automatic annotation approaches are of interest. Existing computational methods are either efficient but non-accurate or they offer increased precision, but present scalability problems. RESULTS: In this article, we present a predictive system based on kernel methods, a type of machine learning algorithm grounded in statistical learning theory. We employ a flexible graph encoding to preserve multiple structural hypotheses and exploit recent advances in representation and model induction to scale to large data volumes. Experimental results on tens of thousands of ncRNA sequences available from the Rfam database indicate that we can not only improve upon state-of-the-art predictors, but also achieve speedups of several orders of magnitude. AVAILABILITY AND IMPLEMENTATION: The code is available from http://www.bioinf.uni-freiburg.de/~costa/EDeN.tgz . CONTACT: [email protected].

In this paper, we show how learning models generated by a recently introduced state-of-the-art kernel for graphs can be optimized from the point of view of memory occupancy. After a brief description of the kernel, we introduce a novel representation of the explicit feature space of the kernel based on an hash function which allows to reduce the amount of memory needed both during the training phase and to represent the final learned model. Subsequently, we study the application of a feature selection strategy based on the F-score to further reduce the number of features in the final model. On two representative datasets involving binary classification of chemical graphs, we show that it is actually possible to sensibly reduce memory occupancy (up to one order of magnitude) for the final model with a moderate loss in classification performance.

This paper proposes a new tree-based kernel for graphs. Graphs are decomposed into multisets of ordered Directed Acyclic Graphs (DAGs) and a family of kernels computed by application of tree kernels extended to the DAG domain. We focus our attention on the efficient development of one member of this family. A technique for speeding up the computation is given, as well as theoretical bounds and practical evidence of its feasibility. State of the art results on various benchmark datasets prove the effectiveness of our approach.