Transformer models, despite their exceptional capabilities in Natural Language Processing (NLP) and Vision tasks, like deep neural network models, often function as "black boxes" as their internal processes remain largely opaque due to their complex architectures. This work extends graph-based knowledge extraction techniques, previously applied to CNNs, to the domain of Transformer models. The inner mechanics of Transformer models are explored by constructing a co-activation graph from their encoder layers. The nodes of the graph represent the hidden unit within each encoder layer, while the edges represent the statistical correlations between these hidden units. The magnitude of co-activation, which is the correlation between activations of two hidden units, determines the strength of their connection within the graph. Our research is focused on encoder-only Transformer classifiers. We conducted experiments involving a custom-built Transformer and a pre-trained BERT model for an NLP task. We used graph analysis to detect semantically related class clusters and their impact on misclassification patterns. We demonstrate a positive correlation between class similarity and the frequency of classification errors. Our findings suggest that co-activation graphs reveal structured, interpretable representations in Transformers, consistent with prior CNN findings on knowledge extraction.

Abstract Data augmentation is essential for enhancing computer vision performance, with the KeepOriginalAugment method standing out for intelligently incorporating salient and less prominent regions. Despite its success in image classification, its potential in addressing biases is unexplored. We introduce FaceKeepOriginalAugment, extending KeepOriginalAugment to address geographical, gender, and stereotypical biases in computer vision models. By balancing data diversity and information preservation, our approach enables models to leverage both salient and non-salient regions, fostering diversity and debiasing. We explore strategies for salient region placement and augmentation selection, quantifying diversity using Image Similarity Score (ISS) across datasets like FFHQ, WIKI, IMDB, LFW, and UTK Faces. We assess FaceKeepOriginalAugment in mitigating gender bias across CEO, Engineer, Nurse, and School Teacher datasets, using the Image-Image Association Score (IIAS) in CNNs and vision transformers (ViTs). Results show FaceKeepOriginalAugment effectively promotes fairness and inclusivity by reducing gender bias and enhancing fairness. Additionally, we introduce a Saliency-Based Diversity and Fairness Metric to quantify diversity and fairness while addressing data imbalance across datasets.

Recent developments in in-situ monitoring and process control in Additive Manufacturing (AM), also known as 3D-printing, allows the collection of large amounts of emission data during the build process of the parts being manufactured. This data can be used as input into 3D and 2D representations of the 3D-printed parts. However the analysis and use, as well as the characterization of this data still remains a manual process. The aim of this paper is to propose an adaptive human-in-the-loop approach using Machine Learning techniques that automatically inspect and annotate the emissions data generated during the AM process. More specifically, this paper will look at two scenarios: firstly, using convolutional neural networks (CNNs) to automatically inspect and classify emission data collected by in-situ monitoring and secondly, applying Active Learning techniques to the developed classification model to construct a human-in-the-loop mechanism in order to accelerate the labeling process of the emission data. The CNN-based approach relies on transfer learning and fine-tuning, which makes the approach applicable to other industrial image patterns. The adaptive nature of the approach is enabled by uncertainty sampling strategy to automatic selection of samples to be presented to human experts for annotation.

The popularity of deep learning has increased tremendously in recent years due to its ability to efficiently solve complex tasks in challenging areas such as computer vision and language processing. Despite this success, low-level neural activity reproduced by Deep Neural Networks (DNNs) generates extremely rich representations of the data. These representations are difficult to characterise and cannot be directly used to understand the decision process. In this paper we build upon our exploratory work where we introduced the concept of a co-activation graph and investigated the potential of graph analysis for explaining deep representations. The co-activation graph encodes statistical correlations between neurons’ activation values and therefore helps to characterise the relationship between pairs of neurons in the hidden layers and output classes. To confirm the validity of our findings, our experimental evaluation is extended to consider datasets and models with different levels of complexity. For each of the considered datasets we explore the co-activation graph and use graph analysis to detect similar classes, find central nodes and use graph visualisation to better interpret the outcomes of the analysis. Our results show that graph analysis can reveal important insights into how DNNs work and enable partial explainability of deep learning models.

The ability to enhance deep representations with prior knowledge is receiving a lot of attention from the AI community as a key enabler to improve the way modern Artificial Neural Networks (ANN) learn. In this paper we introduce our approach to this task, which comprises of a knowledge extraction algorithm, a knowledge injection algorithm and a common intermediate knowledge representation as an alternative to traditional neural transfer. As a result of this research, we envisage a knowledge-enhanced ANN, which will be able to learn, characterise and reuse knowledge extracted from the learning process, thus enabling more robust architecture-agnostic neural transfer, greater explainability and further integration of neural and symbolic approaches to learning.

An increasing number of cities are confronted with challenges resulting from the rapid urbanization and new demands that a rapidly growing digital economy imposes on current applications and information systems. Smart city applications enable city authorities to monitor, manage, and provide plans for public resources and infrastructures in city environments, while offering citizens and businesses to develop and use intelligent services in cities. However, providing such smart city applications gives rise to several issues, such as semantic heterogeneity and trustworthiness of data sources, and extracting up-to-date information in real time from large-scale dynamic data streams. In order to address these issues, we propose a novel framework with an efficient semantic data processing pipeline, allowing for real-time observation of the pulse of a city. The proposed framework enables efficient semantic integration of data streams, and complex event processing on top of real-time data aggregation and quality analysis in a semantic Web environment. To evaluate our system, we use real-time sensor observations that have been published via an open platform called Open Data Aarhus by the City of Aarhus. We examine the framework utilizing symbolic aggregate approximation to reduce the size of data streams, and perform quality analysis taking into account both single and multiple data streams. We also investigate the optimization of the semantic data discovery and integration based on the proposed stream quality analysis and data aggregation techniques.

Stream reasoning is an emerging research area focused on providing continuous reasoning solutions for data streams. The high expressiveness of non-monotonic reasoning enables complex decision making by managing defaults, commonsense, preferences, recursion, and non-determinism, but it is computationally intensive. The exponential growth in the availability of streaming data on the Web has seriously hindered the applicability of state-of-the-art non-monotonic reasoners to be applied to streaming information in a scalable way. In this paper, we address the issue of scalability for nonmonotonic stream reasoning based on Answer Set Programming (ASP) - an expressive reasoning approach based on disjunctive logic Datalog with negation under the stable model semantics, by analyzing input dependency. We introduce an input dependency graph to represent the relationships between input events based on the structure of a given logical rule set. The input dependency graph allows us to dynamically configure the streaming window size in order to maximise the scalability of the non-monotonic reasoner. We conduct an experimental evaluation to demonstrate the effectiveness and ability of our proposed approach in improving the scalability of disjunctive logic programming with ASP in dynamic environments.

The proliferation of sensor devices and services along with the advances in event processing brings many new opportunities as well as challenges. It is now possible to provide, analyze and react upon real-time, complex events in urban environments. When existing event services do not provide such complex events directly, an event service composition maybe required. However, it is difficult to determine which event service candidates (or service compositions) best suit users' and applications' quality-of-service requirements. A sub-optimal service composition may lead to inaccurate event detection, lack of system robustness etc. In this paper, the authors address these issues by first providing a quality-of-service aggregation schema for complex event service compositions and then developing a genetic algorithm to efficiently create near-optimal event service compositions. The authors evaluate their approach with both real sensor data collected via Internet-of-Things services as well as synthesised datasets.

The tables embedded in Wikipedia articles contain rich, semi-structured encyclopaedic content. However, the cumulative content of these tables cannot be queried against. We thus propose methods to recover the semantics of Wikipedia tables and, in particular, to extract facts from them in the form of RDF triples. Our core method uses an existing Linked Data knowledge-base to find pre-existing relations between entities in Wikipedia tables, suggesting the same relations as holding for other entities in analogous columns on different rows. We find that such an approach extracts RDF triples from Wikipedia's tables at a raw precision of 40%. To improve the raw precision, we define a set of features for extracted triples that are tracked during the extraction phase. Using a manually labelled gold standard, we then test a variety of machine learning methods for classifying correct/incorrect triples. One such method extracts 7.9 million unique and novel RDF triples from over one million Wikipedia tables at an estimated precision of 81.5%.

Abstract In recent years there has been a growing interest in solutions for the delivery of clinical care for the elderly because of the large increase in aging population. Monitoring a patient in his home environment is necessary to ensure continuity of care in home settings, but, to be useful, this activity must not be too invasive for patients and a burden for caregivers. We prototyped a system called Secure and INDependent lIving (SINDI), focused on (a) collecting a limited amount of data about the person and the environment through Wireless Sensor Networks (WSN), and (b) inferring from these data enough information to support caregivers in understanding patients' well-being and in predicting possible evolutions of their health. Our hierarchical logic-based model of health combines data from different sources, sensor data, tests results, common-sense knowledge and patient's clinical profile at the lower level, and correlation rules between health conditions across upper levels. The logical formalization and the reasoning process are based on Answer Set Programming. The expressive power of this logic programming paradigm makes it possible to reason about health evolution even when the available information is incomplete and potentially incoherent, while declarativity simplifies rules specification by caregivers and allows automatic encoding of knowledge. This paper describes how these issues have been targeted in the application scenario of the SINDI system.

In this paper we propose an innovative approach to the problem of indoor position estimation that aims at extending tracking to a new level of “awareness” bringing to bear new ambient data and opening the possibility of “reasoning” not only on simple positioning but also on the situation at hand. In order to validate the approach, we implemented a positioning system called Situation-Aware Indoor Tracking (SAIT). The comparison of SAIT with several commercial systems highlights a promising behaviour, showing that exploiting the movement data (e.g. the users' heading and speed) for updating the PF motion models used in the tracking engine together with situation assessment techniques can improve the accuracy of tracking up to 42% in comparison with a Wi-Fi based system.

This paper describes an intelligent home healthcare system characterized by a wireless sensor network (WSN) and a reasoning component. The aim of the system is to allow constant and unobtrusive monitoring of a patient in order to enhance autonomy and increase quality of life. Data collected by the sensor network are used to support a reasoning component, which is based on answer set programming (ASP), in performing three main reasoning tasks: (i) continuous contextualization of the physical, mental and social state of a patient, (ii) prediction of possibly risky situations and (iii) identification of plausible causes for the worsening of a patient's health. Starting from different data sources (sensor data, test results, inference results) the reasoning component applies expressive logic rules aimed at correct interpretation of incomplete or inconsistent contextual information, and evaluates correlation rules expressed by clinicians. The expressive power of ASP allows efficient enough reasoning to support prevention, while declarativity simplifies rule-specification and allows automatic encoding of knowledge. Preliminary evaluations show that the combination of an ASP-based reasoning component and a WSN is a good solution for creating a home-based healthcare system.

We show how a standard grid service architecture can be improved by interposing a policy enforcement engine between a calling application and the relative client stubs. Therefore, with our solution selection and invocations are not hard-coded into client applications but (declaratively) defined and enforced outside the clients; therefore they can be (de)activated and modified online. Our policies are specified using the PDL language which supports specification of preferences and prohibitions in the routing of remote invocations to Web services.