The performance of Active Learning (AL) methods varies widely, influenced by the query strategy, model, and dataset, with the reasons for variation in performance still unclear and insufficiently studied. However, commonly used metrics like accuracy, precision, and recall provide only limited analytical perspectives. No research has effectively uncovered or explained the reasons behind these performance variations, leaving a gap in understanding of the factors that influence the success or failure of AL methods. To address this issue, we propose a novel method and tool leveraging Voronoi Diagrams to visualize AL processes by illustrating interactions between classification decision boundary changes and queried samples across AL iterations. We perform experiments on synthetic and real-world datasets to validate the effectiveness of our method and analyze various AL query strategies. By visualizing the AL process, we illustrate how different query strategies progressively select samples and influence performance in each iteration. This reveals the potential benefits of adapting query strategies at different learning stages to improve active learning efficiency.

We report on a trans-Atlantic PhD student mobility program that connects two graduate research training initiatives in the US and Ireland, centered on developing future researchers in artificial intelligence (AI) and machine learning (ML). We discuss both the structure of the student exchange experiences and share key learnings from this international collaboration. The most important lesson learned is that providing a structured mobility program and matched visiting pairs is a highly effective way to improve learning outcomes compared to more typical ad-hoc individual visits.

In the rapidly evolving Generative AI (GenAI) landscape, source code and natural language are being mixed and used in new ways. This presents opportunities for rethinking teaching practice in Introductory Programming (CS1) courses that includes, but goes beyond, assessment. In this paper we examine the reasons why and how instructors who are early adopters of GenAI are using it in their teaching, and why others are not. We also explore the changes and adaptations that are currently being made to practice. This is achieved by synthesizing insights from several recent studies that have collected primary data from introductory programming instructors who are teaching with, considering teaching with, or actively not teaching with GenAI.

In this study, we aim to reduce generation latency for Named Entity Recognition (NER) with Large Language Models (LLMs). The main cause of high latency in LLMs is the sequential decoding process, which autoregressively generates all labels and mentions for NER, significantly increase the sequence length. To this end, we introduce Parallel Decoding in LLM for NE} (PaDeLLM-NER), a approach that integrates seamlessly into existing generative model frameworks without necessitating additional modules or architectural modifications. PaDeLLM-NER allows for the simultaneous decoding of all mentions, thereby reducing generation latency. Experiments reveal that PaDeLLM-NER significantly increases inference speed that is 1.76 to 10.22 times faster than the autoregressive approach for both English and Chinese. Simultaneously it maintains the quality of predictions as evidenced by the performance that is on par with the state-of-the-art across various datasets. All resources are available at https://github.com/GeorgeLuImmortal/PaDeLLM_NER.

Personalised news recommender systems are effective in disseminating news content based on users’ reading histories but can also amplify and proliferate biased media. This work examines the potential of automated sentence rewriting methods, utilising word replacement methods and large language models (LLMs), to mitigate this side effect of recommender systems. We present a two-step workflow: the application of automated sentence rewriting methods to rewrite biased sentences, and the integration of these rewritten sentences into the recommendation process. We evaluate the effectiveness of sentence rewriting approaches in a simulation framework, to assess how well they mitigate the spread of biased news. Our study demonstrates that applying sentence rewriting to users’ reading histories can result in a significant reduction in the propagation of biased media. Our contributions are threefold: we pioneer the use of LLMs for mitigating the spread of biased news by recommender systems; we demonstrate that algorithms trained on debiased content maintain or improve recommendation accuracy; and we provide a comprehensive exploration of the effectiveness of applying sentence rewriting methods to various components within a recommender system, as well as an investigation of the underlying reasons for their efficacy. This work advances our understanding of media bias mitigation in news content and recommendation algorithms, providing valuable insights into how news recommender systems can prevent the dissemination of biased information.

Much of named entity recognition (NER) research focuses on developing dataset-specific models based on data from the domain of interest, and a limited set of related entity types. This is frustrating as each new dataset requires a new model to be trained and stored. In this work, we present a ``versatile'' model---the Prompting-based Unified NER system (PUnifiedNER)---that works with data from different domains and can recognise up to 37 entity types simultaneously, and theoretically it could be as many as possible. By using prompt learning, PUnifiedNER is a novel approach that is able to jointly train across multiple corpora, implementing intelligent on-demand entity recognition. Experimental results show that PUnifiedNER leads to significant prediction benefits compared to dataset-specific models with impressively reduced model deployment costs. Furthermore, the performance of PUnifiedNER can achieve competitive or even better performance than state-of-the-art domain-specific methods for some datasets. We also perform comprehensive pilot and ablation studies to support in-depth analysis of each component in PUnifiedNER.

Current methods for prompt learning in zeroshot scenarios widely rely on a development set with sufficient human-annotated data to select the best-performing prompt template a posteriori.This is not ideal because in a real-world zero-shot scenario of practical relevance, no labelled data is available.Thus, we propose a simple yet effective method for screening reasonable prompt templates in zero-shot text classification: Perplexity Selection (Perplection).We hypothesize that language discrepancy can be used to measure the efficacy of prompt templates, and thereby develop a substantiated perplexity-based scheme allowing for forecasting the performance of prompt templates in advance.Experiments show that our method leads to improved prediction performance in a realistic zero-shot setting, eliminating the need for any labelled examples.PPL Acc.(%) PPL Acc.(%) PPL Acc.(%) PPL Acc.(%) DOUBAN 24.61 57.12 40.93 50.98 28.80 56.68 71.01 51.31 WEIBO 19.78 61.79 30.37 51.16 22.34 58.35 44.45 50.92WAIMAI 16.44 67.80 23.34 53.15 19.68 69.72 36.07 48.49ECOMMERCE 14.07 73.12 18.45 55.68 16.88 67.

Medical image classification models are frequently trained using training datasets derived from multiple data sources. While leveraging multiple data sources is crucial for achieving model generalization, it is important to acknowledge that the diverse nature of these sources inherently introduces unintended confounders and other challenges that can impact both model accuracy and transparency. A notable confounding factor in medical image classification, particularly in musculoskeletal image classification, is skeletal maturation-induced bone growth observed during adolescence. We train a deep learning model using a Covid-19 chest X-ray dataset and we showcase how this dataset can lead to spurious correlations due to unintended confounding regions. eXplanation Based Learning (XBL) is a deep learning approach that goes beyond interpretability by utilizing model explanations to interactively unlearn spurious correlations. This is achieved by integrating interactive user feedback, specifically feature annotations. In our study, we employed two non-demanding manual feedback mechanisms to implement an XBL-based approach for effectively eliminating these spurious correlations. Our results underscore the promising potential of XBL in constructing robust models even in the presence of confounding factors.

There is often a mixture of very frequent labels and very infrequent labels in multi-label datatsets. This variation in label frequency, a type class imbalance, creates a significant challenge for building efficient multi-label classification algorithms. In this paper, we tackle this problem by proposing a minority class oversampling scheme, UCLSO, which integrates Unsupervised Clustering and Label-Specific data Oversampling. Clustering is performed to find out the key distinct and locally connected regions of a multi-label dataset (irrespective of the label information). Next, for each label, we explore the distributions of minority points in the cluster sets. Only the minority points within a cluster are used to generate the synthetic minority points that are used for oversampling. Even though the cluster set is the same across all labels, the distributions of the synthetic minority points will vary across the labels. The training dataset is augmented with the set of label-specific synthetic minority points, and classifiers are trained to predict the relevance of each label independently. Experiments using 12 multi-label datasets and several multi-label algorithms show that the proposed method performed very well compared to the other competing algorithms.

eXplanation Based Learning (XBL) is an interactive learning approach that provides a transparent method of training deep learning models by interacting with their explanations. XBL augments loss functions to penalize a model based on deviation of its explanations from user annotation of image features. The literature on XBL mostly depends on the intersection of visual model explanations and image feature annotations. We present a method to add a distance-aware explanation loss to categorical losses that trains a learner to focus on important regions of a training dataset. Distance is an appropriate approach for calculating explanation loss since visual model explanations such as Gradient-weighted Class Activation Mapping (Grad-CAMs) are not strictly bounded as annotations and their intersections may not provide complete information on the deviation of a model’s focus from relevant image regions. In addition to assessing our model using existing metrics, we propose an interpretability metric for evaluating visual feature-attribution based model explanations that is more informative of the model’s performance than existing metrics. We demonstrate performance of our proposed method on three image classification tasks.

Currently I am at the beginning of my fourth year of a structured PhD programme with an expectation to graduate in May 2024. The advancement of Internet technology has led to the proliferation of accessible online news media, which has overwhelmed people’s lives. Online news platforms have developed personalised recommendation systems to help readers avoid information overload and enhance their experience. However, the filter bubble, one of the side effects of personalised news recommendations, has received severe criticism for limiting readers’ perspectives. Media bias, which is one of the factors causing the “filter bubble” phenomenon, is widely present in news media. It has been extensively studied in the field of social sciences due to its unconscious distortion of readers’ views. Although many studies have focused on examining the effect of media bias on users and their political choices, there is still a lack of direct research on the impact of media bias on news dissemination platforms, such as personalised news recommender systems. My PhD research project aims to explore the influence of media bias on news recommender systems, and understand the factors that accelerate the recommendation of biased news to readers. To help algorithm designers gain insight into the sensitivity of proposed recommendation algorithms to media bias, and to design debiasing algorithms to weaken the impact of media bias on news recommender systems.

We present a novel rationale-centric framework with human-in-the-loop -Rationales-centric Double-robustness Learning (RDL) -to boost model out-of-distribution performance in few-shot learning scenarios.By using static semi-factual generation and dynamic humanintervened correction, RDL exploits rationales (i.e.phrases that cause the prediction), human interventions and semi-factual augmentations to decouple spurious associations and bias models towards generally applicable underlying distributions, which enables fast and accurate generalisation.Experimental results show that RDL leads to significant prediction benefits on both in-distribution and out-of-distribution tests compared to many state-of-the-art benchmarks-especially for few-shot learning scenarios.We also perform extensive ablation studies to support in-depth analyses of each component in our framework.

Explanatory Interactive Learning (XIL) collects user feedback on visual model explanations to implement a Human-in-the-Loop (HITL) based interactive learning scenario. Different user feedback types will have different impacts on user experience and the cost associated with collecting feedback since different feedback types involve different levels of image annotation. Although XIL has been used to improve classification performance in multiple domains, the impact of different user feedback types on model performance and explanation accuracy is not well studied. To guide future XIL work we compare the effectiveness of two different user feedback types in image classification tasks: (1) instructing an algorithm to ignore certain spurious image features, and (2) instructing an algorithm to focus on certain valid image features. We use explanations from a Gradient-weighted Class Activation Mapping (GradCAM) based XIL model to support both feedback types. We show that identifying and annotating spurious image features that a model finds salient results in superior classification and explanation accuracy than user feedback that tells a model to focus on valid image features.

We present an overview of a "Machine Learning and Artificial Intelligence" course that is part of a large online course platform for upper second level students. We take a novel approach to teaching fundamental AI concepts that does not require code, and assumes little prior knowledge including only basic mathematics. The design ethos is for students to gain an understanding of how algorithms can "learn". Many misconceptions exist about this term with respect to AI and can lead to confusion and more serious misconceptions, particularly for students who engage with AI-enabled tools regularly. This approach aims to provide insights into how AI actually works, to demystify and remove barriers to more advanced learning, and to emphasize the important roles of ethics and bias in AI. We took several steps to engage students, including videos narrated by a final-year second-level student (US 12th grade). We present design and logistics particulars on this course which is currently being taken by ~7,000 students in Ireland. We believe this will be of value to other educators and the wider community.

This paper proposes Random Walk-steered Majority Undersampling (RWMaU), an undersampling approach to address the class imbalance problem for binary classifiers. RWMaU is focused to find the majority points which lie at the overlapped region of the minority and the majority classes. Such points meddle with the learning and detection of the minority points. RWMaU uses random walks to mark the majority points satisfying the above characteristic in a non-parametric fashion. For each majority point, a proximity score is calculated on the basis of - their visit frequencies and the order of visits of the majority points in the random walks. This score is used to perceive the closeness of the majority class points to the minority class. The majority points lying close to the minority class are subsequently undersampled. Empirical evaluations on 21 datasets using 3 classifiers demonstrate substantial improvement in performance of RWMaU over existing methods for addressing class imbalance and show that it is an efficient and effective way to address class imbalance in binary classification problems.

Neural networks are often utilised in critical domain applications, even though they exhibit overconfident predictions for ambiguous inputs. This deficiency demonstrates a fundamental flaw: that neural networks often overfit on spurious correlations. We address this limitation by presenting two novel objectives that improve out-of-distribution (OOD) detection. We empirically demonstrate that our methods outperform the baseline while still maintaining a competitive performance against the rest. Additionally, we empirically demonstrate the robustness of our approach against common corruptions and the importance of regularisation and auxiliary information in OOD detection.

This paper proposes a novel Orthogonal Classification Layer (OCL) utilizing Kasami sequences for neural networks trained for classification problems. OCL consists of a fully connected layer with fixed orthogonal weights and zero biases for all of its output neurons. Attaching the OCL to the end of any neural network encourages the network to generate a unique latent representation (orthogonal code) at its last hidden layer for each data class. This is achieved by associating and fixing the weights for each of the output neurons to a unique code from a set of orthogonal sequences, such as Kasami. When networks use OCL the latent representations they learn for each data class at the end of the network converge to values equivalent to the fixed weights of the OCL. Auto-correlation and cross-correlation properties of orthogonal codes maximize the output of the correct class and increase its separation from the outputs of all other classes. Therefore, networks trained with OCL benefit from a wider classification decision margin than networks without OCL. Moreover, the feature sets extracted by the network for different data classes are well separated and more amenable to human interpretation. The implementation of OCL is simple and OCL can easily be integrated into any neural network architecture without a need for network architecture modifications or changes to the training scheme (for example optimization, normalization, or regularization) adopted in the original network architecture. The computational and memory cost required for the OCL is lower than conventional classification layers since the weights are fixed and no gradient is required. Though simple and practical the proposed OCL enhances learning of discriminative latent representations, generates explainable features, and provides high classification accuracy. We demonstrate this through a set of evaluation experiments comparing the performance of equivalent networks with and without OCL.

Abstract Multi‐label classification algorithms deal with classification problems where a single datapoint can be classified (or labelled) with more than one class (or label) at the same time. Early multi‐label approaches like binary relevance consider each label individually and train individual binary classifier models for each label. State‐of‐the‐art algorithms like RAkEL, classifier chains, calibrated label ranking, IBLR‐ML+, and BPMLL also consider the associations between labels for improved performance. Like most machine learning algorithms, however, these approaches require careful hyper‐parameter tuning, a computationally expensive optimisation problem. There is a scarcity of multi‐label classification algorithms that require minimal hyper‐parameter tuning. This paper addresses this gap in the literature by proposing CascadeML, a multi‐label classification method based on the existing cascaded neural network architecture, which also takes label associations into consideration. CascadeML grows a neural network architecture incrementally (deep as well as wide) in a two‐phase process as it learns network weights using an adaptive first‐order gradient descent algorithm. This omits the requirement of preselecting the number of hidden layers, nodes, activation functions, and learning rate. The performance of the CascadeML algorithm was evaluated using 13 multi‐label datasets and compared with nine existing multi‐label algorithms. The results show that CascadeML achieved the best average rank over the datasets, performed better than BPMLL (one of the earliest well known multi‐label specific neural network algorithms), and was similar to the state‐of‐the‐art classifier chains and RAkEL algorithms.

Compartment-based infectious disease models that consider the transmission rate (or contact rate) as a constant during the course of an epidemic can be limiting regarding effective capture of the dynamics of infectious disease. This study proposed a novel approach based on a dynamic time-varying transmission rate with a control rate governing the speed of disease spread, which may be associated with the information related to infectious disease intervention. Integration of multiple sources of data with disease modelling has the potential to improve modelling performance. Taking the global mobility trend of vehicle driving available via Apple Maps as an example, this study explored different ways of processing the mobility trend data and investigated their relationship with the control rate. The proposed method was evaluated based on COVID-19 data from six European countries. The results suggest that the proposed model with dynamic transmission rate improved the performance of model fitting and forecasting during the early stage of the pandemic. Positive correlation has been found between the average daily change of mobility trend and control rate. The results encourage further development for incorporation of multiple resources into infectious disease modelling in the future.

Lateral Flow Immunoassays (LFA) are low cost, rapid and highly efficacious Point-of-Care devices. Traditional LFA testing faces challenges to detect high-sensitivity biomarkers due to low sensitivity. Unlike most approaches based on averaging image intensity from a region-of-interest (ROI), this paper presents a novel system that considers each row of an LFA image as a time series signal and, consequently, does not require the detection of ROI. Long Short-Term Memory (LSTM) networks are used to classify LFA data obtained from multilevel high-sensitivity cardiovascular biomarkers. Dynamic Time Warping (DTW) was incorporated with LSTM to align the LFA data from different concentration levels to a common reference before feeding the distance maps into an LSTM network. The LSTM network outperforms other classifiers with or without DTW. Furthermore, performance of all classifiers is improved after incorporating DTW. The positive outcomes suggest the potential of the proposed methods for early risk assessment of cardiovascular diseases.

Jensen-Shannon divergence (JSD) is a distribution similarity measurement widely used in natural language processing. In corpus comparison tasks, where keywords are extracted to reveal the divergence between different corpora (for example, social media posts from proponents of different views on a political issue), two variants of JSD have emerged in the literature. One of these uses a weighting based on the relative sizes of the corpora being compared. In this paper we argue that this weighting is unnecessary and, in fact, can lead to misleading results. We recommend that this weighted version is not used. We base this recommendation on an analysis of the JSD variants and experiments showing how they impact corpus comparison results as the relative sizes of the corpora being compared change.

Anomaly detection involves the recognition of patterns outside of what is considered normal, given a certain set of input data. This presents a unique set of challenges for machine learning, particularly if we assume a semi-supervised scenario in which anomalous patterns are unavailable at training time meaning algorithms must rely on non-anomalous data alone. Anomaly detection in time series adds an additional level of complexity given the contextual nature of anomalies. For time series modelling, autoregressive deep learning architectures such as WaveNet have proven to be powerful generative models, specifically in the field of speech synthesis. In this paper, we propose to extend the use of this type of architecture to anomaly detection in raw audio. In experiments using multiple audio datasets we compare the performance of this approach to a baseline autoencoder model and show superior performance in almost all cases.

Flight efficiency indicators reported monthly in the European area by the Performance Review Unit (PRU) help the air traffic management (ATM) community determine if excessive distances are being flown (compared with the ideal lengths of flight routes). Recent research, however, provides more indicators that comprehensively capture flight efficiencies in terms of other factors including fuel consumption, time adherence, and route charges. The efficacy of all of these indicators, however, is diminished as they are currently only available almost a month after flights take place. This is not sufficiently timely to use these indicators for the alleviation of unpredictable hotspots (i.e. sectors with congested air traffic), which often leads to unexpected ground delays. This paper proposes a methodology to calculate general flight efficiency indicators on-line in near real-time using nearest point search. A prototype system called ROGER (compRehensive On-line fliGht Efficiency monitoRing) is implemented using Apache Kafka and Spark. ROGER can digest large-scale heterogeneous datasets (i.e. mainly ADS-B data, the next generation aircraft surveillance technology) to compute indicators every 5 seconds. Our experiments on realistic datasets demonstrate that the proposed on-line indicator calculation method can achieve high accuracy compared with existing off-line approaches, and that ROGER can achieve desirable system performance in throughput and latency. A use case is also described showing how ROGER can assist in alleviating hotspots more effectively.

This paper presents two studies that evaluate the effectiveness of a software visualisation tool which uses a com-posite visualisation to encode the scope chain and information related to the scope chain within source code. The first study evaluates the effectiveness of adding the composite visualisation to a source code editor to help programmers understand scope relationships within source code. The second study evaluates the effectiveness of each individual component within the composite visualisation. The composite visualisation is composed of a packed circle tree diagram (overview component) and a list view (detail view component). The packed circle tree functions as an abstract mini-map to provide viewers with a high-level overview of the scope chain hierarchy within a source code document. The list view provides additional information about identifiers (variables, functions, and parameters) that are accessible from the scope within which the cursor is located, in the source code document. Both studies utilise a between-subject design, in which groups of participants were presented with source code fragments and asked to answer a series of code understanding questions. The results of the studies indicate that adding a composite visualisation to a source code editor can have a positive effect on code understanding, especially when the textual representation of the code no longer corresponds to the actual behaviour of the code (as is the case, for example, in languages such as JavaScript that implement variable hoisting).

Modern source code editors typically include a code mini-map visualisation, which provides programmers with an overview of the currently open source code document. This paper proposes to add a layering mechanism to the code mini-map visualisation in order to provide programmers with visual answers to questions related to conceptual structures that are not manifested directly in the code. Details regarding the design and implementation of this scope information layer, which displays additional encodings that correspond to the scope chain and information related to the scope chain within a source code document, is presented. The scope information layer can be used by programmers to answer questions such as: to which scope does a specific variable belong, and in which scope is the cursor of the source code editor currently located in. Additionally, this paper presents a study that evaluates the effectiveness of adding the scope information layer to a code mini-map visualisation in order to help programmers understand scope relationships within source code. The results of the study show that the incorporating additional layers of information onto the code mini-map visualisation can have a positive effect on code understanding.

This paper presents a design study that investigates the use of compact tree visualisations to provide software developers with an overview of the static structure of a source code document within a code editor in order to facilitate source code understanding and navigation. A prototype is presented which utilises an icicle tree visualisation to encode the control structure hierarchy of a source code document, as well as a circular treemap visualisation to encode the scope hierarchy of a source code document. An overview of the prototype and its functionality is given as well as a detailed discussion on the design rationale behind the tool. Possible applications and future work plans are also discussed.

The behaviour of non-player character game agents can be made more interesting and believable through the use of increased contextual awareness. In this paper, we present smart ambiance which allows information about the ambiance of an environment (determined by the environment itself, objects in the environment and recent events) to be used in agent plan generation. We demonstrate how this leads to contextually influenced action selection and, in turn, more interesting and believable character behaviour.

In making fully immersive augmented reality (AR) applications, real and virtual objects will have to be seen to physically interact together in a realistic and believable way. This paper describes Forked! a system that has been developed to show how physical interactions between real and virtual objects can be simulated realistically and believably through appropriate use of a physics engine. The system allows users control a robotic forklift to manipulate virtual crates in an AR environment. The paper also describes a evaluation experiment in which it is shown that the physical interactions between the forklift and the virtual creates are realistic and believable enough to be comparable with the physical interactions between a forklift and real crates.