Domain generalisation aims to develop models that generalise from source domains to unseen target domains. However, most existing methods assume access to source domain data and require additional training, which may not always be practical. We focus on a more flexible and broadly applicable setting, zero-shot domain generalisation, where models generalise without access to source data, target data, or any additional training. In this work, we propose Prefer (prompt-driven feature refinement), a simple and effective approach that enhances the zero-shot domain generalisation ability of vision-language foundation models. Prefer generates a diverse set of textual prompts for each class by imagining domain-specific variations (e.g., "a painting of a cat under a golden sunset with thick brush strokes"), and uses them to probe the model. We evaluate how reliably each feature channel represents a class across domains by measuring two quantities: (1) how strongly the channel aligns with the original class prompt (e.g., "a photo of a cat") across the generated domain-specific prompts, and (2) how stable the channel remains across those prompts, quantified by its variance. Channels that exhibit both high alignment and low variability are selected at inference time to improve class prediction under domain shift. Without any model updates or external data, Prefer achieves consistent improvements across domain generalisation benchmarks, outperforming existing state-of-the-art methods.

Identifying individual animals at different points in space and time is vital for effective wildlife monitoring and biodiversity conservation. While existing computer vision methods have shown promise in re-identifying animals, their capability in Animal Re-Identification (Animal ReID) remains restricted by the inherent visual variations, specifically high intra- and low inter-identity variations. High intra-identity variations refer to high visual diversity within the same individual due to pose or form changes and occlusions, and low inter-identity variations refer to subtle visual differences between distinct individuals due to fine-grained appearances. To address these challenges, we propose the Clip-based Animal RE-identification (CARE) framework, which leverages the image-conditioned textual description generation and individual-level semantic feature alignment, mitigating the negative impacts of visual variations in Animal ReID. Crucially, we have packaged CARE into a stand-alone toolkit and piloted it with stakeholders, facilitating real-world wildlife monitoring for biodiversity conservation. Extensive experiments on benchmark and in-the-wild datasets further demonstrate that CARE consistently outperforms state-of-the-art methods, validating its effectiveness in Animal ReID. Explore more about CARE at https://ml4sg.auckland.ac.nz/animal-re-identification-model/.

Graphs model latent variable relationships in many real-world systems, and Message Passing Neural Networks (MPNNs) are widely used to learn such structures for downstream tasks. While edge-based MPNNs effectively capture local interactions, their expressive power is theoretically bounded, limiting the discovery of higher-order relationships. We introduce the Higher-Order Graph Attention (HoGA) module, which constructs a k-order attention matrix by sampling subgraphs to maximize diversity among feature vectors. Unlike existing higher-order attention methods that greedily resample similar k-order relationships, HoGA targets diverse modalities in higher-order topology, reducing redundancy and expanding the range of captured substructures. Applied to two single-hop attention models, HoGA achieves at least a 5% accuracy gain on all benchmark node classification datasets and outperforms recent baselines on six of eight datasets. Code is available at https://github.com/TB862/Higher_Order.

Abstract Anomaly detection in live trajectory data is a critical task for ensuring safety, security, and legality in global transport. Traditional anomaly detection methods often struggle with dynamic and evolving trajectory patterns, especially as systems must adapt to new scenarios over time due to increased traffic, geopolitical events, or global warming. We propose a continual learning approach to detect anomalous activity in moving vessels. Unlike conventional static models, our method leverages continual learning to enable the model to learn from new data continuously and recognise specific behaviours dependent on position and recent movements. We implement an adapter-based framework, Continual Learning for AIS Anomalies (CLAISA), that adapts to shifting behavioural environments in transportation, ensuring the system can identify novel and evolving patterns of anomalies, such as deviations from expected routes, irregular speed changes, or unusual local movements. Evaluations on synthetic maritime trajectory datasets spanning sparsely populated waters and heavily trafficked shipping lanes demonstrate that CLAISA achieves up to a $$32\%$$ decrease in error for trajectory forecasting and consistently outperforms benchmark methods in anomaly detection on synthetically generated datasets.

Low-rank adaptation (LoRA) is an efficient strategy for adapting latent diffusion models (LDMs) on a private dataset to generate specific images by minimizing the adaptation loss. However, the LoRA-adapted LDMs are vulnerable to membership inference (MI) attacks that can judge whether a particular data point belongs to the private dataset, thus leading to the privacy leakage. To defend against MI attacks, we first propose a straightforward solution: Membership-Privacy-preserving LoRA (MP-LoRA). MP-LoRA is formulated as a min-max optimization problem where a proxy attack model is trained by maximizing its MI gain while the LDM is adapted by minimizing the sum of the adaptation loss and the MI gain of the proxy attack model. However, we empirically find that MP-LoRA has the issue of unstable optimization, and theoretically analyze that the potential reason is the unconstrained local smoothness, which impedes the privacy-preserving adaptation. To mitigate this issue, we further propose a Stable Membership-Privacy-preserving LoRA (SMP-LoRA) that adapts the LDM by minimizing the ratio of the adaptation loss to the MI gain. Besides, we theoretically prove that the local smoothness of SMP-LoRA can be constrained by the gradient norm, leading to improved convergence. Our experimental results corroborate that SMP-LoRA can indeed defend against MI attacks and generate high-quality images.

Regular school attendance is critical for young people, supporting academic achievement, social development, and the cultivation of lifelong habits. Existing research for analysing attendance patterns often relies on structured survey data targeted at their parents and teachers, which overlooks students' perspectives and experiences. To address this gap, our team developed and deployed the Our Journey platform, which enables young people to share their experiences through multimodal responses such as texts and images, offering unique insights into the factors influencing school attendance. The data is linked to official attendance records from the Ministry of Education, allowing the modelling of attendance outcomes based on students' input. To effectively analyse the data, we propose Thematic Bottleneck Models (TBMs) to enhance the understanding of subjective experiences behind data and the interpretability of attendance modelling. TBMs introduce qualitative concepts as intermediate labels, mapping multimodal data to qualitative insights from thematic analysis before the outcomes. The attendance modelling with TBMs outperforms existing multimodal methods in predicting attendance percentage and persistent absenteeism. Analysis of themes within TBMs reveals motivational and contextual factors associated with regular attendance and persistent absenteeism. The findings are used to inform education policy and guide strategies to support student engagement in New Zealand.

Vision-language models (VLMs) have demonstrated strong zero-shot inference capabilities but may exhibit stereotypical biases toward certain demographic groups. Consequently, downstream tasks leveraging these models may yield unbalanced performance across different target social groups, potentially reinforcing harmful stereotypes. Mitigating such biases is critical for ensuring fairness in practical applications. Existing debiasing approaches typically rely on curated face-centric datasets for fine-tuning or retraining, risking overfitting and limiting generalisability. To address this issue, we propose a novel framework, CABIN (Causal Adjustment Based INtervention). It leverages a causal framework to identify sensitive attributes in images as confounding factors. Employing a learned mapper, which is trained on general large-scale image-text pairs rather than face-centric datasets, CABIN may use text to adjust sensitive attributes in the image embedding, ensuring independence between these sensitive attributes and image embeddings. This independence enables a backdoor adjustment for unbiased inference without the drawbacks of additional fine-tuning or retraining on narrowly tailored datasets. Through comprehensive experiments and analyses, we demonstrate that CABIN effectively mitigates biases and improves fairness metrics while preserving the zero-shot strengths of VLMs. The code is available at: https://github.com/ipangbo/causal-debias

Recent research has demonstrated the effectiveness of knowledge distillation in Domain Generalization. However, existing approaches often overlook domain-specific knowledge and rely on an offline distillation strategy, limiting the effectiveness of knowledge transfer. To address these limitations, we propose Balanced Online knowLedge Distillation (BOLD). BOLD leverages a multi-domain expert teacher model, with each expert specializing in a specific source domain, enabling the student to distill both domain-invariant and domain-specific knowledge. We incorporate the Pareto optimization principle and uncertainty weighting to balance these two types of knowledge, ensuring simultaneous optimization without compromising either. Additionally, BOLD employs an online knowledge distillation strategy, allowing the teacher and student to learn concurrently. This dynamic interaction enables the teacher to adapt based on student feedback, facilitating more effective knowledge transfer. Extensive experiments on seven benchmarks demonstrate that BOLD outperforms state-of-the-art methods. Furthermore, we provide theoretical insights that highlight the importance of domain-specific knowledge and the advantages of uncertainty weighting.

Identifying individual animals within large wildlife populations is essential for effective wildlife monitoring and conservation efforts. Recent advancements in computer vision have shown promise in animal re-identification (Animal ReID) by leveraging data from camera traps. However, existing Animal ReID datasets rely exclusively on visual data, overlooking environmental metadata that ecologists have identified as highly correlated with animal behavior and identity, such as temperature and circadian rhythms. Moreover, the emergence of multimodal models capable of jointly processing visual and textual data presents new opportunities for Animal ReID, but existing datasets fail to leverage these models' text-processing capabilities, limiting their full potential. To address these limitations, we propose MetaWild, a multimodal Animal ReID dataset comprising 20,890 images across six species, paired with environmental metadata extracted from embedded camera trap overlays and scene contexts. Additionally, to facilitate the use of metadata in existing ReID methods, we propose the Meta-Feature Adapter (MFA), a lightweight module that can be incorporated into existing vision-language model (VLM)-based Animal ReID methods, allowing ReID models to leverage both environmental metadata and visual information to improve ReID performance. Experiments on MetaWild show that combining baseline ReID models with MFA to incorporate metadata consistently improves performance compared to using visual information alone, validating the effectiveness of incorporating metadata in re-identification. We hope that our proposed dataset can inspire further exploration of multimodal approaches for Animal ReID. Our dataset and supplementary materials are available at https://jim-lyz1024.github.io/MetaWild/.

Thematic analysis (TA) is a widely used qualitative method for identifying underlying meanings within unstructured text. However, TA requires manual processes, which become increasingly labour-intensive and time-consuming as datasets grow. While large language models (LLMs) have been introduced to assist with TA on small-scale datasets, three key limitations hinder their effectiveness. First, current approaches often depend on interactions between an LLM agent and a human coder, a process that becomes challenging with larger datasets. Second, with feedback from the human coder, the LLM tends to mirror the human coder, which provides a narrower viewpoint of the data. Third, existing methods follow a sequential process, where codes are generated for individual samples without recalling previous codes and associated data, reducing the ability to analyse data holistically. To address these limitations, we propose Thematic-LM, an LLM-based multi-agent system for large-scale computational thematic analysis. Thematic-LM assigns specialised tasks to each agent, such as coding, aggregating codes, and maintaining and updating the codebook. We assign coder agents different identity perspectives to simulate the subjective nature of TA, fostering a more diverse interpretation of the data. We applied Thematic-LM to the Dreaddit dataset and the Reddit climate change dataset to analyse themes related to social media stress and online opinions on climate change. We evaluate the resulting themes based on trustworthiness principles in qualitative research. Our study reveals insights such as assigning different identities to coder agents promotes divergence in codes and themes.

The decision tree (DT) model is widely usedin various applications due to its versatility, speed, and interpretability. However, outsourcing DT training and inference to cloud platforms raises data privacy concerns. While significant strides have been made in developing private DT training and inference using cryptography such as Secure Multi-Party Computation (MPC), the performance is still not ideal in real-world applications. Only a few recent works have explored using GPUs to enhance the performance of MPC-based deep learning. Nevertheless, data-dependent operations and the high communication costs inherent in MPC-based DT make the integration of GPUs a challenge. We introduce the eXpress GPU-based Tree (XGT), a fast MPC-based framework for private DT training and inference on GPUs.XGTconverts the majority of operations in training and inference into parallelizable matrix operations, supplemented by various optimizations, including matrix dimension reductions. This innovative design leads to substantial reductions in communication overhead while maintaining the critical property of obliviousness.XGTalso achieves a stronger security guarantee, where all data items, the tree shape, access patterns, and data distributions generated during the training and inference are protected.XGTonly reveals the tree depth. The experimental results show thatXGTis up to$278{\times }$faster than the previous most efficient CPU-based approach.XGToutperforms the latest GPU-based DT work by$41{\times }$. For inference,XGTis up to$2,800{\times }$faster than previous CPU-based inference schemes and at least$18 \times$faster than GPU-based.

Global Climate Models (GCMs) simulate low resolution climate projections on a global scale. The native resolution of GCMs is generally too low for societal-level decision-making. To enhance the spatial resolution, downscaling is often applied to GCM output. Statistical downscaling techniques, in particular, are well-established as a cost-effective approach. They require significantly less computational time than physics-based dynamical downscaling. In recent years, deep learning has gained prominence in statistical downscaling, demonstrating significantly lower error rates compared to traditional statistical methods. However, a drawback of regression-based deep learning techniques is their tendency to overfit to the mean sample intensity. Extreme values as a result are often underestimated. Problematically, extreme events have the largest societal impact. We propose Quantile-Regression-Ensemble (QRE), an innovative deep learning algorithm inspired by boosting methods. Its primary objective is to avoid trade-offs between fitting to sample means and extreme values by training independent models on a partitioned dataset. Our QRE is robust to redundant models and not susceptible to explosive ensemble weights, ensuring a reliable training process. QRE achieves lower Mean Squared Error (MSE) compared to various baseline models. In particular, our algorithm has a lower error for high-intensity precipitation events over New Zealand, highlighting the ability to represent extreme events accurately.

Deep learning methods often suffer performance degradation due to domain shift, where discrepancies exist between training and testing data distributions. Domain generalization mitigates this problem by leveraging information from multiple source domains to enhance model generalization capabilities for unseen domains. However, existing domain generalization methods typically present examples to the model in a random manner, overlooking the potential benefits of structured data presentation. To bridge this gap, we propose a novel learning strategy, Symmetric Self-Paced Learning (SSPL), for domain generalization. SSPL consists of a Symmetric Self-Paced training scheduler and a Gradient-based Difficulty Measure (GDM). Specifically, the proposed training scheduler initially focuses on easy examples, gradually shifting emphasis to harder examples as training progresses. GDM dynamically evaluates example difficulty through the gradient magnitude with respect to the example itself. Experiments across five popular benchmark datasets demonstrate the effectiveness of the proposed learning strategy.

Continual Learning (CL) methods enable models to learn new tasks without forgetting previously learned ones. Catastrophic Forgetting (CF) occurs when the parameters of a neural network are updated for a new task, causing the model to lose performance on tasks it has previously learned. To mitigate CF, parameter isolation methods use a “task mask” to allocate a subset of weights to each task; these weights are typically frozen to preserve task performance. However, frozen weights can limit positive backward transfer, which is the beneficial reuse of knowledge from new tasks to improve the accuracy of previously learned tasks. To address this gap, we introduce LEarning AFter learning (LEAF), a novel CL method that enables positive backward transfer by dynamically updating frozen task masks based on gradient updates that signal sufficient backward knowledge transfer. This mechanism allows for selective integration of new knowledge without sacrificing previously acquired knowledge. Our experiments show that LEAF surpasses existing state-of-the-art methods in terms of accuracy while maintaining comparable memory and runtime efficiencies. Moreover, it outperforms other backward transfer techniques in improving the accuracy of a prioritized task. Our code is available at https://github.com/wernse/LEAF.

Continual learning closely emulates human learning, which allows a model to learn from a stream of tasks sequentially without forgetting previously learned knowledge. Replay-based continual learning methods mitigate forgetting and improve performance by reintroducing data belonging to old tasks, however a replay method's performance may deteriorate when the reintroduced data does not effectively represent all experienced data. To address this concern, we propose the Sketch-based Replay Projection (SRP) method to capture and retain the original data stream's distribution within stored memory. SRP augments existing replay frameworks and introduces a two-fold approach. First, we develop a sketch-based sample selection technique to approximate feature distributions within distinct tasks, thereby capturing a wide distribution of examples for subsequent replay. Second, we propose a data compression method which projects examples into a reduced-dimensional space while preserving inter-example relationships and emphasizing inter-class disparities, encouraging diverse representations of each class while maintaining memory requirements similar to existing replay methodologies. Our experimental results demonstrate that SRP enhances replay diversity and improves the performance of existing replay models.

Traditional deep learning models often struggle in few-shot learning scenarios, where limited labeled data is available. While the Contrastive Language-Image Pre-training (CLIP) model demonstrates impressive zero-shot capabilities, its performance in few-shot scenarios remains limited. Existing methods primarily aim to leverage the limited labeled dataset, but this offers limited potential for improvement. To overcome the limitations of small datasets in few-shot learning, we introduce a novel framework, SSAT-Adapter, that leverages CLIP's language understanding to generate informative auxiliary tasks and improve CLIP's performance and adaptability in few-shot settings. We utilize CLIP's language understanding to create decision-boundary-focused image latents. These latents form auxiliary tasks, including inter-class instances to bridge CLIP's pre-trained knowledge with the provided examples, and intra-class instances to subtly expand the representation of target classes. A self-paced training regime, progressing from easier to more complex tasks, further promotes robust learning. Experiments show our framework outperforms the state-of-the-art online few-shot learning method by an average of 2.2% on eleven image classification datasets. Further ablation studies on various tasks demonstrate the effectiveness of our approach to enhance CLIP's adaptability in few-shot image classification.

Outsourcing Decision tree (DT) training and inference to cloud platforms raises privacy concerns. Recent Secure Multi-Party Computation (MPC)-based methods are hindered by heavy overhead. Few recent studies explored GPUs to improve MPC-protected deep learning, yet integrating GPUs into MPC-protected DT with massive data-dependent operations remains challenging, raising question: can MPC-protected DT training and inference fully leverage GPUs for optimal performance?We present GTree, the first scheme that exploits GPU to accelerate MPC-protected secure DT training and inference. GTree is built across 3 parties who jointly perform DT training and inference with GPUs. GTree is secure against semi-honest adversaries, ensuring that no sensitive information is disclosed. GTree offers enhanced security than prior solutions, which only reveal tree depth and data size while prior solutions also leak tree structure. With our oblivious array access, access patterns on GPU are also protected. To harness the full potential of GPUs, we design a novel tree encoding method and craft our MPC protocols into GPU-friendly versions. GTree achieves ~11× and ~21× improvements in training SPECT and Adult datasets, compared to prior most efficient CPU-based work. For inference, GTree outperforms the prior most efficient work by 126× when inferring 104instances with a 7-level tree.

In the face of rising surface temperatures from climate change, impacting biodiversity, extreme weather events, and agricultural productivity, understanding the drivers behind temperature changes is imperative. Traditional global climate models (GCMs) are computationally expensive, limiting their applicability, while machine learning approaches, though promising, face interpretability challenges due to their "black box" nature, especially in a dynamic setting where the data is constantly evolving. We propose DUO, a framework to identify shifts in important features and feature combinations as the data distribution changes over time. Our model independently assesses the importance of features and their interactions while also evaluating their relevance when combined with additional features, contributing to the target class. As a case study, we apply DUO to assess the shifts in climate drivers for station-level temperatures in six locations across New Zealand from 1980 to 2020, we identify specific humidity, geopotential height, and air temperature at high atmospheric pressure levels as the most important features for describing temperature variability. By revealing how climate drivers change over time, DUO contributes to a deeper understanding of temperature change patterns, enabling practitioners to develop targeted and adaptive mitigation strategies.

Abstract In Natural Language Processing (NLP), pre-trained language models (LLMs) are widely employed and refined for various tasks. These models have shown considerable social and geographic biases creating skewed or even unfair representations of certain groups. Research focuses on biases toward L2 (English as a second language) regions but neglects bias within L1 (first language) regions. In this work, we ask if there is regional bias within L1 regions already inherent in pre-trained LLMs and, if so, what the consequences are in terms of downstream model performance. We contribute an investigation framework specifically tailored for low-resource regions, offering a method to identify bias without imposing strict requirements for labeled datasets. Our research reveals subtle geographic variations in the word embeddings of BERT, even in cultures traditionally perceived as similar. These nuanced features, once captured, have the potential to significantly impact downstream tasks. Generally, models exhibit comparable performance on datasets that share similarities, and conversely, performance may diverge when datasets differ in their nuanced features embedded within the language. It is crucial to note that estimating model performance solely based on standard benchmark datasets may not necessarily apply to the datasets with distinct features from the benchmark datasets. Our proposed framework plays a pivotal role in identifying and addressing biases detected in word embeddings, particularly evident in low-resource regions such as New Zealand.

A growing number of tasks require adaptive machine learning systems capable of learning continuously from incoming data and adapting to changes in their environment. In order to enable the widespread adoption of machine learning for streaming data, it is crucial that practitioners and researchers have the tools to efficiently build and evaluate adaptive learning systems. In this paper we demonstrate FALL, a Framework for Adaptive Life-long Learning, which we have developed to enable the full adaptive learning pipeline to be built using modular, reusable components, enabling users to easily and efficiently develop, implement, and evaluate state-of-the-art adaptive learning systems. Source code, documentation, and examples may be found at https://benhalstead.dev/FALL/.

Many software packages and toolkits have been developed for machine learning, in particular for natural language processing and automatic speech recognition. However, there are few software packages designed for emotion recognition. Emotion datasets have diverse structures and annotations, and feature extractors often have different interfaces, which requires writing code specific to each interface. To improve the standardisation and reproducibility of emotion recognition research, we present the Emotion Recognition ToolKit (ERTK), a Python library for emotion recognition. ERTK comprises processing scripts for emotion datasets, standard interfaces to feature extractors, and a framework for defining experiments with declarative configuration files. ERTK is modular and extensible, which allows for easily incorporating additional models and processors. The current version of ERTK focuses on emotional speech, however, the library is modular and can be easily extended to other modalities, which we plan for future releases. ERTK is open-source and available from GitHub: https://github.com/Strong-AI-Lab/emotion.

Outsourcing Machine Learning (ML) tasks to cloud servers is a cost-effective solution when dealing with distributed data. However, outsourcing these tasks to cloud servers could lead to data breaches. Secure computing methods, such as Homomorphic Encryption (HE) and Trusted Execution Environments (TEE), have been used to protect outsourced data. Nevertheless, HE remains inefficient in processing complicated functions (e.g., non-linear functions) and TEE (e.g., Intel SGX) is not ideal for directly processing ML tasks due to side-channel attacks and parallel-unfriendly computation. In this paper, we propose a hybrid framework integrating SGX and HE, called HT2ML, to protect user's data and models. In HT2ML, HE-friendly functions are protected with HE and performed outside the enclave, while the remaining operations are performed inside the enclave obliviously. HT2ML leverages optimised HE matrix multiplications to accelerate HE computations outside the enclave while using oblivious blocks inside the enclave to prevent access-pattern-based attacks. We evaluate HT2ML using Linear Regression (LR) training and Convolutional Neural Network (CNN) inference as two instantiations. The performance results show that HT2ML is up to ∼11× faster than HE only baseline with 6-dimensional data in LR training. For CNN inference, HT2ML is ∼196× faster than the most recent approach (Xiao et al., ICDCS'21).

Phishing is recognized as a serious threat to organizations and individuals. While there have been significant technical advances in blocking phishing attacks, end-users remain the last line of defence after phishing emails reach their email inboxes. Most of the existing literature on this subject has focused on the technical aspects related to phishing. The factors that cause humans to be susceptible to phishing attacks are still not well-understood. To fill this gap, we reviewed the available literature and systematically categorized the phishing susceptibility variables studied. We classify variables based on their temporal scope, which led us to propose a three-stage Phishing Susceptibility Model (PSM) for explaining how humans are vulnerable to phishing attacks. This model reveals several research gaps that need to be addressed to understand and improve protection against phishing susceptibility. Our review also systematizes existing studies by their sample size and generalizability and further suggests a practical impact assessment of the value of studying variables: Some more easily lead to improvements than others. We believe that this article can provide guidelines for future phishing susceptibility research to improve experiment design and the quality of findings.

Learning from streaming data is challenging as the distribution of incoming data may change over time, a phenomenon known as concept drift. The predictive patterns, or experience learned under one distribution may become irrelevant as conditions change under concept drift, but may become relevant once again when conditions reoccur. Adaptive learning methods adapt a classifier to concept drift by identifying which distribution, or concept , is currently present in order to determine which experience is relevant. Identifying a concept requires some representation to be stored for comparison, with the quality of the representation being key to accurate identification. Existing concept representations are based on meta-features, efficient univariate summaries of a concept. However, no single meta-feature can fully represent a concept, leading to severe accuracy loss when existing representations cannot describe concept drift. To avoid these failure cases, we propose the first general framework for combining a diverse range of meta-features into a single representation. We solve two main challenges, first presenting a method of efficiently computing, storing, and querying an arbitrary set of meta-features as a single representation, showing that a combination of meta-features may successfully avoid failure cases seen with existing methods. Second, we present the first method for dynamically learning which meta-features distinguish concepts in any given dataset, significantly improving performance. Our proposed approach enables state-of-the-art feature selection methods, such as mutual information, to be applied to concept representation meta-features for the first time. We investigate tradeoffs between memory budget and classification performance, observing accuracy increases of up to 16% by dynamically weighting the contribution of each meta-feature.

Conditional Density Estimation (CDE) has wide-reaching applicability to various real-world problems, such as spatial density estimation and environmental modelling. CDE estimates the probability density of a random variable rather than a single value and can thus model uncertainty and inverse problems. This task is inherently more complex than regression, and many algorithms suffer from overfitting, particularly when modelled with few labelled data points. For applications where unlabelled data is abundant but labelled data is scarce, we propose Wasserstein Laplacian Regularisation, a semi-supervised learning framework that allows CDE algorithms to leverage these unlabelled data. The framework minimises an objective function which ensures that the learned model is smooth along the manifold of the underlying data, as measured by Wasserstein distance. When applying our framework to Mixture Density Networks, the resulting semi-supervised algorithm can achieve similar performance to a supervised model with up to three times as many labelled data points on baseline datasets. We additionally apply our technique to the problem of remote sensing for chlorophyll-a estimation in inland waters.

Pollution from wood burners has profound health implications for the general population. Typically, monitoring the level of airborne particulate matter, PM2.5, in these areas often requires making inferences about missing or corrupted readings. Air Quality inference in these cases often poses critical challenges. The factors can evolve over time, changing the distribution of data. Such changes in the distribution of data are known as concept drift. Moreover, air pollution inference for a location typically would require historical data to be collected for the location. We investigate five air quality studies in New Zealand rural towns. We explore two different research problems: (1) an adaptive recurrent drift algorithm to model recurrence patterns in PM2.5levels for a town with the ability to recover after accuracy deterioration after a concept drift using an adaptive recurrent drift algorithm, and (2) transfer learning for the data stream whereby we reuse a pre-trained air pollution inference model from a town as the starting point for an air pollution inference model on another town. We further investigate the relationship between the changes we detected and changes within the prediction horizon. We showed that the average accuracy of the air quality inference for the five towns is between 70% and 94% using the recurrent drift algorithm. We also show that transfer learning was advantageous between two of the five towns.

The distribution of streaming data often changes over time as conditions change, a phenomenon known as concept drift. Only a subset of previous experience, collected in similar conditions, is relevant to learning an accurate classifier for current data. Learning from irrelevant experience describing a different concept can degrade performance. A system learning from streaming data must identify which recent experience is irrelevant when conditions change and which past experience is relevant when concepts reoccur, e.g., when weather events or financial patterns repeat. Existing streaming approaches either do not consider experience to change in relevance over time and thus cannot handle concept drift, or only consider the recency of experience and thus cannot handle recurring concepts, or only sparsely evaluate relevance and thus fail when concept drift is missed. To enable learning in changing conditions, we propose SELeCT, a probabilistic method for continuously evaluating the relevance of past experience. SELeCT maintains a distinct internal state for each concept, representing relevant experience with a unique classifier. We propose a Bayesian algorithm for estimating state relevance, combining the likelihood of drawing recent observations from a given state with a transition pattern prior based on the system’s current state. The current state is continuously maintained using a Hoeffding bound based algorithm, which unlike existing methods, guarantees that every observation is classified using the state estimated as the most relevant, while also maintaining temporal stability. We find SELeCT is able to choose experience relevant to ground truth concepts with recall and precision above 0.9, significantly outperforming existing methods and close to a theoretical optimum, leading to significantly higher accuracy and enabling new opportunities for learning in complex changing conditions.

In the online data stream environment, a model is typically not effective until a sufficient number of data instances have been seen. One solution to alleviate this issue is using model transfer. A major gap in current research is that they do not address the cost-effectiveness of model transfer in the online context, where processing time is crucial as data instances continuously arrive at high-speed. Model transfer in data streams involves continuously adapting the model to new data in the target stream. Suppose the target stream contains hard-to-learn patterns not covered by the transferred model. In that case, adaptation may incur extra processing time with little accuracy performance gains compared to simply building a new model for the target stream from scratch. Therefore, considering the cost of model transfer and target model construction is essential to balance the accuracy performance gains against computation cost for cost-effective model transfer. To address this gap, we propose a framework called OPERA (Online Transfer using Phantom Tree for Real-Time Adaptation), that orchestrates transfer learning based on the accuracy gain and runtime tradeoffs between transferring and adapting a source model, versus no transfer and constructing a new model for the target stream. We conduct extensive empirical studies to show that our framework can balance accuracy performance gains and runtime for cost-effective transfer learning in the data stream environment.

Streaming data has become more common as our ability to collect data in real-time increases. A primary concern in dealing with data streams is concept drift, which describes changes in the underlying distribution of streaming data. Measuring drift severity is crucial for model adaptation. Drift severity can be a proxy in choosing concept drift adaptation strategies. Current methods measure drift severity by monitoring the changes in the learner performance or measuring the difference between data distributions. However, these methods cannot measure the drift severity if the ground truth labels are unavailable. Specifically, performance-based methods cannot measure marginal drift, and distribution-based methods cannot measure conditional drift. We propose a novel framework named Tree-based Drift Measurement (TDM) that measures both marginal and conditional drift without revisiting historical data. TDM measures the difference between tree classifiers by transforming them into sets of binary vectors. An experiment shows that TDM achieves similar performance to the state-of-the-art methods and provides the best trade-off between runtime and memory usage. A case study shows that the online learner performance can be improved by adapting different drift adaptation strategies based on the drift severity.

Feedback from software users is vital for engineering better software requirements. One tool for extracting requirements from online user feedback is clustering, where the most mentioned topics are found by grouping similar feedback together. For these topics to be understood, clusters have been summarized in previous work using characterizing phrases or sentences. This work evaluates which method of characterization (unigrams, bigrams, trigrams, or sentences) is most effective for understanding the semantic meaning of a whole cluster using feedback from multiple feedback sources. We evaluate multiple characterization methods to determine the ability of each method to create distinct, descriptive characterizations. We further evaluate the amount of requirements relevant characterizations created by each characterization method. We find that unigrams, bigrams, trigrams, and full sentences all perform similarly in distinguishing clusters from each other. However, we find that fewer and more expressive characterizations, such as full sentences, contain more requirements relevant information from a feedback cluster compared to more numerous but less expressive unigrams, meaning a sentence will better summarize the important requirement relevant information from a cluster. Our findings inform the future development of user feedback clustering tools, with different cluster characterization methods being quantitatively measured for the first time.

In many real-world applications, data are often produced in the form of streams. Consider, for example, data produced by sensors. In data streams there can be concept drift where the distribution of the data changes. When we deal with multiple streams from the same domain, concepts that have occurred in one stream may occur in another. Therefore, being able to reuse knowledge across multiple streams can help models recover from concept drifts more quickly. A major challenge is that these data streams may be only partially identical and a direct adoption would not suffice. In this paper, we propose a novel framework to transfer both identical and partially identical concepts across different streams. In particular, we propose a new technique called Adaptive Online TrAdaBoost that tunes weight adjustments during boosting based on model performance. The experiments on synthetic data verify the desired properties of the proposed method, and the experiments on real-world data show the better performance of the method for data stream mining compared with its baselines.

Data collected over time often exhibit changes in distribution, or concept drift, caused by changes in hidden context relevant to the classification task, e.g. weather conditions. Adaptive learning methods are able to retain performance in changing conditions by explicitly detecting concept drift and changing the classifier used to make predictions. However, in realworld conditions, existing methods often select classifiers which poorly represent current data due to adaptation errors, where change in context is misidentified. We propose the AiRStream system, which uses a novel repair algorithm to identify and correct adaptation errors. We identify errors by periodically testing the performance of inactive classifiers. If an error is identified, a backtracking procedure repairs training done under the misidentified context. AiRStream achieves higher accuracy compared to baseline methods and selects classifiers which better match changes in context. A case study on a real-world air quality inference task shows that AiRStream is able to build a robust model of environmental conditions, allowing the adaptions made to concept drift to be analysed and related to changes in weather.

Streaming sources of data are becoming more common as the ability to collect data in real-time grows. A major concern in dealing with data streams is concept drift, a change in the distribution of data over time, for example, due to changes in environmental conditions. Representing concepts (stationary periods featuring similar behaviour) is a key idea in adapting to concept drift. By testing the similarity of a concept representation to a window of observations, we can detect concept drift to a new or previously seen recurring concept. Concept representations are constructed using meta-information features, values describing aspects of concept behaviour. We find that previously proposed concept representations rely on small numbers of meta-information features. These representations often cannot distinguish concepts, leaving systems vulnerable to concept drift. We propose FiCSUM, a general framework to represent both supervised and unsupervised behaviours of a concept in a fingerprint, a vector of many distinct meta-information features able to uniquely identify more concepts. Our dynamic weighting strategy learns which meta-information features describe concept drift in a given dataset, allowing a diverse set of meta-information features to be used at once. FiCSUM outperforms state-of-the-art methods over a range of 11 real world and synthetic datasets in both accuracy and modeling underlying concept drift.

Catastrophic forgetting is an issue for all continual learning models with neural network architectures, however, research in this area often concentrates on prevention of catastrophic forgetting and not on the potential to improve existing knowledge with new information. In this paper, we explore using smart memory replay to augment a state-of-the-art regularization model. Our method continually corrects model errors for learned tasks, without compromising the learning of new tasks. We demonstrate how our method can achieve higher accuracy of predictions than a standard regularization model on multiple datasets, while incurring similar computational costs. In our experiments we show how using our method gains between five to ten percent accuracy on a large image classification dataset. We also illustrate how our method is able to regain task knowledge after catastrophic forgetting occurs.

Most machine learning algorithms rely on a set of hyperparameters, which are tuned either by domain experts or using automated methods. In the context of data streams, the complexity of this task is compounded by the need for tuning every time a concept drift occurs. The problem of automated tuning may lead to classification performance improvements. Current methods optimise the hyperparameters from scratch every time concept drift is detected. Additionally, meta-learning techniques developed for static machine learning tasks allow pre-built knowledge to adapt faster to new tasks. We propose a meta-learning approach for automated tuning in evolving data streams, specifically for tuning parameters in Adaptive Random Forest (ARF) and parameters for several drift detectors. We compare our approach against non-adaptive methods on both synthetic and real-world datasets. Empirical results show that our approach makes a compromises between predictive performance and resource consumption for ARF, and between true positives and false positives for drift detectors.

Concept drift detection is used to signal to a learning algorithm that there has been a change in the underlying distribution of the data stream. However, there is a delay in detecting the actual drifts, leading to performance loss between the start of the drift and the detection point. There are two major challenges in reducing such performance loss, specifically the difficulty in anticipating the location of the next drift point and determining the exact concept that will appear for timely concept adaptation. In this research, we leverage concept recurrences in data streams. We proposed a framework called Nacre, which can perform proactive drift detection and online updates to allow for smooth adaptation of concept drifts. We present a novel technique, called drift coordinator, that anticipates the next drift point and assesses the incoming concept. This will ultimately increase accuracy in the classification performance. We demonstrate that our method is able to learn and predict drift trends in streams with recurring drifts. This allows the anticipation of future changes which enables users and detection methods to be more proactive. We empirically show that our technique outperforms baselines in terms of accuracy, kappa, accuracy gain per drift and cumulative accuracy gain on both synthetic and real-world datasets.

Clinical narratives host vast accumulations of patient data pivotal for research and development of health related products. In order for this data to be utilized, the underlying protected health information needs to be de-identified to ensure medical confidentiality. Given the voluminous size of clinical texts, manual de-identification of such large datasets is both expensive and impractical. Therefore, the concept of automated de-identification is a highly appealing prospect. Machine learning or model based sequential labeling algorithms, such as the named entity recognition algorithms and rule-based algorithms are among the most effective approaches to automated de-identification. A natural question to ask is how we can combine them to have the best of both worlds. In this paper, we present an analytical and easy to interpret framework to dynamically combine a sequential labeling model and a soft-rule-based model in an incremental learning setup. This framework is applied to a case study, which is part of a project prototyping automated de-identification system for New Zealand clinical free text data. Evaluations show that our approach can accommodate changes in the incoming data through dynamic updating. The simplicity of the framework also allowed us to gain insights on behaviour e.g. change of importance between the machine learning and rule models.

Concept drift detection refers to the process of detecting changes in the underlying distribution of data. Interest in the data stream mining community has increased, because of their role in improving the performance of online learning algorithms. Over the years, a myriad of drift detection methods have been proposed. However, most of these methods are single detectors, which usually work well only with a single type of drift. In this research, we propose a semi-supervised locally-weighted ensemble detector (SLED), where the relative performance among its base detectors is characterized by a set of weights learned in a semi-supervised manner. The aim of this technique is to effectively deal with both abrupt and gradual concept drifts. In our experiments, SLED is configured with ten well-known drift detectors. To evaluate the performance of SLED, we compare it with single detectors as well as state-of-the-art ensemble methods on both synthetic and real-world datasets using different performance measures. The experimental results show that SLED has fewer false positives, higher precision, and higher Matthews correlation coefficient while maintaining reasonably good performance for other measures.

Random forest is an ensemble method used to improve the performance of single tree classifiers. In evolving data streams, the classifier needs to be adaptive and work under constraints of space and time. One benefit of random forest is its ability to be executed in parallel. In our research we introduce a random forest model utilizing a hybrid of both GPU and CPU, called GPU-based State-Adaptive Random Forest (GSARF). We address the pre-existing challenges of adapting random forest for data streams, specifically in the area of continual learning. Our novel approach reuses previously seen trees in the random forest when previous concepts reappear. This allows us to retain prior knowledge and provide a more stable predictive accuracy when changes occur in the data stream. Our random forest for data streams stores three types of trees, foreground trees which are trees that are currently used in prediction, background trees which are trees that are built when we are aware of possible changes in the data streams, and candidate trees which are trees that had been highly used in the previous concepts, but are now discarded due to changes in the data stream. We store candidate trees as they may be potentially useful at a later period in a repository and can be accessed when needed. We empirically show our technique performs up to 138 times the speed compared to current CPU-based random forest benchmarks. Our approach has shown to outperform a baseline GPU-based approach in terms of cumulative accuracy performance.

in their accompanying referral documents, which contain a mix of free text and structured data. By training a model to predict triage decisions from these referral documents, we can partially automate the triage process, resulting in more efficient and systematic triage decisions. One of the difficulties of this task is maintaining robustness against changes in triage priorities due to changes in policy, funding, staff, or other factors. This is reflected as changes in relationship between document features and triage labels, also known as concept drift. These changes must be detected so that the model can be retrained to reflect the new environment. We introduce a new concept drift detection algorithm for this domain called calibrated drift detection method (CDDM). We evaluated CDDM on benchmark and synthetic medical triage datasets, and find it competitive with state-of-the-art detectors, while also being less prone to false positives from feature drift.

Learning a sequence of tasks is a long-standing challenge in machine learning. This setting applies to learning systems that observe examples of a range of tasks at different points in time. A learning system should become more knowledgeable as more related tasks are learned. Although the problem of learning sequentially was acknowledged for the first time decades ago, the research in this area has been rather limited. Research in transfer learning, multitask learning, metalearning and deep learning has studied some challenges of these kinds of systems. Recent research in lifelong machine learning and continual learning has revived interest in this problem. We propose Proficiente, a full framework for long-term learning systems. Proficiente relies on knowledge transferred between hypotheses learned with Support Vector Machines. The first component of the framework is focused on transferring forward selectively from a set of existing hypotheses or functions representing knowledge acquired during previous tasks to a new target task. A second component of Proficiente is focused on transferring backward, a novel ability of long-term learning systems that aim to exploit knowledge derived from recent tasks to encourage refinement of existing knowledge. We propose a method that transfers selectively from a task learned recently to existing hypotheses representing previous tasks. The method encourages retention of existing knowledge whilst refining. We analyse the theoretical properties of the proposed framework. Proficiente is accompanied by an agnostic metric that can be used to determine if a long-term learning system is becoming more knowledgeable. We evaluate Proficiente in both synthetic and real-world datasets, and demonstrate scenarios where knowledgeable supervised learning systems can be achieved by means of transfer.

Semi-supervised learning usually assumes the distribution of the unlabelled data to be the same as that of the labelled data. This assumption does not always hold in practice. We empirically show that unlabelled data containing novel examples and classes from outside the distribution of the labelled data can lead to a performance degradation for semi-supervised learning algorithms. We propose a 1-nearest-neighbour based method to assign a weight to each unlabelled example in order to reduce the negative effect of novel classes in unlabelled data. Experimental results on MNIST, Fashion-MNIST and CIFAR-10 datasets suggest that the negative effect of novel classes becomes statistically insignificant when the proposed method is applied. Using our proposed technique, models trained on unlabelled data with novel classes can achieve similar performance as ones trained on clean unlabelled data.

Selective transfer has been proposed as an alternative for transferring fragments of knowledge. Previous work showed that transferring selectively from a group of hypotheses helps to speed learning on a target task. Similarly, existing hypotheses could benefit by selective backward transfer of recent knowledge. This setting applies to supervised machine learning systems that observe a sequence of related tasks. We propose a novel scheme for bi-directional transfer between hypotheses learned sequentially using Support Vector Machines. Transfer occurs in two directions: forward and backward. During transfer forward, a new binary classification task is to be learned. Existing knowledge is used to reinforce the importance of subspaces on the target training data that are related to source support vectors. While this target task is learned, subspaces of shared knowledge between each source hypothesis and the target hypothesis are identified. Representations of these subspaces are learned and used to refine the sources by transferring backward. Albeit fundamental, the exploration of the problem of hypothesis refinement has been very limited. We define this problem and propose a solution. Our experiments show that a learning system can gain up to 5.5 units in mean classification accuracy of tasks learned sequentially using our scheme, within 26.6% of the number of iterations when these tasks are learned from scratch.

Network embedding learns the vector representations of nodes. Most real world networks are heterogeneous and evolve over time. There are, however, no network embedding approaches designed for dynamic heterogeneous networks so far. Addressing this research gap is beneficial for analyzing and mining real world networks. We develop a novel representation learning method, change2vec, which considers a dynamic heterogeneous network as snapshots of networks with different time stamps. Instead of processing the whole network at each time stamp, change2vec models changes between two consecutive static networks by capturing newly-added and deleted nodes with their neighbour nodes as well as newly-formed or deleted edges that caused core structural changes known as triad closure or open processes. Change2vec leverages metapath based node embedding and change modeling to preserve both heterogeneous and dynamic features of a network. Experimental results show that change2vec outperforms two state-of-the-art methods in terms of clustering performance and efficiency.

The reasons for concept drift in a data stream can vary widely, from deterioration of a machine to a change in peoples' buying patterns. In order to effectively detect concept drifts, most predictive stream mining systems contain a drift detector that monitors and signals concept drifts. However, few of these systems are designed to find drifts in transactional datasets, which have unlabelled data. Transactional datasets describe events, such as orders or payments, which are traditionally analysed using association rules. In this paper, we propose a novel drift detection technique, ProChange, that has two parts. The first part is a drift detector, VR-Change, that finds both real and virtual drifts in unlabelled transactional data streams using the Hellinger distance. The second part is a drift predictor, which models the volatility of drifts using a probabilistic network to predict the location of future drifts. Using the predictor, we can dynamically adapt the confidence threshold, enabling VR-Change to be more sensitive around potential future drift points. We evaluated the performance of ProChange by comparing it against traditional detectors showing that it detects both real and virtual drifts effectively and efficiently in terms of accuracy.

Concept drifts cause data streams to change over time, making classification harder. Drift detection signals when change occurs so remedial action can be taken. However, existing approaches do not allow drift detectors to improve drift detection by using patterns associated with drift across multiple stream metadata. Metadata Drift Predictor (MDP) is a novel framework which allows this by adapting drift sensitivity mid-stream to reduce errors in drift detection. MDP clusters metadata patterns at drift points and uses recurring patterns to encourage drift detection where metadata looks similar to past drift points and/or discourage drift detection where metadata looks different from past drift points. It avoids relying on statistical assumptions of underlying drift patterns. Our experiments demonstrate that MDP works consistently across traditional stream classifiers and drift detectors to reduce delay and false positive drift detection and/or reduce false negative drift detection. We show that MDP can be used to reduce drifts detected in real-world datasets while improving the accuracy that each detected drift provides when compared to other popular meta-learning frameworks.

In our research, we consider transfer learning scenarios where a target learner does not have access to the source data, but instead to hypotheses or models induced from it. This is called the Hypothesis Transfer Learning (HTL) problem. Previous approaches concentrated on transferring source hypotheses as a whole. We introduce a novel method for selectively transferring elements from previous hypotheses learned with Support Vector Machines. The representation of an SVM hypothesis as a set of support vectors allows us to treat this information as privileged to aid learning during a new task. Given a possibly large number of source hypotheses, our approach selects the source support vectors that more closely resemble the target data, and transfers their learned coefficients as constraints on the coefficients to be learned. This strategy increases the importance of relevant target data points based on their similarity to source support vectors, while learning from the target data. Our method shows important improvements on the convergence rate on three classification datasets of varying sizes, decreasing the number of iterations by up to 56% on average compared to learning with no transfer and up to 92% compared to regular HTL, while maintaining similar accuracy levels.

The application of current drift detection methods to real data streams show trends in the rate of change found by the detectors. We observe that these patterns of change vary across different data streams. We use the term stream volatility pattern to describe change rates with a distinct mean and variance. First, we propose a novel drift prediction algorithm to predict the location of future drift points based on historical drift trends which we model as transitions between stream volatility patterns. Our method uses a probabilistic network to learn drift trends and is independent of the drift detection technique. We demonstrate that our method is able to learn and predict drift trends in streams with reoccurring stream volatility patterns. This allows the anticipation of future changes which enables users and detection methods to be more proactive. Second, we apply our drift prediction algorithm by incorporating the drift estimates into a drift detector, ProSeed, to improve its performance by decreasing the false positive rate.

Online mining is a difficult task especially when such data streams evolve over time. Evolving data stream occurs when concepts drift or change completely, is becoming one of the core issues. A large portion of change detection research are carried out in the area of supervised learning, very little has been carried out for unlabeled data specifically in the area of transactional data streams. Overall when we monitor changes in transactional data we can consider two different types of changes: local and global change. Local changes are changes in distribution of the data, whereas global changes are data composition changes within the data stream. To detect changes in transactional data streams containing unlabeled data, we introduce a new technique called CD-TDS, that detects both these changes. Our change detector can identifies changes in relationships between items as data evolves with the progression of a stream. Crucially, detection of global drift enables us to better understand the dynamics in relationships that takes place over time. Experimental results using both real world and synthetic data show that the proposed approach is robust to noise and identifies structural changes with a high true positive rate while preserving a low false alarm rate.

Association rule mining was first introduced to examine patterns among frequent items. The original motivation for seeking these rules arose from need to examine customer purchasing behaviour in supermarket transaction data. It seeks to identify combinations of items or itemsets, whose presence in a transaction affects the likelihood of the presence of another specific item or itemsets. In recent years, there has been an increasing demand for rare association rule mining. Detecting rare patterns in data is a vital task, with numerous high-impact applications including medical, finance, and security. This survey aims to provide a general, comprehensive, and structured overview of the state-of-the-art methods for rare pattern mining. We investigate the problems in finding rare rules using traditional association rule mining. As rare association rule mining has not been well explored, there is still specific groundwork that needs to be established. We will discuss some of the major issues in rare association rule mining and also look at current algorithms. As a contribution, we give a general framework for categorizing algorithms: Apriori and Tree based. We highlight the differences between these methods. Finally, we present several real-world application using rare pattern mining in diverse domains. We conclude our survey with a discussion on open and practical challenges in the field.

Optimization based techniques have emerged as important methods to tackle the problems of efficiency and accuracy in data mining. One of the current application areas is outlier detection that has not been fully explored yet but has enormous potential. Web bots are an example of outliers, which can be found in the web usage analysis process. Web bot requests are different from a genuine web user as web bots crawl large numbers of pages in a very short time. If web bots remains undetected they can skew the analysis process which can result in incorrect patterns that can cause wrong decisions. In this paper we use one of the popular Swarm Intelligence (SI) based techniques called Particle Swarm Optimization (PSO) to detect web bots among genuine user requests. We use our Particle Swarm Optimization (PSO) based clustering algorithm, Hierarchical Particle Swarm Optimization based clustering (HPSO-clustering) to cluster the web-usage data and detect the abnormal behaviour caused by the web bots. We present the results of detection which shows that our proposed approach is capable of detecting such abnormal behaviour. We then compare the characteristics of the detected web bots with genuine web-users using cross validation.

Current drift detection techniques detect a change in distribution within a stream. However, there are no current techniques that analyze the change in the rate of these detected changes. We coin the term stream volatility, to describe the rate of changes in a stream. A stream has a high volatility if changes are detected frequently and has a low volatility if changes are detected infrequently. We are particularly interested in a volatility shift which is a change in the rate of change (e.g. From high volatility to low volatility). We introduce and define the concept of stream volatility, and propose a novel technique to detect volatility on data streams in the presence of concept drifts. In the experiments we show our algorithm to be both fast and efficient. We also propose a new algorithm for drift detection called SEED that is faster and more memory efficient than the existing state-of-the-art drift detection approach. A faster drift detection algorithm has a flow-on benefit to the subsequent volatility detection stage because both algorithms run concurrently on the data stream.

Recommender systems are highly vulnerable to attacks. Attackers who introduce biased ratings in order to affect recommendations, have been shown to be effective against collaborative filtering algorithms. In this paper, we study the use of statistical metrics to detect rating patterns of attackers. Two metrics, Rating Deviation from Mean Agreement (RDMA) and Degree of Similarity with Top Neighbors (DegSim), are used for analysing rating patterns between malicious profiles and genuine profiles in shilling attacks. Building upon this, we propose and evaluate an algorithm for detecting shilling attacks in recommender systems using a statistical approach. We look at two attack models: random attack and average attack. The experimental results show that our detection technique based on target item analysis is an effective approach in detecting shilling attacks for both the random and average attack model.

Recommender systems using Collaborative Filtering techniques are capable of make personalized predictions. However, these systems are highly vulnerable to profile injection attacks. Group attacks are attacks that target a group of items instead of one, and there are common attributes among these items. Such profiles will have a good probability of being similar to a large number of user profiles, making them hard to detect. We propose a novel technique for identifying group attack profiles which uses an improved metric based on Degree of Similarity with Top Neighbors (DegSim) and Rating Deviation from Mean Agreement (RDMA). We also extend our work with a detailed analysis of target item rating patterns. Experiments show that the combined methods can improve detection rates in user-based recommender systems.

Typically association rule mining only considers positive frequent itemsets in rule generation, where rules involving only the presence of items are generated. In this paper we consider the complementary problem of negative association rule mining, w

Recommender systems have become one of the necessary tools to help a web user find a potentially interesting resource based on their preferences. In implicit recommender systems, the recommendations are made based on the implicit information of the w

Data clustering aims to group data based on similarities between the data elements. Recently, due to the increasing complexity and amount of heterogenous data, modeling of such data for clustering has become a serious challenge. In this paper we tackle the problem of modeling heterogeneous web usage data for clustering. The main contribution is a new similarity measure which we propose to cluster heterogeneous web usage data. We then use this similarity measure in our Particle Swarm Optimization (PSO) based clustering algorithm, Hierarchical Particle Swarm Optimization based clustering (HPSO-clustering). HPSO-clustering combines the qualities of hierarchical and partitional clustering to cluster data in a hierarchical agglomerative manner. We present the clustering results and explain the effects of the new similarity measure on inter-cluster and intra-cluster distances. These measures verify the applicability of the proposed similarity measure on web usage data.

Due to a marked increase in the number of web users and their activities, many application areas that use patterns generated from their activities has been proposed. Web-based implicit recommender systems are one such application. An implicit recommender system is a tool that helps guide a user to a particular web resource based on implicit data. Implicit data comes from the web users activities without their active participation. Building such a system is a complex process due to two reasons, there is a huge amount of data and the quality of the data is poor. In this research, we tackle the first problem of generating patterns efficiently for recommender system by proposing Hierarchical Particle Swarm Optimization based clustering (HPSO-clustering). HPSO-clustering is a clustering approach based on Particle Swarm Optimization which combines both the properties of hierarchical and partitional clustering. We grouped the users' session into different clusters. Recommendations for an active user are generated from these clusters. In this paper we report the results of accuracy of recommendations. We achieved an overall 60% to 65% of precision for an active user, while in some clusters the precision achieved was 100% when top 5 ranked recommendations were selected.