Phishing attacks exploit a variety of hosting and domain ownership models, making them a dynamic and challenging threat to address. Many existing proactive detection methods focus on identifying newly registered domains, often relying on data sources like Certificate Transparency (CT) logs and Domain Name System (DNS) records. Although effective, these approaches capture only part of the attacks, leaving unanswered questions about the proportion of threats they can detect. To answer this question, we conducted a year-long study, collecting phishing reports from multiple sources and using a machine-learning classifier to analyze phishing domains. As a result of this study, we introduce two new datasets—PhishXtract and PhishXtract-Class—designed to support and advance future research in this area. By analyzing features such as the Internet presence and historical activity, we classified these domains into three types: attacker-owned, compromised, and third-party hosted. This helped us build a clearer framework for understanding the phishing domain ownership patterns observed during the year-long study. Our findings show that most phishing attacks are not hosted on attacker-owned domains. Instead, attackers often abuse legitimate infrastructures and third-party platforms to create their campaigns. Furthermore, by analyzing the reporting feeds, we were able to assess their unique and overlapping contributions to phishing detection and identify the platforms most frequently exploited by attackers. Overall, our findings provide an understanding of current phishing strategies, highlighting the reliance on legitimate infrastructures over attacker-owned domains and emphasizing the need for targeted detection strategies tailored to different hosting models exploited by attackers.

Phishing attacks have evolved, increasingly leveraging legitimate platforms to bypass traditional domain and URL-based detection methods. Existing proactive approaches such as monitoring Domain Name System (DNS) anomalies and Certificate Transparency (CT) logs have become less effective, as attackers now exploit trusted websites and services. To address this challenge, we present TelePhish, a dataset of annotated Telegram messages collected from public groups in high-risk categories such as Cryptocurrency, Games, and Darknet. Unlike prior works, TelePhish emphasizes the propagational behavior of phishing by capturing Telegram group interactions, message patterns, and user engagement, rather than relying on easily manipulated textual or URL features. Using this dataset, we investigate how modeling granularity impacts phishing detection performance by adopting a macro-to-micro approach. We start from a general model that captures broad phishing patterns across Telegram, then progressively narrow to more specialized, category-aware models. We evaluate four modeling strategies: (i) a general model trained on all data, (ii) general-cluster models that segment the dataset via clustering and then train models, (iii) category models trained on each category, and (iv) categorycluster models that discover sub-patterns within each category. We show that localized models can outperform general models in capturing category-specific patterns. These findings provide guidelines for designing detection models that can be tailored to either capture general phishing patterns across the entire platform or address the unique characteristics of individual categories.

As Large Language Models (LLMs) evolve in understanding and generating code, accurately evaluating their reliability in analyzing source code vulnerabilities becomes in-creasingly vital. While studies have examined LLM capabilities in tasks like vulnerability detection and repair, they often over-look the importance of both structure and semantic reasoning crucial for trustworthy vulnerability analysis. To address this gap, we introduce SV-TRUSTEVAL-C, a benchmark designed to evaluate LLMs' abilities for vulnerability analysis of code written in the C programming language through two key di-mensions: structure reasoning-assessing how models identify relationships between code elements under varying data and control flow complexities; and semantic reasoning-examining their logical consistency in scenarios where code is structurally and semantically perturbed. Our results show that current LLMs are far from satisfactory in understanding complex code relationships and that their vulnerability analyses rely more on pattern matching than on robust logical reasoning. These findings underscore the effectiveness of the SV-TRUSTEVAL-C benchmark and highlight critical areas for enhancing the reasoning capabilities and trustworthiness of LLMs in real-world vulnerability analysis tasks. Our ini-tial benchmark dataset is available at https://huggingface.co/datasets/LLMs4CodeSecurity/SV-TrustEval-C-1.0

Globally, cyberattacks are growing and mutating each month. Intelligent Intrusion Network Detection Systems are developed to analyze and detect anomalous traffic to face these threats. A way to address this is by using network flows, an aggregated version of communications between devices. Network Flow datasets are used to train Artificial Intelligence (AI) models to classify specific attacks. Training these models requires threat samples usually generated synthetically in labs as capturing them on operational network is a challenging task. As threats are fast-evolving, new network flows are continuously developed and shared. However, using old datasets is still a popular procedure when testing models, hindering a more comprehensive characterization of the advantages and opportunities of recent solutions on new attacks. Moreover, a standardized benchmark is missing rendering a poor comparison between the models produced by algorithms. To address these gaps, we present a benchmark with fourteen recent and preprocessed datasets and study seven categories of algorithms for Network Intrusion Detection based on Network Flows. We provide a centralized source of pre-processed datasets to researchers for easy download. All dataset are also provided with a train, validation and test split to allow a straightforward and fair comparison between existing and new solutions. We selected open state-of-the-art publicly available algorithms, representatives of diverse approaches. We carried out an experimental comparison using the Macro F1 score of these algorithms. Our results highlight each model operation on dataset scenarios and provide guidance on competitive solutions. Finally, we discuss the main characteristics of the models and benchmarks, focusing on practical implications and recommendations for practitioners and researchers.

Growth in system complexity increases the need for automated techniques dedicated to different log analysis tasks such as Log-based Anomaly Detection (LAD). The latter has been widely addressed in the literature, mostly by means of a variety of deep learning techniques. However, despite their many advantages, that focus on deep learning techniques is somewhat arbitrary as traditional Machine Learning (ML) techniques may perform well in many cases, depending on the context and datasets. In the same vein, semi-supervised techniques deserve the same attention as supervised techniques since the former have clear practical advantages. Further, current evaluations mostly rely on the assessment of detection accuracy. However, this is not enough to decide whether or not a specific ML technique is suitable to address the LAD problem in a given context. Other aspects to consider include training and prediction times as well as the sensitivity to hyperparameter tuning, which in practice matters to engineers. In this paper, we present a comprehensive empirical study, in which we evaluate a wide array of supervised and semi-supervised, traditional and deep ML techniques w.r.t. four evaluation criteria: detection accuracy, time performance, sensitivity of detection accuracy and time performance to hyperparameter tuning. Our goal is to provide much stronger and comprehensive evidence regarding the relative advantages and drawbacks of alternative techniques for LAD. The experimental results show that supervised traditional and deep ML techniques fare similarly in terms of their detection accuracy and prediction time on most of the benchmark datasets considered in our study. Moreover, overall, sensitivity analysis to hyperparameter tuning with respect to detection accuracy shows that supervised traditional ML techniques are less sensitive than deep learning techniques. Further, semi-supervised techniques yield significantly worse detection accuracy than supervised techniques.

Federated Learning (FL) enables collaborative training of Machine Learning (ML) models while maintaining user data privacy. However, leaked model updates can reveal private training data. Existing solutions using additive secret sharing introduce intermediary servers, increasing complexity and communication overhead, and often lack privacy guarantees. We propose AddShare+, which enhances efficiency and scalability by creating additive shares for a subset of model weight parameters and using the Elliptic Curve Integrated Encryption Scheme (ECIES) for faster, lighter model encryption. By sampling and splitting a percentage of local weight parameters, AddShare+ reduces computation and communication costs while maintaining model accuracy. We implemented and evaluated AddShare+ on multiple datasets, comparing it with baseline approaches including FedAvg, SCOTCH, FedShare, and AddShare. Results demonstrate that AddShare+ maintains accuracy while significantly reducing running time per round. Notably, sharing as low as 25% of model weights decreases bandwidth demands by over 5x while preserving accuracy within 0.05 % of the full model. Our empirical results demonstrate significant reductions in running time per round with strong privacy guarantees, highlighting the potential of lightweight partial sharing solutions for privacy-preserving FL in resource-constrained environments, paving the way for more efficient and secure collaborative learning systems.

The Bitcoin generator scam is one example of existing deceptive schemes enticing users with promises of free or effortless Bitcoin generation. These scams predominantly exploit individuals unfamiliar with cryptocurrency seeking low-effort avenues to obtain Bitcoin without financial investment. In this paper, we propose and analyze methods to improve the performance of Graph Neural Networks (GNNs) in detecting fraudulent cases within Bitcoin transactional data. We explore multiple GNN variants, alongside various graph sampling methodologies. To overcome the shortcomings of these sampling methods, we propose a new sampling method BFRON—a hybrid approach mixing Breadth-First Search and Frontier Sampling. Additionally, we introduce an enhanced optimization pipeline and a new metric to improve fraudulent node detection. Evaluation metrics, including Instance Information Gain and Group Distance Ratio, are employed to analyze the challenges of over-smoothing in Graph Neural Networks and the efficacy of diverse graph sampling techniques. Our results show that overall BFRON is the best solution and RGGCN is the best-performing GNN. Moreover, we show that our enhanced pipeline and the usage of graph normalization have important advantages.

Diffusion models represent state-of-the-art deep learning architectures behind many popular and powerful image-synthesizing generative Artificial Intelligence (AI) systems. Their underlying approach, which relies on scheduled noise addition and optimized noise removal, has been applicable to the syn-thesis of sophisticated and nearly photorealistic images across various domains; however, the potential impacts of compro-mised diffusion models remain an underexplored area in the literature. While prior studies have investigated the insertion of explicit triggers into the noise or prompt spaces of diffusion models, it is unrealistic to assume that benign users would intentionally input such triggers into their own diffusion process. In our DevilDiffusion approach, we demonstrate the capability to surreptitiously embed triggers that leverage uncommon but naturally -occurring characteristics of Gaussian noise directly into the noise space of conditional diffusion models. When these specific characteristics occur in naturally -occurring noise, the model will instead construct our target backdoor image. By adjusting the trigger size and the ratio of poisoned images, we can control the trigger rates of the specified target image, ranging from less than 0.01 % to 25 % of all generated images, while still maintaining performance on the target task.

Abstract Anti-Asian speech during the COVID-19 pandemic has been a serious problem with severe consequences. A hate speech wave swept social media platforms. The timely detection of Anti-Asian COVID-19-related hate speech is of utmost importance, not only to allow the application of preventive mechanisms but also to anticipate and possibly prevent other similar discriminatory situations. In this paper, we address the problem of detecting Anti-Asian COVID-19-related hate speech from social media data. Previous approaches that tackled this problem used a transformer-based model, BERT/RoBERTa, trained on the homologous annotated dataset and achieved good performance on this task. However, this requires extensive and annotated datasets with a strong connection to the topic. Both goals are difficult to meet without employing reliable, vast, and costly resources. In this paper, we propose a robust semi-supervised model, SSL-GAN-RoBERTa, that learns from a limited heterogeneous dataset and whose performance is further enhanced by using vast amounts of unlabeled data from another related domain. Compared with the RoBERTa baseline model, the experimental results show that the model has substantial performance gains in terms of Accuracy and Macro-F1 score in different scenarios that use data from different domains. Our proposed model achieves state-of-the-art performance results while efficiently using unlabeled data, showing promising applicability to other complex classification tasks where large amounts of labeled examples are difficult to obtain.

Abstract House price prediction is an important problem that could benefit home buyers and sellers. Traditional models for house price prediction use numerical attributes such as the number of rooms but disregard the house description text. The recent developments in text processing suggest these can be valuable attributes, which motivated us to use house descriptions. This paper focuses on the house asking/advertising price and studies the impact of using house description texts to predict the final house price. To achieve this, we collected a large and diverse set of attributes on house postings, including the house advertising price. Then, we compare the performance of three scenarios: using only the house description, only numeric attributes, or both. We processed the description text through three word embedding techniques: TF-IDF, Word2Vec, and BERT. Four regression algorithms are trained using only textual data, non-textual data, or both. Our results show that by using exclusively the description data with Word2Vec and a Deep Learning model, we can achieve good performance. However, the best overall performance is obtained when using both textual and non-textual features. An $R^2$ of 0.7904 is achieved by the deep learning model using only description data on the testing data. This clearly indicates that using the house description text alone is a strong predictor for the house price. However, when observing the RMSE on the test data, the best model was gradient boosting using both numeric and description data. Overall, we observe that combining the textual and non-textual features improves the learned model and provides performance benefits when compared against using only one of the feature types. We also provide a freely available application for house price prediction, which is solely based on a house text description and uses our final developed model with Word2Vec and Deep Learning to predict the house price.

Federated Learning (FL) promises to offer a major paradigm shift in the way deep learning models are trained at scale, yet malicious clients can surreptitiously embed backdoors into models via trivial augmentation on their own subset of the data. This is especially true in small- and medium-scale FL systems, which consist of dozens, rather than millions, of clients. In this work, we investigate a novel attack scenario for an FL architecture consisting of multiple non-i.i.d. silos of data in which each distribution has a unique backdoor attacker and where the model convergences of adversaries are not more similar than those of benign clients. We propose a new method, dubbed Haywire, as a security-in-depth approach to respond to this novel attack scenario. Our defense utilizes a combination of kPCA dimensionality reduction of fully-connected layers in the network, KMeans anomaly detection to drop anomalous clients, and server aggregation robust to outliers via the Geometric Median. Our solution prevents the contamination of the global model despite having no access to the backdoor triggers. We evaluate the performance of Haywire from model-accuracy, defense-performance, and attack-success perspectives against multiple baselines. Through an extensive set of experiments, we find that Haywire produces the best performances at preventing backdoor attacks while simultaneously not unfairly penalizing benign clients. We carried out additional in-depth experiments across multiple runs that demonstrate the reliability of Haywire.

A Bitcoin Generator Scam (BGS) is a type of cyberattack in which scammers promise to provide individuals with free cryptocurrencies if they pay a mining fee. Although graph neural networks (GNNs) have been used for detecting other cryptocurrency frauds, the usefulness of these methods for BGS detection has not been studied. In this paper, we carry out extensive experiments to assess the use of both standard machine learning (ML) methods and GNNs to detect Bitcoin transactions associated with activities stemming from Bitcoin Generator Scams. We observe that the over-smoothing problem exists in GNNs designed for BGS detection and show that Random Walk Positional Encoding (RWPE) allows representing long-range interactions between far-away transactions in GNNs without causing over-smoothing. We show that the General, Powerful, Scalable (GPS) Graph Transformer with RWPE outperforms both GNN and ML based state-of-the-art fraud detection methods in Bitcoin Generator Scams. We also analyze the effectiveness of Breadth First Search (BFS) for graph sampling and show that it should not be used as it induces bias toward the subnetwork structure. We propose the Random First Search (RFS) sampling alternative and show that this is a more suitable solution.

Input cost reductions and yield optimization can be done using yield precision maps created by machine learning models to address the increase in food demand predicted by 2050. However, without taking into account the spatial structure of the data, the precision map’s accuracy evaluation assessment runs the risk of being overly optimistic. In the current work, a corn yield prediction study was conducted, and the predictive abilities of two vegetation indices (VIs) and five spectral bands for a single image acquisition date were evaluated. We also examined the impacts of image spatial and spectral resolution on model performance. We used a Canadian smart farm’s yield data, multispectral (MS) and red-green-blue (RGB) imagery captured by unmanned aerial vehicles (UAVs), and we trained deep neural networks (DNN), random forest (RF), and linear regression (LR) models using standard cross-validation and spatial cross-validation approaches. We found that multi-band datasets led to better performance than single-VI datasets. MS imagery led to generally better performance than RGB imagery. High spatial resolution imagery led to better performance than lower spatial resolution imagery. RF was the best performing model while LR was the worst. The choice of RF’s hyperparameters had more of an impact on performance when the number of features was small and less of an impact when the number of features was large or when the input dataset had a lot of spatial structure.

The class imbalance problem is associated with harmful classification bias and presents itself in a wide variety of important applications of supervised machine learning. Measures have been developed to determine the imbalance complexity of datasets with imbalanced classes. The most common such measure is the Imbalance Ratio (IR). It is, however, widely accepted that the complexity of a classification task is the combined result of class imbalance and other factors, such as class overlap. Thus, in order to accurately assess the complexity of a problem, the data complexity measures ought to account for more than the simple IR. In this paper, we demonstrate that IR has a weak correlation with classifier performance in terms of macro averaged recall, gmean score, and precision. Other more complete measures such as the adapted N1 and N3 measures use neighborhood information to assess overlap. These measures show a strong negative correlation with classifier performance, but their reported values were hard to interpret. This motivates a new measure that estimates overlap complexity and returns a value with a clear interpretation. Here we propose such a measure based on the number of minority instances entangled in a Tomek Link. The proposed measure is evaluated on a large selection of synthetic and real datasets and is found to be as good as or better than the best competitors in terms of its negative correlation with respect to mean classifier performance.

Artificial Intelligence and Machine Learning have witnessed rapid, significant improvements in Natural Language Processing (NLP) tasks. Utilizing Deep Learning, researchers have taken advantage of repository comments in Software Engineering to produce accurate methods for detecting Self-Admitted Technical Debt (SATD) from 20 open-source Java projects’ code. In this work, we improve SATD detection with a novel approach that leverages the Bidirectional Encoder Representations from Transformers (BERT) architecture. For comparison, we re-evaluated previous deep learning methods and applied stratified 10-fold cross-validation to report reliable F1-scores. We examine our model in both cross-project and intra-project contexts. For each context, we use re-sampling and duplication as augmentation strategies to account for data imbalance. We find that our trained BERT model improves over the best performance of all previous methods in 19 of the 20 projects in cross-project scenarios. However, the data augmentation techniques were not sufficient to overcome the lack of data present in the intra-project scenarios, and existing methods still perform better. Future research will look into ways to diversify SATD datasets in order to maximize the latent power in large BERT models.

The detection of computer-generated text is an area of rapidly increasing significance as nascent generative models allow for efficient creation of compelling human-like text, which may be abused for the purposes of spam, disinformation, phishing, or online influence campaigns. Past work has studied detection of current state-of-the-art models, but despite a developing threat landscape, there has been minimal analysis of the robustness of detection methods to adversarial attacks. To this end, we evaluate neural and non-neural approaches on their ability to detect computer-generated text, their robustness against text adversarial attacks, and the impact that successful adversarial attacks have on human judgement of text quality. We find that while statistical features underperform neural features, statistical features provide additional adversarial robustness that can be leveraged in ensemble detection models. In the process, we find that previously effective complex phrasal features for detection of computer-generated text hold little predictive power against contemporary generative models, and identify promising statistical features to use instead. Finally, we pioneer the usage of $\Delta$MAUVE as a proxy measure for human judgement of adversarial text quality.

Recently, we have witnessed that deep learning-based approaches have been widely applied. Particularly, some applications involve data that are high dimensional, sparse or imbalanced, which are different from those applications with dense data processing, such as image classification and speech recognition, where deep learning-based approaches have been extensively studied. One of the main applications is the user-centric platform that consists of great deal of users, items and user generated tabular data which are quite high-dimensional. The characteristics of such data pose unique challenges to the adoption of deep learning in these applications, including modeling, training, and online serving, etc. More and more communities from both academia and industry have initiated the endeavors to solve these challenges. This workshop will provide a venue for both the research and engineering communities to discuss and formulate the challenges, utilize opportunities, and propose new ideas in the practice and theory of deep learning on high-dimensional, sparse and imbalanced data.

Due to the rapid technological advances that have been made over the years, more people are changing their way of living from traditional ways of doing business to those featuring greater use of electronic resources. This transition has attracted (and continues to attract) the attention of cybercriminals, referred to in this article as "attackers", who make use of the structure of the Internet to commit cybercrimes, such as phishing, in order to trick users into revealing sensitive data, including personal information, banking and credit card details, IDs, passwords, and more important information via replicas of legitimate websites of trusted organizations. In our digital society, the COVID-19 pandemic represents an unprecedented situation. As a result, many individuals were left vulnerable to cyberattacks while attempting to gather credible information about this alarming situation. Unfortunately, by taking advantage of this situation, specific attacks associated with the pandemic dramatically increased. Regrettably, cyberattacks do not appear to be abating. For this reason, cyber-security corporations and researchers must constantly develop effective and innovative solutions to tackle this growing issue. Although several anti-phishing approaches are already in use, such as the use of blacklists, visuals, heuristics, and other protective solutions, they cannot efficiently prevent imminent phishing attacks. In this paper, we propose machine learning models that use a limited number of features to classify COVID-19-related domain names as either malicious or legitimate. Our primary results show that a small set of carefully extracted lexical features, from domain names, can allow models to yield high scores; additionally, the number of subdomain levels as a feature can have a large influence on the predictions.

Active learning strategies are used to acquire an enlarged labelled training set that allows the learner to achieve a better performance. To this end, unlabelled instances are carefully selected for labelling by an human expert in order to achieve the best performance with the smallest number of questions to this oracle. Several techniques exist to select the most informative samples within active learning strategies. However, the effectiveness of these methods when applied to problems with imbalanced classes was not studied before. In this paper, we focus on the improvement of instance selection strategies for active learning techniques in the presence of class imbalance. In an imbalanced setting, learning algorithms have difficulties to focus on the minority class due to its under-representation. Still, this is typically the class of interest for the end-user. This mismatch between the classes distribution and the goals of the end user is known as the class imbalance problem. In an active learning setting for class imbalance problems, this becomes an even more challenging issue. We propose two novel active learning algorithms, ALOD and ALOD-RE centered around selecting the most informative samples to be labelled, while considering the selection of possible minority class cases and the generation of synthetic minority class examples to improve the learner performance. To this end, our two proposed solutions combine: active learning procedures, resampling strategies, and anomaly detection methods. Through an extensive set of experiments we show that the incorporation of outlier detection and resampling techniques in the active learning procedure benefits the learners performance on imbalanced domains. The performance advantages of our proposed ALOD and ALOD-RE algorithms are clearly supported by our experimental results.

Predictive modelling in cybersecurity domains usually involves dealing with complex settings. The class imbalance problem is a well-know challenge typically present in the cybersecurity domain. For instance, in a real-world intrusion detection scenario, the number of attacks is expected to be a a very small percentage of the normal cases. Moreover, in these applications, the number of available examples labelled is also small due to the complexity and cost of the labelling process: teams of domain experts need to be involved in the process which becomes expensive, time consuming and prone to errors. To address these problems is critical to the success of predictive modelling in cybersecurity applications. In this paper we tackle the class imbalance and small sample size through the use of a CGAN-based up-sampling procedure. We carry out an extensive set of experiments that show the positive impact of applying this solution to address the class imbalance and small sample size problems. A large data repository is built and freely provided to the research community containing 114 binary datasets based on real-world cybersecurity problems that are generated with diversified levels of imbalance and sample size. Our experiments show a clear advantage of using the CGAN-based up-sampling method specially for situations where the sample size is small and there is a large imbalance between the problem classes. In the most critical scenarios associated with extreme rarity and very small sample size, an impressive performance boost is achieved. We also explore the behaviour of this approach when the presence of these problems is less marked and we found that, while CGAN-based up-sampling is not able to further improve the minority class performance, it also has no negative impact. Thus, it is a safe to use solution, also in these scenarios.

Vital signs, such as blood pressure and heart rate, are continuously and closely monitored during surgery and in the intensive care unit to ensure patient health. There has been increasing interest in studying the large data sets of electronic vital sign records to improve patient outcomes, predict issues, and detect complications early. However, the records of vital signs, particularly one called invasive arterial blood pressure, may be populated with artifacts (noise) due to various situations. In order to use this large volume of data in research, it is essential to accurately remove the artifacts to ensure data quality and avoid drawing conclusions from non-physiologic data. Manual labelling of artifacts is not a viable solution because of the significant time required to go through large volumes of data. We studied several solutions for artifact removal, including forecasting methods and machine learning strategies such as standard and anomaly detection algorithms. We also performed experiments using the information of one or multiple feature variables. We observed that XGBoost algorithm achieved the best performance amongst all algorithms tested. Forecasting methods exhibited a poor performance when compared to other machine learning algorithms. Anomaly detection methods showed a good overall performance. However, these special-purpose methods were not able to achieve a performance comparable to the XGBoost learner.

Malware detection is a well-known problem with severe consequences in terms of damages and financial losses. The typical imbalance of this domain causes serious problems to the learning algorithms that are not able to focus correctly in the scarce malware cases. Although resampling techniques have shown to be effective, their impact has not yet been studied for the particular domain of malware detection. This paper focus on the evaluation of resampling methods to tackle the malware detection problem under a realistic assumption of malware rarity. Several machine learning-based solutions proposed for this domain use non-public data sets and consider different malware prevalence in the data. This makes difficult the comparison of solutions. We use the freely available NSL-KDD data set which has a rarity associated with the malware cases that resembles a more realistic scenario. Our main goal is to assess the potential advantages of applying resampling techniques in this setting as well as their impact when using different standard learning algorithms. We also carry out an extensive analysis of resampling techniques using different parameters which allows to test the best configuration of these techniques. We explore not only a balanced class scenario but also other class configurations that may be more beneficial to improve the predictive performance of the algorithms. We present a systematic study of the effectiveness of existing resampling approaches for tackling the imbalance problem in malware domains. We show that resampling techniques present an advantage and explore their impact in the predictive accuracy of each class. Finally, we also show that different resampling strategies have a different impact in the importance of the features used by the learning algorithms.

The class imbalance problem has been thoroughly studied over the past two decades. More recently, the research community realized that the problem of imbalanced distributions also occurred in other tasks beyond classification. Regression problems are among these newly studied tasks where the problem of imbalanced domains also poses important challenges. Imbalanced regression problems occur in a diversity of real world domains such as meteorological (predicting weather extreme values), financial (extreme stock returns forecasting) or medical (anticipate rare values). In imbalanced regression the end-user preferences are biased towards values of the target variable that are under-represented on the available data. Several pre-processing methods were proposed to address this problem. These methods change the training set to force the learner to focus on the rare cases. However, as far as we know, the relationship between the data intrinsic characteristics and the performance achieved by these methods has not yet been studied for imbalanced regression tasks. In this paper we describe a study of the impact certain data characteristics may have in the results of applying pre-processing methods to imbalanced regression problems. To achieve this goal, we define potentially interesting data characteristics of regression problems. We then conduct our study using a synthetic data repository build for this purpose. We show that all the different characteristics studied have a different behaviour that is related with the level at which the data characteristic is present and the learning algorithm used. The main contributions of our work are: i) to define interesting data characteristics for regression tasks; ii) to create the first repository of imbalanced regression tasks containing 6000 data sets with controlled data characteristics; and iii) to provide insights on the impact of intrinsic data characteristics in the results of pre-processing methods for handling imbalanced regression tasks.

Abstract Imbalanced domains are an important problem that arises in predictive tasks causing a loss in the performance on the most relevant cases for the user. This problem has been extensively studied for classification problems, where the target variable is nominal. Recently, it was recognized that imbalanced domains occur in several other contexts and for multiple tasks, such as regression tasks, where the target variable is continuous. This paper focuses on imbalanced domains in both classification and regression tasks. Resampling strategies are among the most successful approaches to address imbalanced domains. In this work, we propose variants of existing resampling strategies that are able to take into account the information regarding the neighbourhood of the examples. Instead of performing sampling uniformly, our proposals bias the strategies to reinforce some regions of the data sets. With an extensive set of experiments, we provide evidence of the advantage of introducing a neighbourhood bias in the resampling strategies for both classification and regression tasks with imbalanced data sets.

Accounting for misclassification costs is important in many practical applications of machine learning, and cost-sensitive techniques for classification have been studied extensively. Utility-based learning provides a generalization of purely cost-based approaches that considers both costs and benefits, enabling application to domains with complex cost-benefit settings. However, there is little work on utility- or cost-based learning for regression. In this paper, we formally define the problem of utility-based regression and propose a strategy for maximizing the utility of regression models. We verify our findings in a large set of experiments that show the advantage of our proposal in a diverse set of domains, learning algorithms and cost/benefit settings.

Time series forecasting is a challenging task, where the non-stationary characteristics of the data portrays a hard setting for predictive tasks. A common issue is the imbalanced distribution of the target variable, where some intervals are very important to the user but severely underrepresented. Standard regression tools focus on the average behaviour of the data. However, the objective is the opposite in many forecasting tasks involving time series: predicting rare values. A common solution to forecasting tasks with imbalanced data is the use of resampling strategies, which operate on the learning data by changing its distribution in favor of a given bias. The objective of this paper is to provide solutions capable of significantly improving the predictive accuracy of rare cases in forecasting tasks using imbalanced time series data. We extend the application of resampling strategies to the time series context and introduce the concept of temporal and relevance bias in the case selection process of such strategies, presenting new proposals. We evaluate the results of standard regression tools and the use of resampling strategies, with and without bias over 24 time series data sets from 6 different sources. Results show a significant increase in predictive accuracy of rare cases associated with the use of resampling strategies, and the use of biased strategies further increases accuracy over the non-biased strategies.

Abstract Several real world prediction problems involve forecasting rare values of a target variable. When this variable is nominal, we have a problem of class imbalance that was thoroughly studied within machine learning. For regression tasks, where the target variable is continuous, few works exist addressing this type of problem. Still, important applications involve forecasting rare extreme values of a continuous target variable. This paper describes a contribution to this type of tasks. Namely, we propose to address such tasks by resampling approaches that change the distribution of the given data set to decrease the problem of imbalance between the rare target cases and the most frequent ones. We present two modifications of well‐known resampling strategies for classification tasks: the under‐sampling and the synthetic minority over‐sampling technique (SMOTE) methods. These modifications allow the use of these strategies on regression tasks where the goal is to forecast rare extreme values of the target variable. In an extensive set of experiments, we provide empirical evidence for the superiority of our proposals for these particular regression tasks. The proposed resampling methods can be used with any existing regression algorithm, which means that they are general tools for addressing problems of forecasting rare extreme values of a continuous target variable.