This study explores dataset complexity estimation and its relationship with classification performance. Widely-used measures like the k-Disagreeing Neighbors (kDN) are limited by their reliance on a fixed number of neighbors ( k ) and their discrete output, which can lead to coarse estimations. To address these issues, we introduce Dynamic Disagreeing Neighbors (DDN), a measure that incorporates dynamic, density-aware neighborhoods and distance-based weighting to provide a smoother, more flexible complexity estimation. We conduct a large-scale empirical study on 65 binary datasets, comparing DDN against kDN and other state-of-the-art measures, analyzing the impact of the k parameter and class imbalance on the alignment with classifier performance. Our results show that DDN provides a more robust and stable correspondence with performance. Furthermore, we demonstrate that in a predictive modeling task, a regression model using DDN as a feature is significantly more accurate at estimating dataset performance than models using other complexity measures. These findings not only deepen our understanding of complexity estimation but also pave the way for more informed classifier selection and data preprocessing strategies.

Abstract Data augmentation is a widely used strategy to enhance the predictive power of machine learning (ML) models. This is the case of intent classification problems, where the end goal of a utterance needs to be categorized using text mining techniques. Nevertheless, recent augmentation methods based on general off-the-shelf Large Language Models (LLMs) have room for improvement. They can struggle to effectively capture the nuances associated with domain-specific scenarios. This paper presents a novel utterance augmentation method that uses LLMs and word embedding models to address the issue, particularly in domain-specific problems. The proposed method starts from a given reference set. Then, paraphrases are generated using LLMs to obtain new utterances that are semantically similar to the original ones, but with different word choices and syntax. Next, synonym replacement is accomplished using a previously trained domain-specific word embedding model. This entails the incorporation of relevant vocabulary to a particular topic into the final augmented dataset, effectively capturing the nuances of domain-specific problems. Experiments were conducted to assess the quality of the proposal in two intent classification problems related to financial trading compliance. The proposal has an advantage over many well-known approaches in the first problem, comprising eight reference utterances and 375 challenging examples in general language. In the second problem, with 13 reference utterances and 525 challenging examples in general and financial trading vocabulary, the proposal outperforms the best-analyzed state-of-the-art methods by up to 15%.

Abstract Complexity measures aim to characterize the underlying complexity of supervised data. These measures tackle factors hindering the performance of Machine Learning (ML) classifiers like overlap, density, linearity, etc. The state-of-the-art has mainly focused on the dataset perspective of complexity, i.e., offering an estimation of the complexity of the whole dataset. Recently, the instance perspective has also been addressed. In this paper, the hostility measure, a complexity measure offering a multi-level (instance, class, and dataset) perspective of data complexity is proposed. The proposal is built by estimating the novel notion of hostility: the difficulty of correctly classifying a point, a class, or a whole dataset given their corresponding neighborhoods. The proposed measure is estimated at the instance level by applying the k-means algorithm in a recursive and hierarchical way, which allows to analyze how points from different classes are naturally grouped together across partitions. The instance information is aggregated to provide complexity knowledge at the class and the dataset levels. The validity of the proposal is evaluated through a variety of experiments dealing with the three perspectives and the corresponding comparative with the state-of-the-art measures. Throughout the experiments, the hostility measure has shown promising results and to be competitive, stable, and robust.

Customer acquisition is an issue that continues to receive attention from companies worldwide.Various marketing campaigns using psychological methodologies have been designed to address this issue.However, once a campaign is launched, it is highly complicated to detect which sets of customers are most likely to purchase an offered product.This fact is key since it allows companies to focus their efforts on specific clients and discard others.Several selection techniques have been implemented, but most of them are usually very demanding in terms of time and human resources for the companies.Artificial Intelligence techniques appear to help to simplify the process.Thus, companies have started to use Machine Learning (ML) models trained to efficiently detect those clients with certain proneness to purchase.Toward this goal, this paper presents a novel purchase propensity detection ML system based on Sentiment Analysis techniques able to consider customer comments regarding the offered products.The tourist domain was selected for the case study, where the obtained product was successfully embedded in an initial prototype.