Monitoring traffic streams through vehicle trajectories offers valuable insights into traffic flow characteristics. In recent years, there has been a surge in the availability of vehicle trajectory datasets. At the same time, the number of autonomously-driven vehicles on the road is increasing, largely due to the adoption of systems like adaptive cruise control. However, distinguishing system-controlled vehicles from human-driven vehicles remains challenging, despite its potential to enable valuable applications and informed policy-making. The differences between the driving behavior of human-driven (HDs) and automated (ADs) vehicles in the longitudinal direction are highlighted in the literature and hold promise for novel methodologies that exploit them to identify the type of driver. Here, we propose a novel online-offline framework with three key contributions. First, a feature design component performs feature disentanglement to increase the performance of downstream deep learning models. Second, a bidirectional LSTM (bLSTM) architecture demonstrates excellent accuracy in differentiating between HD and AD vehicles. Third, a data drift detection component identifies changes in data distributions, enabling the framework to generalize effectively to unseen datasets with minimal new labeled observations.

Abstract The automation of data science and other data manipulation processes depend on the integration and formatting of ‘messy’ data. Data wrangling is an umbrella term for these tedious and time-consuming tasks. Tasks such as transforming dates, units or names expressed in different formats have been challenging for machine learning because (1) users expect to solve them with short cues or few examples, and (2) the problems depend heavily on domain knowledge. Interestingly, large language models today (1) can infer from very few examples or even a short clue in natural language, and (2) can integrate vast amounts of domain knowledge. It is then an important research question to analyse whether language models are a promising approach for data wrangling, especially as their capabilities continue growing. In this paper we apply different variants of the language model Generative Pre-trained Transformer (GPT) to five batteries covering a wide range of data wrangling problems. We compare the effect of prompts and few-shot regimes on their results and how they compare with specialised data wrangling systems and other tools. Our major finding is that they appear as a powerful tool for a wide range of data wrangling tasks. We provide some guidelines about how they can be integrated into data processing pipelines, provided the users can take advantage of their flexibility and the diversity of tasks to be addressed. However, reliability is still an important issue to overcome.

CRISP-DM(CRoss-Industry Standard Process for Data Mining) has its origins in the second half of the nineties and is thus about two decades old. According to many surveys and user polls it is still the de facto standard for developing data mining and knowledge discovery projects. However, undoubtedly the field has moved on considerably in twenty years, with data science now the leading term being favoured over data mining. In this paper we investigate whether, and in what contexts, CRISP-DM is still fit for purpose for data science projects. We argue that if the project is goal-directed and process-driven the process model view still largely holds. On the other hand, when data science projects become more exploratory the paths that the project can take become more varied, and a more flexible model is called for. We suggest what the outlines of such a trajectory-based model might look like and how it can be used to categorise data science projects (goal-directed, exploratory or data management). We examine seven real-life exemplars where exploratory activities play an important role and compare them against 51 use cases extracted from the NIST Big Data Public Working Group. We anticipate this categorisation can help project planning in terms of time and cost characteristics.

One classical way of characterising the rich range of machine learning techniques is by defining 'families', according to their formulation and learning strategy (e.g., neural networks, Bayesian methods, etc.).However, this taxonomy of learning techniques does not consider the extent to which models built with techniques from the same or different family agree on their outputs, especially when their predictions have to extrapolate in sparse zones where insufficient training data was available.In this paper we present a new taxonomy of machine learning techniques for classification, where families are clustered according to their degree of (dis)agreement in behaviour considering both dense and sparse zones, using Cohen's kappa statistic.To this end, we use a representative collection of datasets and learning techniques.We finally validate the taxonomy by performing a number of experiments for technique selection.We show that ranking techniques by only following prejudice -the reputation they have for other problems-is worse than selecting techniques based on family diversity.

A more effective vision of machine learning systems entails tools that are able to improve task after task and to reuse the patterns and knowledge that are acquired previously for future tasks. This incremental, long-life view of machine learning goes beyond most of state-of-the-art machine learning techniques that learn throw-away models. In this paper we present a long-life knowledge acquisition, evaluation and consolidation framework that is designed to work with any rule-based machine learning or inductive inference engine and integrate it into a long-life learner. In order to do that we work over the graph of working memory rules and introduce several topological metrics over it from which we derive an oblivion criterion to drop useless rules from working memory and a consolidation process to promote the rules to the knowledge base. We evaluate the framework on a series of tasks in a chess rule learning domain.

Distance‐based and generalization‐based methods are two families of artificial intelligence techniques that have been successfully used over a wide range of real‐world problems. In the first case, general algorithms can be applied to any data representation by just changing the distance. The metric space sets the search and learning space, which is generally instance‐oriented. In the second case, models can be obtained for a given pattern language, which can be comprehensible. The generality‐ordered space sets the search and learning space, which is generally model‐oriented. However, the concepts of distance and generalization clash in many different ways, especially when knowledge representation is complex (e.g., structured data). This work establishes a framework where these two fields can be integrated in a consistent way. We introduce the concept of distance‐based generalization, which connects all the generalized examples in such a way that all of them are reachable inside the generalization by using straight paths in the metric space. This makes the metric space and the generality‐ordered space coherent (or even dual). Additionally, we also introduce a definition of minimal distance‐based generalization that can be seen as the first formulation of the Minimum Description Length (MDL)/Minimum Message Length (MML) principle in terms of a distance function. We instantiate and develop the framework for the most common data representations and distances, where we show that consistent instances can be found for numerical data, nominal data, sets, lists, tuples, graphs, first‐order atoms, and clauses. As a result, general learning methods that integrate the best from distance‐based and generalization‐based methods can be defined and adapted to any specific problem by appropriately choosing the distance, the pattern language and the generalization operator.

Quantification is the name given to a novel machine learning task which deals with correctly estimating the number of elements of one class in a set of examples. The output of a quantifier is a real value, since training instances are the same as a classification problem, a natural approach is to train a classifier and to derive a quantifier from it. Some previous works have shown that just classifying the instances and counting the examples belonging to the class of interest classify count typically yields bad quantifiers, especially when the class distribution may vary between training and test. Hence, adjusted versions of classify count have been developed by using modified thresholds. However, previous works have explicitly discarded (without a deep analysis) any possible approach based on the probability estimations of the classifier. In this paper, we present a method based on averaging the probability estimations of a classifier with a very simple scaling that does perform reasonably well, showing that probability estimators for quantification capture a richer view of the problem than methods based on a threshold.

Machine learning can be applied to solve the knowledge acquisition bottleneck in many areas where an expert makes predictions to single cases, such as diagnosis, estimation, etc. The idea is to query the expert with as many cases as possible and get their answers. With this data we train a machine learning model which mimics the expert's behaviour. This is just a simple application of a modelling technique known as "mimetism", which has many other applications. This "soft" approach to knowledge acquisition has many advantages: any machine learning technique can be used, the expert must only answer simple questions (cases) and we can combine the decisions of several experts easily. However, one problem of this approach is that we do not know in advance how many cases we will need to ask in order to get a good model which is accurate wrt. the expert's knowledge. Obviously, as more data is labelled by the expert better results are obtained. However, asking thousands of cases to the expert is usually impractical. In this paper, we analyse the behaviour of knowledge acquisition through mimetic learning according to two factors: accuracy and comprehensibility of the resulting model and we devise a method to compute the minimum number of cases that we need to ask the expert to attain a certain quality level.