International audience

Self-supervision is often used for pre-training to foster performance on a downstream task by constructing meaningful representations of samples. Self-supervised learning (SSL) generally involves generating different views of the same sample and thus requires data augmentations that are challenging to construct for tabular data. This constitutes one of the main challenges of self-supervision for structured data. In the present work, we propose a novel augmentation-free SSL method for tabular data. Our approach, T-JEPA, relies on a Joint Embedding Predictive Architecture (JEPA) and is akin to mask reconstruction in the latent space. It involves predicting the latent representation of one subset of features from the latent representation of a different subset within the same sample, thereby learning rich representations without augmentations. We use our method as a pre-training technique and train several deep classifiers on the obtained representation. Our experimental results demonstrate a substantial improvement in both classification and regression tasks, outperforming models trained directly on samples in their original data space. Moreover, T-JEPA enables some methods to consistently outperform or match the performance of traditional methods likes Gradient Boosted Decision Trees. To understand why, we extensively characterize the obtained representations and show that T-JEPA effectively identifies relevant features for downstream tasks without access to the labels. Additionally, we introduce regularization tokens, a novel regularization method critical for training of JEPA-based models on structured data.

Deep learning for tabular data has garnered increasing attention in recent years, yet employing deep models for structured data remains challenging. While these models excel with unstructured data, their efficacy with structured data has been limited. Recent research has introduced retrieval-augmented models to address this gap, demonstrating promising results in supervised tasks such as classification and regression. In this work, we investigate using retrieval-augmented models for anomaly detection on tabular data. We propose a reconstruction-based approach in which a transformer model learns to reconstruct masked features ofnormal samples. We test the effectiveness of KNN-based and attention-based modules to select relevant samples to help in the reconstruction process of the target sample. Our experiments on a benchmark of 31 tabular datasets reveal that augmenting this reconstruction-based anomaly detection (AD) method with sample-sample dependencies via retrieval modules significantly boosts performance. The present work supports the idea that retrieval module are useful to augment any deep AD method to enhance anomaly detection on tabular data. Our code to reproduce the experiments is made available on GitHub.

Anomaly detection is vital in many domains, such as finance, healthcare, and cybersecurity. In this paper, we propose a novel deep anomaly detection method for tabular data that leverages Non-Parametric Transformers (NPTs), a model initially proposed for supervised tasks, to capture both feature-feature and sample-sample dependencies. In a reconstruction-based framework, we train an NPT to reconstruct masked features of normal samples. In a non-parametric fashion, we leverage the whole training set during inference and use the model’s ability to reconstruct the masked features to generate an anomaly score. To the best of our knowledge, this is the first work to successfully combine feature-feature and sample-sample dependencies for anomaly detection on tabular datasets. Through extensive experiments on 31 benchmark tabular datasets, we demonstrate that our method achieves state-of-the-art performance, outperforming existing methods by 2.4% and 1.2% in terms of F1-score and AUROC, respectively. Our ablation study further proves that modeling both types of dependencies is crucial for anomaly detection on tabular data.

Deep Reinforcement Learning (DRL) approaches to Online Portfolio Selection (OLPS) have grown in popularity in recent years. The sensitive nature of training Reinforcement Learning agents implies a need for extensive efforts in market representation, behavior objectives, and training processes, which have often been lacking in previous works. We propose a training and evaluation process to assess the performance of classical DRL algorithms for portfolio management. We compare combinations of RL algorithms (DDPG, PPO, A2C, SAC), market representations (Prices, Windows, Indicators) and rewards (Returns and Risk). We found that most DRL algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.

As with many other tasks, neural networks prove very effective for anomaly detection purposes. However, very few deep-learning models are suited for detecting anomalies on tabular datasets. This paper proposes a novel methodology to flag anomalies based on TracIn, an influence measure initially introduced for explicability purposes. The proposed methods can serve to augment any unsupervised deep anomaly detection method. We test our approach using Variational Autoencoders and show that the average influence of a subsample of training points on a test point can serve as a proxy for abnormality. Our model proves to be competitive in comparison with state-of-the-art approaches: it achieves comparable or better performance in terms of detection accuracy on medical and cyber-security tabular benchmark data.

International audience

Personalization in the field of Technology Enhanced Learning (TEL) is a topic that received a lot of concern by researchers. At the same time, there is a growing amount of Open Educational Resources (OER) indexed according to the W3C standards. Relevant OERs can usefully complement the contents delivered to a learner during an online course. Computing the best OERs to offer to the learner at each point of his course is an aspect of personalization that we address in this paper. We designed our MORS system to solve this problem in the context of Massive Open Online Courses (MOOC). Our MORS system described in this paper, is based on a learner profile, on metadata describing the course and on a carefully crafted process to query the SparQL endpoints for OERs.

In formative assessments, one wants to provide a useful feedback to the examinee at the end of the test. In order to reduce the number of questions asked in an assessment, adaptive testing models have been developed for cognitive diagnosis, such as the ones encountered in knowledge space theory. However, when the number of skills assessed is very huge, such methods cannot scale. In this paper, we present a new method to provide adaptive tests and useful feedback to the examinee, even with large databases of skills. It will be used in Pix, a platform for certification of digital competencies for every French citizen.

Personalized applications like recommendation systems or open adaptive hypermedia require to facilitate reuse and interoperability of educational resources. Thus we have to solve the problem of discovering educational datasets and the problem of the fragmented landscape of competing metadata schemas. In this paper, we compare a subset of metadata schemas used by educational datasets obeying the Linked Open Data principles and we propose mappings between them.

Personalized systems are a response to the increasing number of resources on the Internet. In order to facilitate their design and creation, we aim at formalizing them. In this paper, we consider the relationship between a personalized application and its non-personalized counterpart. We argue that a personalized application is a formal extension of a non-personalized one. We aim at characterizing the syntactic differences between the expression of the personalized and non-personalized versions of the application. Situation calculus is our framework to formalize applications. We introduce two scenarios of non-personalized application that we personalize to illustrate our approach.

Personalized systems are a response to the increasing number of resources on the Internet, but can be difficult to create. In order to facilitate the design and creation of such personalized systems, we aim at formalizing them. The situation calculus is a logical framework that has often been proposed to model web applications and even personalized ones. However, the details of its use are much more rarely explained. In this paper we will show that it is needed to carefully consider which variant of the situation calculus to choose. We will precisely show why we want to use the so-called guarded action theories. We explain why and how it fits into an architecture. We introduce two scenarios of personalized applications to illustrate this choice.

We present an adaptive hypermedia architecture based on Golog and conceptual graphs. We propose a conceptual graph model for the description of the hypermedia structure and the use of Golog, a logic programming language, for the generation of the personalized hypermedia. This approach has been validated by a prototype and two examples based on different domains.