International audience

Self-supervision is often used for pre-training to foster performance on a downstream task by constructing meaningful representations of samples. Self-supervised learning (SSL) generally involves generating different views of the same sample and thus requires data augmentations that are challenging to construct for tabular data. This constitutes one of the main challenges of self-supervision for structured data. In the present work, we propose a novel augmentation-free SSL method for tabular data. Our approach, T-JEPA, relies on a Joint Embedding Predictive Architecture (JEPA) and is akin to mask reconstruction in the latent space. It involves predicting the latent representation of one subset of features from the latent representation of a different subset within the same sample, thereby learning rich representations without augmentations. We use our method as a pre-training technique and train several deep classifiers on the obtained representation. Our experimental results demonstrate a substantial improvement in both classification and regression tasks, outperforming models trained directly on samples in their original data space. Moreover, T-JEPA enables some methods to consistently outperform or match the performance of traditional methods likes Gradient Boosted Decision Trees. To understand why, we extensively characterize the obtained representations and show that T-JEPA effectively identifies relevant features for downstream tasks without access to the labels. Additionally, we introduce regularization tokens, a novel regularization method critical for training of JEPA-based models on structured data.

Deep learning for tabular data has garnered increasing attention in recent years, yet employing deep models for structured data remains challenging. While these models excel with unstructured data, their efficacy with structured data has been limited. Recent research has introduced retrieval-augmented models to address this gap, demonstrating promising results in supervised tasks such as classification and regression. In this work, we investigate using retrieval-augmented models for anomaly detection on tabular data. We propose a reconstruction-based approach in which a transformer model learns to reconstruct masked features ofnormal samples. We test the effectiveness of KNN-based and attention-based modules to select relevant samples to help in the reconstruction process of the target sample. Our experiments on a benchmark of 31 tabular datasets reveal that augmenting this reconstruction-based anomaly detection (AD) method with sample-sample dependencies via retrieval modules significantly boosts performance. The present work supports the idea that retrieval module are useful to augment any deep AD method to enhance anomaly detection on tabular data. Our code to reproduce the experiments is made available on GitHub.

Anomaly detection is vital in many domains, such as finance, healthcare, and cybersecurity. In this paper, we propose a novel deep anomaly detection method for tabular data that leverages Non-Parametric Transformers (NPTs), a model initially proposed for supervised tasks, to capture both feature-feature and sample-sample dependencies. In a reconstruction-based framework, we train an NPT to reconstruct masked features of normal samples. In a non-parametric fashion, we leverage the whole training set during inference and use the model’s ability to reconstruct the masked features to generate an anomaly score. To the best of our knowledge, this is the first work to successfully combine feature-feature and sample-sample dependencies for anomaly detection on tabular datasets. Through extensive experiments on 31 benchmark tabular datasets, we demonstrate that our method achieves state-of-the-art performance, outperforming existing methods by 2.4% and 1.2% in terms of F1-score and AUROC, respectively. Our ablation study further proves that modeling both types of dependencies is crucial for anomaly detection on tabular data.

Deep Reinforcement Learning (DRL) approaches to Online Portfolio Selection (OLPS) have grown in popularity in recent years. The sensitive nature of training Reinforcement Learning agents implies a need for extensive efforts in market representation, behavior objectives, and training processes, which have often been lacking in previous works. We propose a training and evaluation process to assess the performance of classical DRL algorithms for portfolio management. We compare combinations of RL algorithms (DDPG, PPO, A2C, SAC), market representations (Prices, Windows, Indicators) and rewards (Returns and Risk). We found that most DRL algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.

As with many other tasks, neural networks prove very effective for anomaly detection purposes. However, very few deep-learning models are suited for detecting anomalies on tabular datasets. This paper proposes a novel methodology to flag anomalies based on TracIn, an influence measure initially introduced for explicability purposes. The proposed methods can serve to augment any unsupervised deep anomaly detection method. We test our approach using Variational Autoencoders and show that the average influence of a subsample of training points on a test point can serve as a proxy for abnormality. Our model proves to be competitive in comparison with state-of-the-art approaches: it achieves comparable or better performance in terms of detection accuracy on medical and cyber-security tabular benchmark data.

This paper presents a flexible and semi-automatic extractor to build personalized learning objects. The extraction is made using an ontology and a matching grammar defined by the user. The ontology describes the concepts/classes and the relations between these concepts/classes. The learning objects we want to obtain from the web pages are instances of the ontology. The grammar describes rules for matching the ontology concepts/classes to the HTML content. Its rules apply to both HTML format and text. The result is a graph of objects matched in the HTML files, objects that are instances of the concepts/classes in ontology.

This paper presents experiments using an algorithm of web page topic segmentation that show significant precision improvement in the retrieval of documents issued from the Web track corpus of TREC 2001. Instead of processing the whole document, a web page is segmented into different semantic blocks according to visual criteria (such as horizontal lines, colors) and structural tags (such as headings ~ , paragraph ). We conclude that combining visual and content layout criteria gives the best results for increasing the precision: the ranking of the page is calculated for relevant segments of pages resulting from the segmentation algorithm.

Metadata are required for finding, exchanging, and managing learning objects. But, their semantics is not clear. Furthermore, the IEEE LOM (Learning Object Metadata) is an IEEE standard since July 2002 but, even if it is widely used, it is not the only one standard used to describe learning objects. The use of ontologies is a key step forward for describing the semantics of information on the Web. We propose to specify the semantics of metadata by using a formal language of ontology, OWL. Furthermore, assuming that we are in an open world and in order to facilitate the exchange of learning resources with other institutions, we propose a solution based on several ontologies.

The approach presented in this paper is intended for the semi-automatic construction of a learning object repository from HTML pages. An extraction method consists of applying the extraction patterns defined in a configuration file, and using a pre-defined ontology, tries also to discover partially matched objects and to assign them proper tags conformed to an XML schema.

The World Wide Web offers an increasing amount of complex and rich educational Web resources that are available for free in various domains. Unfortunately, it is difficult today to have a Web agent that answers precisely a simple query. Semantic Web aims to make Web resources meaningful to automated agents. Ontologies are proposed to provide a formal representation of a shared and common conceptualization of a specific domain. For the description of educational resources several communities are working on the definition of metadata elements. The Learning Technology Standards Committee (LTSC) specifies the Learning Object Metadata (LOM), a set of elements describing the relevant characteristics for learning resources. The goal of this paper is to give a formal and more comprehensive content description of learning resources in order to allow better reusability and retrievals. This description is particularly focused on the semantic relationships between learning resources which constitute an important aspect to access information. It uses OWL, an ontology language for the semantic Web, recently developed by the W3C. OWL provides powerful expressiveness combined with desirable computational properties for reasoning systems due to its correspondence with description logics. The query of the corresponding knowledge base is illustrated using OWL Query Language OWL-QL.

In this paper we present an example of a part of a pedagogical ontology for a grande ecole. OWL is intended to help users to formalize ontologies and to be a support for the semantic Web, for example to enable the interchange of resources and the inference of knowledge while querying these resources.

We discuss a vision of the semantic Web for learning resources and some problems found. We consider RDF as the first step to the semantic Web. But using all the possibilities of RDF is not that simple. Constructing an RDF schema for a given metadata schema, translating RDF descriptions and querying a set of descriptions are the main problems we discuss here. The example of the LOM (learning object metadata), the IEEE standard for education, is used. But even if some problems are not solved, it is important to use RDF now in a simple way to find more easily resources.

We present an adaptive hypermedia architecture based on Golog and conceptual graphs. We propose a conceptual graph model for the description of the hypermedia structure and the use of Golog, a logic programming language, for the generation of the personalized hypermedia. This approach has been validated by a prototype and two examples based on different domains.

This paper describes a new information retrieval system for pedagogical content. Each pedagogical resource is explicit thanks to metadata and structure. We explain how to manipulate the structure of these multimedia resources at an appropriate level of granularity (resources, set of resources). Finally we propose combining the structural and the textual analysis of the resources to improve the performance of the system.