International audience

Self-supervision is often used for pre-training to foster performance on a downstream task by constructing meaningful representations of samples. Self-supervised learning (SSL) generally involves generating different views of the same sample and thus requires data augmentations that are challenging to construct for tabular data. This constitutes one of the main challenges of self-supervision for structured data. In the present work, we propose a novel augmentation-free SSL method for tabular data. Our approach, T-JEPA, relies on a Joint Embedding Predictive Architecture (JEPA) and is akin to mask reconstruction in the latent space. It involves predicting the latent representation of one subset of features from the latent representation of a different subset within the same sample, thereby learning rich representations without augmentations. We use our method as a pre-training technique and train several deep classifiers on the obtained representation. Our experimental results demonstrate a substantial improvement in both classification and regression tasks, outperforming models trained directly on samples in their original data space. Moreover, T-JEPA enables some methods to consistently outperform or match the performance of traditional methods likes Gradient Boosted Decision Trees. To understand why, we extensively characterize the obtained representations and show that T-JEPA effectively identifies relevant features for downstream tasks without access to the labels. Additionally, we introduce regularization tokens, a novel regularization method critical for training of JEPA-based models on structured data.

Deep learning for tabular data has garnered increasing attention in recent years, yet employing deep models for structured data remains challenging. While these models excel with unstructured data, their efficacy with structured data has been limited. Recent research has introduced retrieval-augmented models to address this gap, demonstrating promising results in supervised tasks such as classification and regression. In this work, we investigate using retrieval-augmented models for anomaly detection on tabular data. We propose a reconstruction-based approach in which a transformer model learns to reconstruct masked features ofnormal samples. We test the effectiveness of KNN-based and attention-based modules to select relevant samples to help in the reconstruction process of the target sample. Our experiments on a benchmark of 31 tabular datasets reveal that augmenting this reconstruction-based anomaly detection (AD) method with sample-sample dependencies via retrieval modules significantly boosts performance. The present work supports the idea that retrieval module are useful to augment any deep AD method to enhance anomaly detection on tabular data. Our code to reproduce the experiments is made available on GitHub.

Anomaly detection is vital in many domains, such as finance, healthcare, and cybersecurity. In this paper, we propose a novel deep anomaly detection method for tabular data that leverages Non-Parametric Transformers (NPTs), a model initially proposed for supervised tasks, to capture both feature-feature and sample-sample dependencies. In a reconstruction-based framework, we train an NPT to reconstruct masked features of normal samples. In a non-parametric fashion, we leverage the whole training set during inference and use the model’s ability to reconstruct the masked features to generate an anomaly score. To the best of our knowledge, this is the first work to successfully combine feature-feature and sample-sample dependencies for anomaly detection on tabular datasets. Through extensive experiments on 31 benchmark tabular datasets, we demonstrate that our method achieves state-of-the-art performance, outperforming existing methods by 2.4% and 1.2% in terms of F1-score and AUROC, respectively. Our ablation study further proves that modeling both types of dependencies is crucial for anomaly detection on tabular data.

Deep Reinforcement Learning (DRL) approaches to Online Portfolio Selection (OLPS) have grown in popularity in recent years. The sensitive nature of training Reinforcement Learning agents implies a need for extensive efforts in market representation, behavior objectives, and training processes, which have often been lacking in previous works. We propose a training and evaluation process to assess the performance of classical DRL algorithms for portfolio management. We compare combinations of RL algorithms (DDPG, PPO, A2C, SAC), market representations (Prices, Windows, Indicators) and rewards (Returns and Risk). We found that most DRL algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.

As with many other tasks, neural networks prove very effective for anomaly detection purposes. However, very few deep-learning models are suited for detecting anomalies on tabular datasets. This paper proposes a novel methodology to flag anomalies based on TracIn, an influence measure initially introduced for explicability purposes. The proposed methods can serve to augment any unsupervised deep anomaly detection method. We test our approach using Variational Autoencoders and show that the average influence of a subsample of training points on a test point can serve as a proxy for abnormality. Our model proves to be competitive in comparison with state-of-the-art approaches: it achieves comparable or better performance in terms of detection accuracy on medical and cyber-security tabular benchmark data.

We are interested in Quantum Annealing (QA), an algorithm inspired by quantum theory and Simulated Annealing (SA). It is based on quantum replicas, which explore an energy surface, and are less prone to be trapped in local minima. Moreover, kinetic energy helps replicas to find a global minimum. This method has proved its efficiency for several optimization problems. We start this study by presenting the application of QA to a new problem: the Multidimensional Knapsack Problem (MKP). We then present a new idea to speed up the quantum annealing process by detecting the resemblance between replicas. If many of the replicas exhibit the same properties, our assumption is that these properties will also be present with a high probability in a global solution. Consequently, the QA may restrict certain mutations in order to preserve those similarities. We call this algorithm Restrictive Quantum Annealing (RQA). We establish that RQA has better performances than QA and SA by carrying out an adequate analysis of the RQA performance, taking the Traveling Salesman Problem (TSP) and the above-mentioned MKP as references. We also advance guidelines indicating types of NP-hard problems for which our algorithm is particularly well adapted.

This paper presents the recent technical advances in Monte Carlo tree search (MCTS) for the game of Go, shows the many similarities and the rare differences between the current best programs, and reports the results of the Computer Go event organized at the 2009 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE2009), in which four main Go programs played against top level humans. We see that in 9 × 9, computers are very close to the best human level, and can be improved easily for the opening book; whereas in 19 × 19, handicap 7 is not enough for the computers to win against top level professional players, due to some clearly understood (but not solved) weaknesses of the current algorithms. Applications far from the game of Go are also cited. Importantly, the first ever win of a computer against a 9th Dan professional player in 9 × 9 Go occurred in this event.

In order to stimulate the development and research in computer Go, several Taiwanese Go players, including three professional Go players and four amateur Go players, were invited to play against the famous computer Go program, MoGo, in the Taiwan Open 2009. The MoGo program combines the online game values, offline values extracted from databases, and expert rules defined by Go expert that shows an excellent performance in the games. The results reveal that MoGo can reach the level of 3 Dan in Taiwan amateur Go environment. But there are still some drawbacks for MoGo that should be solved, for example, the weaknesses in semeai and how to flexibly practice the human knowledge through the embedded opening books. In this paper, a new game record ontology for computer Go knowledge management is proposed to solve the problems that MoGo is facing. It is hoped that the advances in intelligent agent and ontology model can provide much more knowledge to make a progress in computer Go and achieve as much as computer chess or Chinese chess in the future.

The problem of choosing fast implementations for a class of recursive algorithms such as the fast Fourier transforms can be formulated as an optimization problem over the language generated by a suitably defined grammar. We propose a novel algorithm that solves this problem by reducing it to maximizing an objective function over the sinks of a directed acyclic graph. This algorithm valuates nodes using Monte-Carlo and grows a subgraph in the most promising directions by considering local maximum k-armed bandits. When used inside an adaptive linear transform library, it cuts down the search time by an order of magnitude compared to the existing algorithm. In some cases, the performance of the implementations found is also increased by up to 10% which is of considerable practical importance since it consequently improves the performance of all applications using the library.

In order to promote computer Go and stimulate further development and research in the field, the event activities, Computational Intelligence Forum and World 9$\,\times\,$9 Computer Go Championship, were held in Taiwan. This study focuses on the invited games played in the tournament Taiwanese Go Players Versus the Computer Program MoGo held at the National University of Tainan (NUTN), Tainan, Taiwan. Several Taiwanese Go players, including one 9-Dan (9D) professional Go player and eight amateur Go players, were invited by NUTN to play against MoGo from August 26 to October 4, 2008. The MoGo program combines all-moves-as-first (AMAF)/rapid action value estimation (RAVE) values, online “upper confidence tree (UCT)-like” values, offline values extracted from databases, and expert rules. Additionally, four properties of MoGo are analyzed including: 1) the weakness in corners, 2) the scaling over time, 3) the behavior in handicap games, and 4) the main strength of MoGo in contact fights. The results reveal that MoGo can reach the level of 3 Dan (3D) with: 1) good skills for fights, 2) weaknesses in corners, in particular, for “semeai” situations, and 3) weaknesses in favorable situations such as handicap games. It is hoped that the advances in AI and computational power will enable considerable progress in the field of computer Go, with the aim of achieving the same levels as computerChessorChinese Chessin the future.