Biological sequences encode fundamental instructions for the building blocks of life, in the form of DNA, RNA, and proteins. Modeling these sequences is key to understand disease mechanisms and is an active research area in computational biology. Recently, Large Language Models have shown great promise in solving certain biological tasks but current approaches are limited to a single sequence modality (DNA, RNA, or protein). Key problems in genomics intrinsically involve multiple modalities, but it remains unclear how to adapt general-purpose sequence models to those cases. In this work we propose a multi-modal model that connects DNA, RNA, and proteins by leveraging information from different pre-trained modality-specific encoders. We demonstrate its capabilities by applying it to the largely unsolved problem of predicting how multiple \rna transcript isoforms originate from the same gene (i.e. same DNA sequence) and map to different transcription expression levels across various human tissues. We show that our model, dubbed IsoFormer, is able to accurately predict differential transcript expression, outperforming existing methods and leveraging the use of multiple modalities. Our framework also achieves efficient transfer knowledge from the encoders pre-training as well as in between modalities. We open-source our model, paving the way for new multi-modal gene expression approaches.

QDax is an open-source library with a streamlined and modular API for Quality-Diversity (QD) optimisation algorithms in Jax. The library serves as a versatile tool for optimisation purposes, ranging from black-box optimisation to continuous control. QDax offers implementations of popular QD, Neuroevolution, and Reinforcement Learning (RL) algorithms, supported by various examples. All the implementations can be just-in-time compiled with Jax, facilitating efficient execution across multiple accelerators, including GPUs and TPUs. These implementations effectively demonstrate the framework's flexibility and user-friendliness, easing experimentation for research purposes. Furthermore, the library is thoroughly documented and has 93% test coverage.

Quality Diversity (qd) algorithms have been proposed to search for a large collection of both diverse and high-performing solutions instead of a single set of local optima. While early qd algorithms view the objective and descriptor functions as black-box functions, novel tools have been introduced to use gradient information to accelerate the search and improve overall performance of those algorithms over continuous input spaces. However a broad range of applications involve discrete spaces, such as drug discovery or image generation. Exploring those spaces is challenging as they are combinatorially large and gradients cannot be used in the same manner as in continuous spaces. We introduce map-elites with a Gradient-Informed Discrete Emitter (me-gide), which extends qd optimisation with differentiable functions over discrete search spaces. me-gide leverages the gradient information of the objective and descriptor functions with respect to its discrete inputs to propose gradient-informed updates that guide the search towards a diverse set of high quality solutions. We evaluate our method on challenging benchmarks including protein design and discrete latent space illumination and find that our method outperforms state-of-the-art qd algorithms in all benchmarks.

In the context of neuroevolution, Quality-Diversity algorithms have proven effective in generating repertoires of diverse and efficient policies by relying on the definition of a behavior space. A natural goal induced by the creation of such a repertoire is trying to achieve behaviors on demand, which can be done by running the corresponding policy from the repertoire. However, in uncertain environments, two problems arise. First, policies can lack robustness and repeatability, meaning that multiple episodes under slightly different conditions often result in very different behaviors. Second, due to the discrete nature of the repertoire, solutions vary discontinuously. Here we present a new approach to achieve behavior-conditioned trajectory generation based on two mechanisms: First, MAP-Elites Low-Spread (ME-LS), which constrains the selection of solutions to those that are the most consistent in the behavior space. Second, the Quality-Diversity Transformer (QDT), a Transformer-based model conditioned on continuous behavior descriptors, which trains on a dataset generated by policies from a ME-LS repertoire and learns to autoregressively generate sequences of actions that achieve target behaviors. Results show that ME-LS produces consistent and robust policies, and that its combination with the QDT yields a single policy capable of achieving diverse behaviors on demand with high accuracy.

The controller area network (CAN) protocol, used in many modern vehicles for real-time inter-device communications, is known to have cybersecurity vulnerabilities, putting passengers at risk for data exfiltration and control system sabotage. To address this issue, researchers have proposed to utilize security measures based on cryptography and message authentication; unfortunately, such approaches are often too computationally expensive to be deployed in real time on CAN devices. Additionally, they have developed machine learning (ML) techniques to detect anomalies in CAN traffic and thereby prevent attacks. The main disadvantage of existing ML-based techniques is that they either depend on additional computational hardware or they heuristically assume that all communication anomalies are malicious. In this article, we show that tree-based learning ensembles outperform anomaly-based techniques like AutoRegressive Integrated Moving Average (ARIMA) and Z-Score when used to detect attacks that result in increased bus utilization. We evaluated the detection capacity of three tree-based ensembles, Adaboost, gradient boosting, and random forests, and collectively refer to these as DT-DS. We conclude that the decision tree ensemble with Adaboost performs best with an area under curve (AUC) score of 0.999, closely followed by gradient boosting and random forests with 0.997 and 0.991 AUC scores, respectively, when trained using message profiles. We observe that with an increase in the observation window, the DT-DS models present an average AUC score of 0.999, and offer a nearly perfect detection of attacks, at the cost of increased latency in detection of attacked messages. We evaluate the performance of the IDS for Aeronautical Radio, Incorporated– (ARINC) encoded CAN communication traffic in avionic systems, generated using an aerospace testbench, ARINC-825TBv2. The IDS has been evaluated against the active attacks of a state-of-the-art predictive attacker model. Additionally, we observed that the performance of IDS approaches such as ARIMA and Z-Score degrade considerably with a decrease in the size of the observation time window. In contrast, the performance of DT-DS models is consistent, with only an average drop of 0.005 in the AUC score.

In this work, we consider the problem of Quality-Diversity (QD) optimization with multiple objectives. QD algorithms have been proposed to search for a large collection of both diverse and high-performing solutions instead of a single set of local optima. Searching for diversity was shown to be useful in many industrial and robotics applications. On the other hand, most real-life problems exhibit several potentially conflicting objectives to be optimized. Hence being able to optimize for multiple objectives with an appropriate technique while searching for diversity is important to many fields. Here, we propose an extension of the map-elites algorithm in the multi-objective setting: Multi-Objective map-elites (mome). Namely, it combines the diversity inherited from the map-elites grid algorithm with the strength of multi-objective optimizations by filling each cell with a Pareto Front. As such, it allows to extract diverse solutions in the descriptor space while exploring different compromises between objectives. We evaluate our method on several tasks, from standard optimization problems to robotics simulations. Our experimental evaluation shows the ability of mome to provide diverse solutions while providing global performances similar to standard multi-objective algorithms.

We present a novel instance-based approach to handle regression tasks in the context of supervised domain adaptation under an assumption of covariate shift. The approach developed in this paper is based on the assumption that the task on the target domain can be efficiently learned by adequately reweighting the source instances during training phase. We introduce a novel formulation of the optimization objective for domain adaptation which relies on a discrepancy distance characterizing the difference between domains according to a specific task and a class of hypotheses. To solve this problem, we develop an adversarial network algorithm which learns both the source weighting scheme and the task in one feed-forward gradient descent. We provide numerical evidence of the relevance of the method on public data sets for regression domain adaptation through reproducible experiments.