SUMMARY: In the analysis of diverse omics data, a common and important preliminary step involves computing low-dimensional embeddings using techniques such as PCA, UMAP, t-SNE, or variational autoencoders. These embeddings provide a global overview of sample distributions and their relationships, often serving as the basis for formulating biological hypotheses. To facilitate rapid and intuitive exploration of such low-dimensional embeddings, we developed Yomix, an interactive omics-agnostic visualization and data exploration tool. Yomix enables users to flexibly define subsets of interest using a lasso selection tool, instantly compute their feature signatures, and compare their distributions. Yomix is a fast and efficient tool for interactive exploration of diverse omics datasets. AVAILABILITY AND IMPLEMENTATION: Yomix and its documentation are publicly available at https://github.com/perrin-isir/yomix.

In the context of neuroevolution, Quality-Diversity algorithms have proven effective in generating repertoires of diverse and efficient policies by relying on the definition of a behavior space. A natural goal induced by the creation of such a repertoire is trying to achieve behaviors on demand, which can be done by running the corresponding policy from the repertoire. However, in uncertain environments, two problems arise. First, policies can lack robustness and repeatability, meaning that multiple episodes under slightly different conditions often result in very different behaviors. Second, due to the discrete nature of the repertoire, solutions vary discontinuously. Here we present a new approach to achieve behavior-conditioned trajectory generation based on two mechanisms: First, MAP-Elites Low-Spread (ME-LS), which constrains the selection of solutions to those that are the most consistent in the behavior space. Second, the Quality-Diversity Transformer (QDT), a Transformer-based model conditioned on continuous behavior descriptors, which trains on a dataset generated by policies from a ME-LS repertoire and learns to autoregressively generate sequences of actions that achieve target behaviors. Results show that ME-LS produces consistent and robust policies, and that its combination with the QDT yields a single policy capable of achieving diverse behaviors on demand with high accuracy.

A fascinating aspect of nature lies in its ability to produce a large and diverse collection of organisms that are all high-performing in their niche. By contrast, most AI algorithms focus on finding a single eficient solution to a given problem. Aiming for diversity in addition to performance is a convenient way to deal with the exploration-exploitation trade-off that plays a central role in learning. It also allows for increased robustness when the returned collection contains several working solutions to the considered problem, making it well-suited for real applications such as robotics. Quality-Diversity (QD) methods are evolutionary algorithms designed for this purpose. This paper proposes a novel algorithm, qd-pg, which combines the strength of Policy Gradient algorithms and Quality Diversity approaches to produce a collection of diverse and high-performing neural policies in continuous control environments. The main contribution of this work is the introduction of a Diversity Policy Gradient (DPG) that exploits information at the time-step level to drive policies towards more diversity in a sample-efficient manner. Specifically, qd-pg selects neural controllers from a map-elites grid and uses two gradient-based mutation operators to improve both quality and diversity. Our results demonstrate that qd-pg is significantly more sample-eficient than its evolutionary competitors.

International audience

The exploration-exploitation trade-off is at the heart of reinforcement learning (RL). However, most continuous control benchmarks used in recent RL research only require local exploration. This led to the development of algorithms that have basic exploration capabilities, and behave poorly in benchmarks that require more versatile exploration. For instance, as demonstrated in our empirical study, state-of-the-art RL algorithms such as DDPG and TD3 are unable to steer a point mass in even small 2D mazes. In this paper, we propose a new algorithm called "Plan, Backplay, Chain Skills" (PBCS) that combines motion planning and reinforcement learning to solve hard exploration environments. In a first phase, a motion planning algorithm is used to find a single good trajectory, then an RL algorithm is trained using a curriculum derived from the trajectory, by combining a variant of the Backplay algorithm and skill chaining. We show that this method outperforms state-of-the-art RL algorithms in 2D maze environments of various sizes, and is able to improve on the trajectory obtained by the motion planning phase.

We propose a novel reinforcement learning algorithm, AlphaNPI, that incorpo- rates the strengths of Neural Programmer-Interpreters (NPI) and AlphaZero. NPI contributes structural biases in the form of modularity, hierarchy and recursion, which are helpful to reduce sample complexity, improve generalization and in- crease interpretability. AlphaZero contributes powerful neural network guided search algorithms, which we augment with recursion. AlphaNPI only assumes a hierarchical program specification with sparse rewards: 1 when the program execution satisfies the specification, and 0 otherwise. This specification enables us to overcome the need for strong supervision in the form of execution traces and consequently train NPI models effectively with reinforcement learning. The experiments show that AlphaNPI can sort as well as previous strongly supervised NPI variants. The AlphaNPI agent is also trained on a Tower of Hanoi puzzle with two disks and is shown to generalize to puzzles with an arbitrary number of disks. The experiments also show that when deploying our neural network policies, it is advantageous to do planning with guided Monte Carlo tree search.

While only continuous motions are possible, the way in which contacts appear and disappear confers to legged locomotion a characteristic discontinuous nature that is traditionally shared by the algorithms used for legged locomotion planning. In this paper, we show that this discontinuous nature can disappear if the notion of collision is well redefined and we efficiently solve two different practical problems of legged locomotion planning with algorithms based on an approach that establishes a bridge between discrete and continuous planning. The first problem consists of reactive footstep planning with a biped robot and the second one consists of nongaited locomotion planning with a hexapod.

This paper focuses on how to generate dynamic transitions in order to make our robot COMAN (COmpliant huMANoid) dynamically traverse inclined terrains. The novel approach addresses dynamic walking on inclined surfaces by dividing the walking motion into two phases: transition and incline walking. During the transition phase, the humanoid robot performs a 3-dimensional movement in order to transfer its body between surfaces of different inclinations, which is then followed by the incline-walking phase. The transition phase is less trivial to execute than the incline walking itself. In this paper, we first formulate the equations of a 3D (non linear) Inverted Pendulum, and then we derive an equivalent model. Subsequently, we introduce a trajectory generator based on this model and validate it experimentally by performing, with COMAN, dynamic transitions from the horizontal ground to a 10° slope.

This work introduces a novel balance monitoring strategy for humanoid robots. The proposed method addresses the problem of ensuring the balance maintenance of a humanoid robot, through the online monitoring of its state of balance by means of a Lyapunov (energy) function. The proposed method involves the use of dynamical models accounting for both the link and motor states. Energy limits corresponding to the front and rear edges of the support polygon are computed using a closed-loop Lyapunov function. Therefore, this method focuses on the resolution of two issues through a single control scheme, namely, guaranteeing asymptotical stability of the robot at the joint level, in addition to ensuring that it maintains its dynamical balance. A mathematical proof of the previous claims, as well as of the method's validity, is provided in the paper, whereby a direct relationship between the CoP and the system's energy has been established for the first time. Experimental results of step recovery and walking tests performed on the COmpliant huMANoid (COMAN) corroborate the method's applicability and performance as a balance monitor.

In this paper we describe an approach for hu-manoid robot balance recovery that combines a novel attitude control algorithm adding compliance to the robot's behavior and increasing the smoothness of its motion, and an omnidirectional stepping strategy that can trigger one or two steps based on a measured disturbance vector. The proposed method is validated through experiments with the inherently compliant humanoid COMAN.

In this paper we introduce a new bounding box method for footstep planning for humanoid robots. Similar to the classic bounding box method (which uses a single rectangular box to encompass the robot) it is computationally efficient, easy to implement and can be combined with any rigid body motion planning library. However, unlike the classic bounding box method, our method takes into account the stepping over capabilities of the robot, and generates precise leg trajectories to avoid obstacles on the ground. We demonstrate that this method is well suited for footstep planning in cluttered environments.

In this paper, we propose a novel and coherent framework for fast footstep planning for legged robots on a flat ground with 3-D obstacle avoidance. We use swept volume approximations that are computed offline in order to considerably reduce the time spent in collision checking during the online planning phase, in which a rapidly exploring random tree variant is used to find collision-free sequences of half-steps (which are produced by a specific walking pattern generator). Then, an original homotopy is used to smooth the sequences into natural motions, gently avoiding the obstacles. The results are experimentally validated on the robot HRP-2.

We present a new biped walking pattern generator based on "half-steps". Its key features are a) a 3-dimensional parametrization of the input space, and b) a simple homotopy that efficiently smooths the walking trajectory corresponding to a fixed sequence of steps. We show how these features can be ideally combined in the framework of sampling-based footstep planning. We apply our approach to the robot HRP-2 and are able to quickly produce smooth and dynamically stable trajectories that are solutions to a difficult problem of footstep planning.

In this paper we demonstrate an original equivalence between footstep planning problems, where discrete sequences of steps are searched for, and the more classical problem of motion planning for a 2D rigid shape, where a continuous collision-free path has to be found. This equivalence enables a lot of classical motion planning techniques (such as PRM, RRT, etc.) to be applied almost effortlessly to the specific problem of footstep planning for a humanoid robot.

We present here an original approach to test the feasibility of footsteps for a given walking pattern generator. It is based on a new approximation algorithm intended to cope with this specific problem. The result obtained is used on the robot HRP-2, and enables it to guess a step feasibility 40,000 times faster (in 9μs) than with the normal verification process. As a consequence some advance is made towards fast online motion (re)planning based on a continuous set of possible steps.

We demonstrate in this paper our motion generation sheme for the generation of stable bipedal walking motions and we expand it to enhance its flexibility and independency. An algorithm for the control of appropriate orientations of the feet and the trunk permits the robot to turn in a natural and safe way. Polygonal constraints on the positions of the computed feet positions serve to improve its reliability. A logic for the succession of the support phases and an algorithm for the automatic control of their orientations bridge the gap to more autonomy and to more practicability.

Tree automata with one memory have been introduced in 2001. They generalize both pushdown (word) automata and the tree automata with constraints of equality between brothers of Bogaert and Tison. Though it has a decidable emptiness problem, the main weakness of this model is its lack of good closure properties. We propose a generalization of the visibly pushdown automata of Alur and Madhusudan to a family of tree recognizers which carry along their (bottom-up) computation an auxiliary unbounded memory with a tree structure (instead of a symbol stack). In other words, these recognizers, called Visibly Tree Automata with Memory (VTAM) define a subclass of tree automata with one memory enjoying Boolean closure properties. We show in particular that they can be determinized and the problems like emptiness, membership, inclusion and universality are decidable for VTAM. Moreover, we propose several extensions of VTAM whose transitions may be constrained by different kinds of tests between memories and also constraints a la Bogaert and Tison comparing brother subtrees in the tree in input. We show that some of these classes of constrained VTAM keep the good closure and decidability properties, and we demonstrate their expressiveness with relevant examples of tree languages.