International audience

International audience

Assessing human movement quality is essential in healthcare for diagnosing and monitoring musculoskeletal disorders. Traditional annotation methods are time-consuming and provide limited insights. In this work we present a web-based annotation tool tailored for efficient expert annotation of human exercises, featuring dual-angle RGB video views coupled with interactive 3D visualization of the movement and patient data integration. Through the concurrent streaming of 2D and 3D visualizations of a movement, the platform aims to streamline dataset creation, enhance annotation precision, and support the development of machine-learning based, Functional Capacity Evaluation (FCE) and rehabilitation systems.

We address the problem of learning imitation policies that generalize across environments sharing the same underlying causal structure between the system dynamics and the task.We introduce a novel loss for learning invariant state representations that draws inspiration from adversarial robustness.Our approach is algorithm-agnostic and does not require knowledge of domain labels.Yet, evaluation in visual and non-visual environments reveals improved zero-shot generalization in the presence of spurious features compared to previous works.

International audience

Abstract We aim to facilitate the daily-life activities of frail or elderly people in collaboration with mobile assistive robots through the means of a digital twin-powered smart home. Being able to quickly and efficiently produce a digital twin of the human user’s environment, can help to further develop personalized assistive solutions. As our first investigation toward this goal, we describe our proof-of-concept “digital twin-driven smart home” implementation. It consists of a virtual representation, robot navigation and environment semantics using open-source software. The initial obtained results on the building process of the digital twin are encouraging and suggest the possibility of integration of digital twin for smart spaces.

The use of deep learning in semantic segmentation of point clouds enables a drastic improvement of segmentation precision. However, available datasets are restrained to a few applications with limited applicability to other fields. Using synthetic and real data can alleviate the burden of creating a dedicated dataset at the cost of domain-shift that is mostly addressed during training, while treating the problem directly on the data has been less explored. Towards this goal, two methods to alleviate domain shift are proposed, firstly by enhanced generation and sampling of synthetic data and secondly by leveraging color information of unlabeled point clouds to color synthetic, uncoloured data. Obtained results confirm their usefulness in improving semantic segmentation result (+3.43 into mIoU for a network trained on S3DIS zone 1). More importantly, the devised coloring method shows the ability of a point-based network to link color information with recurrent geometric features. Finally, the presented methods are able to bridge the domain-shift gap even in cases where inclusion of raw synthetic data during training impedes learning.

Point cloud acquisition techniques are an essential tool for the digitization of industrial plants, yet the bulk of a designer's work remains manual. A first step to automatize drawing generation is to extract the semantics of the point cloud. Towards this goal, we investigate the use of deep learning to semantically segment oil and gas industrial scenes. We focus on domain characteristics such as high variation of object size, increased concavity and lack of annotated data, which hampers the use of conventional approaches. To address these issues, we advocate the use of synthetic data, adaptive downsampling and context sharing.

Assistive robots introduce a new paradigm for developing advanced personalized services. At the same time, the variability and stochasticity of environments, hardware and unknown parameters of the interaction complicates their modelling, as in the case of staircase traversal. For this task, we propose to treat the problem of robot configuration control within a reinforcement learning framework, using policy gradient optimization. In particular, we examine the use of safety or traction measures as a means for endowing the learned policy with desired properties. Using the proposed framework, we present extensive qualitative and quantitative results where a simulated robot learns to negotiate staircases of variable size, while being subjected to different levels of sensing noise.

In order to offer optimal and personalized assistance services to frail people, smart homes or assistive robots must be able to understand the context and activities of users. With this outlook, we propose a vision-based approach for understanding activities of daily living (ADL) through skeleton data captured using an RGB-D camera. Upon decomposition of a skeleton sequence into short temporal segments, activities are classified via a hierarchical two-layer Long-Short Term Memory Network (LSTM) allowing to analyse the sequence at different levels of temporal granularity. The proposed approach is evaluated on a very challenging daily activity dataset wherein we attain superior performance. Our main contribution is a multi-scale, temporal dependency model of activities, founded on a comparison of context features that characterize previous recognition results and a hierarchical representation with a low-level behaviour-unit recognition layer and a high-level units chaining layer.

Staircase detection in an important ability required by indoor robots, allowing for multi-floor exploration in 3D environments. We present an algorithm for stair-case detection from point-cloud data based on a new minimal 3D map representation and the estimation of step-like features that are grouped based on adjacency in order to emerge dominant staircase structures. Experiments performed using noisy RGB-D sensor data from robot exploration trials showed a reliable detection performance under varying conditions.

Social spacing in human-robot interactions is among the main useful features when integrating human social intelligence into robot perception and action skills. One of the main challenges, is to capture the transitions incurred by the human and further take into account robot constraints. Towards this goal, we introduce a novel methodology that can instantiate diverse social spacing models depending on the context and further as a function of uncertainty and robot perception capacity. Our method is based on the use of non-stationary, skew-normal probability density functions for the space of individuals and on treating multi-person space interactions through social mapping. The utility of our approach is shown on an indoor robot operating in the presence of humans, allowing it to exhibit socially intelligent responses.

We present a generative approach to perform robot mapping that is based on an intelligent integration of static and dynamic entity classes within an environment, in order to extrapolate map information at various resolutions. Our framework differentiates from the conventional standpoint where different mapping levels are overlaid on one another, by fusing information from different mapping levels that allows us to infer new information within partially mapped environments. Towards this goal, we develop a class-dependent map extrapolation function that captures the discriminative relation between an environment entity and the mapping procedure. We illustrate the advantages in using heterogeneous contextual information when mapping an environment using a prototype implementation of our approach on an indoor robot platform, giving very promising results.

With robots technology shifting towards entering human populated environments, the need for augmented perceptual and planning robotic skills emerges that complement to human presence. In this integration, perception and adaptation to the implicit human social conventions plays a fundamental role. Toward this goal, we propose a novel framework that can model context-dependent human spatial interactions, encoded in the form of a social map. The core idea of our approach resides in modelling human personal spaces as non-linearly scaled probability functions within the robotic state space and devise the structure and shape of a social map by solving a learning problem in kernel space. The social borders are subsequently obtained as isocontours of the learned implicit function that can realistically model arbitrarily complex social interactions of varying shape and size. We present our experiments using a rich dataset of human interactions, demonstrating the feasibility and utility of the proposed approach and promoting its application to social mapping of human-populated environments.

In this paper, we present an overview of the 3D object retrieval method that we employed in our participation to the generic models track of SHREC 2008 organized by the AIM@SHAPE network of excellence. The proposed methodology is detailed in [2]. Our method is based on a hybrid scheme where 2D features as well as 3D features are extracted from a 3D model which has been previously normalized for rotation using two alternative alignment techniques. The alignment methods that are used are CPCA and NPCA. The 2D features are Fourier coefficients extracted from a set of depth buffers and the 3D features are spherical harmonic coefficients extracted from a spherical function-based representation of a 3D model.