Augmented Reality has proven to be a viable solution compared to paper instructions for the supervision of maintenance procedures in industrial contexts to reduce error rates and increase overall performance. With the use of computer vision and cyber physical systems, the validation of specific tasks in AR-supervised environments can be automated to increase process quality even more. However, in the context of human-centered approaches such as industry 5.0, the impact of such methods on operator’s acceptability has yet to be estimated. Therefore, the paper presents a quantitative study on 24 participants to compare 4 modalities of validation of operational procedures. SCRAM, a web application based on WebXR, is developed for the supervision of maintenance operations to compare 2 types of devices: tablets and AR headsets. All scenarios are compared on an assembly use case based on performance criteria and subjective questionnaires such as NASATLX, System Usability Scale (SUS) and Technology Acceptance Model to measure acceptability. In the end, results mostly show significant differences in completion times, usability, and perceived ease of use of the SCRAM application regarding the choice of the device. On the other hand, the study does not highlight any significant difference between scenarios with automated validation of operations and scenarios with manual validation.

Immersive authoring tools have emerged as key enablers for trainers-designers to create Virtual Reality Learning Systems (VRLS) without requiring extensive programming skills. However, the design of such tools presents significant challenges in terms of interaction, usability, and interface complexity. These challenges underscore the need to validate an appropriate design interface to ensure these tools can be effectively utilized by non-technical users. This study evaluates the usability of an immersive authoring tool for VRLS using interactive panoramic videos. Two types of interfaces were compared: one that integrates storyboarding and rendering visualization, and one that separates these functionalities. Quantitative and qualitative data were collected from 24 participants divided into two groups. The results indicated that the separated interface was more effective, efficient, and satisfactory, particularly for more complex scenarios. Moreover, the integrated interface led to more pronounced symptoms of cybersickness while motivation levels did not differ significantly between the two groups. These findings highlight that integrating all functionalities into a single interface may not always be the best approach, especially for complex tasks, as it can lead to decreased usability and increased physical discomfort.

Visual-inertial simultaneous localization and mapping methods (SLAM) process and store large amounts of data based on image sequences to estimate accurate and robust real-time trajectories. Real-time performances, memory management and low power consumption are critical for embedded SLAM with restrictive hardware resources. We aim at reducing the amount of injected input data in SLAM algorithms and, thereby, the memory footprint while providing improved real-time performances. Two decimation approaches are used, constant filtering and adaptive filtering. The first one decimates input images to reduce frame rate (from 20 to 10, 7, 5 and 2 fps). The latter one uses inertial measurements to reduce the frame rate when no significant motion is detected. Applied to SLAM methods, it produces more accurate trajectories than the constant filtering approach, while further reducing the amount of injected data up to 85%. It also impacts the resource utilization by reducing up to an average of 91% the peak of memory consumption.

Embedded simultaneous localization and mapping (SLAM) aims at providing real-time performances with restrictive hardware resources of advanced perception functions. Localization methods based on visible cameras include image processing functions that require frame memory management. This work reduces the dynamic range of input frame and evaluates the accuracy and robustness of real-time SLAM algorithms with quantified frames. We show that the input data can be reduced up to 62% and 75% while maintaining a similar trajectory error lower than 0.15m compared to full precision input images.

This paper introduces a Wearable SLAM system that performs indoor and outdoor SLAM in real time. The related project is part of the MALIN challenge which aims at creating a system to track emergency response agents in complex scenarios (such as dark environments, smoked rooms, repetitive patterns, building floor transitions and doorway crossing problems), where GPS technology is insufficient or inoperative. The proposed system fuses different SLAM technologies to compensate the lack of robustness of each, while estimating the pose individually. LiDAR and visual SLAM are fused with an inertial sensor in such a way that the system is able to maintain GPS coordinates that are sent via radio to a ground station, for real-time tracking. More specifically, LiDAR and monocular vision technologies are tested in dynamic scenarios where the main advantages of each have been evaluated and compared. Finally, 3D reconstruction up to three levels of details is performed.

Recognizing human actions from video sequences is an active research area in computer vision. This paper describes an effective approach to generate compact and informative representations for action recognition. We design a new action feature descriptor inspired from Laban Movement Analysis method. An efficient preprocessing step based on view invariant human motion is presented. Our descriptor is applied in four known machine learning methods, Random Decision Forest, Multi-Layer Perceptron and Multi-class Support Vector Machines (One-Against-One and One-Against-All). Our proposed approach has been evaluated on two challenging benchmarks of action recognition, Microsoft Research Cambridge-12 (MSRC-12) and MSR-Action3D. We follow the same experimental settings to make a direct comparison between the four classifiers and to show the robustness of our descriptor vector. Experimental results demonstrate that our approach outperforms the state-of-the-art methods.

Interactive robotics is a vast and expanding research field. Interactions must be sufficiently natural, with robots having socially acceptable behavior by Humans, adaptable to user expectations. Thus allowing easy integration in our daily lives in various fields (science, industry, health, etc). Natural interaction during Human-Robot collaborative action needs suitable interaction techniques. In our paper we develop a gesture recognition system for natural and intuitive communication between Human and NAO robot. However recognizing meaningful gesture patterns from whole-body gestures is a complex task. That is why we used the Laban Movement Analysis technique to describe high level gestures for NAO tele-operation. The major contributions of the present work is: (1) an efficient preprocessing step based on view invariant Human motion, (2) a robust descriptor vector based on Laban Movement Analysis technique in order to generate compact and informative representations of Human movement, and (3) a gesture recognition system based on Hidden Markov Model method was applied to teleoperate NAO based on our proper database dedicated to the tele-operation of NAO. Our approach was evaluated with two challenging datasets, Microsoft Research Cambridge-12 (MSRC-12) and UTKinect-Action. Experimental results show that our approach outperforms the state-of-the-art methods.

Mixed reality systems overlay real data with virtual information in order to assist users in their current task, they are used in many fields (surgery, maintenance, entertainment). Such systems generally combine several hardware components operating at different time scales, and software that has to cope with these timing constraints. MIRELA, for Mixed Reality Language, is a framework aimed at modelling, analysing and implementing systems composed of sensors, processing units, shared memories and rendering loops, communicating in a well-defined manner and submitted to timing constraints. The paper describes how harmful software behaviour, which may result in possible hardware deterioration or revert the system's primary goal from user assistance to user impediment, may be detected such as (global and local) deadlocks or starvation features. This also includes a study of temporal properties resulting in a finer understanding of the software timing behaviour, in order to fix it if needed.

3D camera tracking is an important issue for many kinds of applications such as augmented reality and robotics navigation. When image sequences are used, tracking needs to be initialized by matching 2D data extracted from images with 3D data representing a priori knowledge of the world. However, this process is difficult and the accuracy of the pose estimation strongly depends on the accuracy of the previous matching step. In this paper, we present two original approaches that achieve a 2D/3D points-based matching in order to initialize the 3D camera tracking. The first one is semi-automatic and requires the intervention of the user. The second one is completely automatic. Both approaches are based on SURF descriptors, which have the advantages of being fast and robust against outliers. A description of these two approaches is given and results obtained from experiments performed on real data are exposed and discussed.

We propose a compositional modeling framework for mixed reality (MR) software architectures in order to express, simulate and validate formally the time depending properties of such systems. Our approach is first based on a functional decomposition of such systems into generic components. The obtained elements as well as their typical interactions give rise to generic representations in terms of timed automata. A whole application is then obtained as a composition of such defined components. To ease writing specifications, we propose a textual language (named MIRELA: mixed reality language) along with the corresponding compilation tools. The generated output contains timed automata in UPPAAL format for simulation and verification of time constraints, and which also may be used to generate source code skeletons for an implementation on a MR platform.

In mobile outdoor augmented reality applications, accurate localization is critical to register virtual augmentations over a real scene. Vision-based approaches provide accurate localization estimates but are still too sensitive to outdoor conditions (brightness changes, occlusions, etc.). This drawback can be overcome by adding other types of sensors. In this work, we combine a GPS and an inertial sensor with a camera to provide accurate localization. We will present the calibration process and we will discuss how to quantify the 3D localization accuracy. Experimental results on real data are presented.

One of the key problems in Augmented Reality systems, is the synchronization of real and virtual world. In the context of our research efforts, we need to augment a view of the real world with virtual elements which first requires to determine position and orientation of the point of view and then compute a virtual view according to this location. Augmented reality uses a lot of different sensors in order to estimate camera or operator’s point of view. These sensors could provide samples faster than the mixing of virtual and real informations could be displayed. We expose here a way to take into account samples that are generated during the mixing process by combining augmented reality and virtual reality techniques. This method is using a post-rendering technique involving a texture carried out by a software graphics pipeline to perform this task mainly devoted to optical see-through augmented reality systems. This algorithm can lead to deformations on the final rendered image. We will compute the theoretical errors introduced by this method and confront these results to those obtained with a simulation test-bench which is implementing our proposal.

One of the key problems in augmented reality systems is registration, that is to say the synchronization of real and virtual world. Augmented reality uses a lot of different sensors in order to estimate camera or operator's point of view. These sensors could provide samples faster than mixing of virtual and real information could be displayed. We expose here a way to fake into account samples that are generated during the mixing process. This method is using a post-rendering technique involving a texture to perform this task. We expose the errors reduction obtained by performing such technique with a simulation test-bench implementing our proposal.