ABSTRACT Remote industrial assistance is increasingly mediated through robotic avatars, yet existing telepresence systems provide limited support for non‐verbal behaviors such as hesitation, urgency, or emphasis. Manually controlling such expressive motion can increase operator workload. We present a feasibility and system‐integration study of a semantic‐to‐motion telepresence pipeline that interprets expert utterances and scene context into parameterized expressive robot behaviors. The system combines visual grounding with language‐based intent inference to produce structured motion descriptors, which are mapped to procedural motion primitives executed in a unity‐based digital twin and streamed to a physical robot via ROS. As the semantic control vocabulary for this prototype, we define SAEH (show, alert, encourage, hesitate), a four‐class operational vocabulary derived from a thematic review of 104 HRI papers. We implement the approach on a Niryo Ned 1 manipulator and show that it can generate and execute kinematically distinct expressive motion profiles, including deictic, warning, supportive, and hesitant behaviors. The proposed framework partially separates communicative intent from robot‐specific execution through parameterized procedural primitives, providing a technical foundation for context‐aware expressive telepresence; however, a measured mean end‐to‐end latency of 10 s currently limits its use to supervisory and asynchronous task loops rather than continuous live interaction.

Training machine learning models to identify human behavior is a difficult yet essential task to develop autonomous and adaptive systems such as smart homes. These models require large and diversified amounts of labeled data to be trained effectively. Due to the high variety of home environments and occupant behaviors, collecting datasets that are representative of all possible homes is a major challenge. In addition, privacy and cost are major hurdles to collect real home data. To avoid these difficulties, one solution consists of training these models using purely synthetic data, which can be generated through the simulation of home and their occupants. Two challenges arise from this approach: designing a methodology with a simulation able to generate credible simulated data and evaluating this credibility. In this article, we explain the methodology used to generate diversified synthetic data of daily activities, through the combination of an agent model to simulate an occupant and a simulated 3D house enriched with sensors and effectors to produce such data. We demonstrate the credibility of the generated synthetic data by comparing their efficacy for training human context understanding models against the efficacy generated by real data. To achieve this, we replicate a real dataset collection setting with our smart home simulator. The occupant is replaced by an autonomous agent following the same experimental protocol used for the real dataset collection. This agent is a BDI-based model enhanced with a scheduler designed to offer a balance between control and autonomy. This balance is useful in synthetic data generation since strong constraints can be imposed on the agent to simulate desired situations while allowing autonomous behaviors outside these constraints to generate diversified data. In our case, the constraints are those imposed during the real dataset collection that we want to replicate. The simulated sensors and effectors were configured to react to the agent’s behaviors similarly to the real ones. We experimentally show that data generated from this simulation are valuable for two human context understanding tasks: current human activity recognition and future human activity prediction. In particular, we show that models trained solely with simulated data can give reasonable predictions about real situations occurring in the original dataset. We also report experimental results regarding statistical analysis and C2ST to assess the credibility of generated data. We discuss the generality of our approach for evaluating the credibility of simulated data from their use as training data.

Augmented Reality (AR) offers new opportunities for interacting with our surroundings. However, creating AR services for specific environments such as homes, factories, or buildings remains challenging for users without development skills, as it involves complex 3D editors and advanced coding workflows. This paper presents a comparative study of three distinct tools, dedicated to such novice users, for prototyping augmented reality AR services. These tools include desktop, Virtual Reality (VR), and AR editors, focusing on the positioning of AR assets. In a between-subjects design experiment, where each user tested only one editor, we used two scenarios (smart-home and smart-building) to assess performance, usability, induced workload, and global user experience for each tool. Additionally, the two scenarios allowed us to examine the impact of the target environment size on the results, with a fivefold difference between the sizes of the two environments tested. Our observations indicate that the AR and VR tools outperformed the desktop editor in several criteria, such as task completion duration, usability and enjoyment, suggesting they not only provide a viable alternative to desktop editors for novice users but could also be prioritized. The differences induced by the scenario and environment size were minimal, suggesting their low impact. Future studies should explore this further with larger differences in environment size.

Prototyping Augmented Reality (AR) applications for smart environments is still a difficult task. Therefore, we propose a pipeline to help designers and developers to create AR applications for monitoring and controlling indoor environments equipped with connected objects. This pipeline starts with the capture (geometry and objects) of the real environment with an AR device. Then, it proposes a Virtual Reality (VR) tool to configure augmentations in this captured environment. This tool includes a feature to simulate AR devices to help anticipate the application’s rendering on real devices. The created application can then be seamlessly deployed on various AR devices including smartphones,tablets and glasses.

In a co-located collaborative virtual environment, multiple users share the same physical tracked space and the same virtual workspace. When the virtual workspace is larger than the real workspace, navigation interaction techniques must be deployed to let the users explore the entire virtual environment. When a user navigates in the virtual space while remaining static in the real space, his/her position in the physical workspace and in the virtual workspace are no longer the same. Thus, in the context where each user is immersed in the virtual environment with a Head-Mounted-Display, a user can still perceive where his/her collaborators are in the virtual environment but not where they are in real world. In this paper, we propose and compare three methods to warn users about the position of collaborators in the shared physical workspace to ensure a proper cohabitation and safety of the collaborators. The frst one is based on a virtual grid shaped as a cylinder, the second one is based on a ghost representation of the user and the last one displays the physical safe-navigation space on the foor of the virtual environment. We conducted a user-study with two users wearing a Head-Mounted-Display in the context of a collaborative First-Person-Shooter game. Our three methods were compared with a condition where the physical tracked space was separated into two zones, one per user, to evaluate the impact of each condition on safety, displacement freedom and global satisfaction of users. Results suggest that the ghost avatar and the cylinder grid can be good alternatives to the separation of the tracked space.

When dealing with real estate business, it is quite difficult for estate agents to make customers understand the potential and the volumes of free spaces. Thus, we propose an application that aims to solve these issues based on a laying out scenario in which a seller and a customer collaborate. As the roles of both users are different, we propose an asymmetric collaboration where the two users do not use the same interaction setup and do not benefit from the same interaction capabilities.