ABSTRACT Remote industrial assistance is increasingly mediated through robotic avatars, yet existing telepresence systems provide limited support for non‐verbal behaviors such as hesitation, urgency, or emphasis. Manually controlling such expressive motion can increase operator workload. We present a feasibility and system‐integration study of a semantic‐to‐motion telepresence pipeline that interprets expert utterances and scene context into parameterized expressive robot behaviors. The system combines visual grounding with language‐based intent inference to produce structured motion descriptors, which are mapped to procedural motion primitives executed in a unity‐based digital twin and streamed to a physical robot via ROS. As the semantic control vocabulary for this prototype, we define SAEH (show, alert, encourage, hesitate), a four‐class operational vocabulary derived from a thematic review of 104 HRI papers. We implement the approach on a Niryo Ned 1 manipulator and show that it can generate and execute kinematically distinct expressive motion profiles, including deictic, warning, supportive, and hesitant behaviors. The proposed framework partially separates communicative intent from robot‐specific execution through parameterized procedural primitives, providing a technical foundation for context‐aware expressive telepresence; however, a measured mean end‐to‐end latency of 10 s currently limits its use to supervisory and asynchronous task loops rather than continuous live interaction.

Training machine learning models to identify human behavior is a difficult yet essential task to develop autonomous and adaptive systems such as smart homes. These models require large and diversified amounts of labeled data to be trained effectively. Due to the high variety of home environments and occupant behaviors, collecting datasets that are representative of all possible homes is a major challenge. In addition, privacy and cost are major hurdles to collect real home data. To avoid these difficulties, one solution consists of training these models using purely synthetic data, which can be generated through the simulation of home and their occupants. Two challenges arise from this approach: designing a methodology with a simulation able to generate credible simulated data and evaluating this credibility. In this article, we explain the methodology used to generate diversified synthetic data of daily activities, through the combination of an agent model to simulate an occupant and a simulated 3D house enriched with sensors and effectors to produce such data. We demonstrate the credibility of the generated synthetic data by comparing their efficacy for training human context understanding models against the efficacy generated by real data. To achieve this, we replicate a real dataset collection setting with our smart home simulator. The occupant is replaced by an autonomous agent following the same experimental protocol used for the real dataset collection. This agent is a BDI-based model enhanced with a scheduler designed to offer a balance between control and autonomy. This balance is useful in synthetic data generation since strong constraints can be imposed on the agent to simulate desired situations while allowing autonomous behaviors outside these constraints to generate diversified data. In our case, the constraints are those imposed during the real dataset collection that we want to replicate. The simulated sensors and effectors were configured to react to the agent’s behaviors similarly to the real ones. We experimentally show that data generated from this simulation are valuable for two human context understanding tasks: current human activity recognition and future human activity prediction. In particular, we show that models trained solely with simulated data can give reasonable predictions about real situations occurring in the original dataset. We also report experimental results regarding statistical analysis and C2ST to assess the credibility of generated data. We discuss the generality of our approach for evaluating the credibility of simulated data from their use as training data.

Augmented Reality (AR) offers new opportunities for interacting with our surroundings. However, creating AR services for specific environments such as homes, factories, or buildings remains challenging for users without development skills, as it involves complex 3D editors and advanced coding workflows. This paper presents a comparative study of three distinct tools, dedicated to such novice users, for prototyping augmented reality AR services. These tools include desktop, Virtual Reality (VR), and AR editors, focusing on the positioning of AR assets. In a between-subjects design experiment, where each user tested only one editor, we used two scenarios (smart-home and smart-building) to assess performance, usability, induced workload, and global user experience for each tool. Additionally, the two scenarios allowed us to examine the impact of the target environment size on the results, with a fivefold difference between the sizes of the two environments tested. Our observations indicate that the AR and VR tools outperformed the desktop editor in several criteria, such as task completion duration, usability and enjoyment, suggesting they not only provide a viable alternative to desktop editors for novice users but could also be prioritized. The differences induced by the scenario and environment size were minimal, suggesting their low impact. Future studies should explore this further with larger differences in environment size.