Enhancing human-robot interaction (HRI) requires a deep understanding of engagement. This paper explores how to capture engagement within conversational HRI by examining multiple non-verbal communication cues. Specifically, we analyze the engagement of 16 participants in the context of a reminiscence task, which establishes a foundational dialogue between the robot and the user. Head pose and voice activation were the non-verbal communication cues used in this study. We developed a methodology that identifies and correlates these cues with engagement levels during specific epochs of interaction. By contrasting these cues against video annotations and self-reported feedback, we refine our understanding of engagement. Our results demonstrate that a multimodal approach to analysing engagement significantly outperforms unimodal analysis, where 11 out of 16 cases (68%) correlate with engagement, increasing accuracy compared to using them independently. Critically, the study reveals that objective and subjective methods to analyse engagement closely align with users' perceptions of engagement, emphasising the importance of integrated communication strategies in HRI.

In both the United Kingdom and abroad, the demand for robotics professionals is increasing rapidly due to the widespread adoption of automation and artificial intelligence technologies. This highlights the critical need for robotics and artificial intelligence professionals to support the advancement of automation technologies while also addressing its potential impact on the workforce. These trends underscore a growing gap between the demand for robotics professionals and the available talent pool, emphasizing the necessity for more specialised education and training programmes. Historically, undergraduate engineering and computer science programmes in British universities have been designed to focus on conventional technological domains, such as mechanical, electrical, and computing engineering. However, careers in robotics require a comprehensive blend of these fields, alongside training in systems engineering and advanced applications. Therefore, there is a clear need for new educational programmes dedicated to robotics and artificial intelligence. Despite some recent progress in introducing specialised robotics and artificial intelligence programmes in the UK, there is limited systematic research on how these programmes should be structured to align with evolving market and societal needs. This paper examines the structure and learning outcomes of robotics and artificial intelligence programmes offered in the UK. The authors evaluate the similarities and differences among these programmes, identify subject-specific learning outcomes, and outline core topic streams in robotics and artificial intelligence. We analysed 12 undergraduate educational programmes currently offered in the UK with titles explicitly including both ‘Robotics’ and ‘Artificial Intelligence’. Our findings revealed that modules in the areas of artificial intelligence and computer science typically account for approximately one-third of the total available credits in these programmes. This proportion is often comparable to the combined share of modules related to electrical, electronic, and general engineering. However, a preliminary analysis of job advertisement keywords for positions in the areas of robotics in AI in the UK posted on LinkedIn in the during the last quarter of 2024 indicates that over 55% of the keywords fall into the AI or computer science categories. This disparity suggests a potential misalignment between the curricula of these educational programmes and the demands of the job market.

Effective execution of long-horizon tasks with dexterous robotic hands remains a significant challenge in real-world problems. While learning from human demonstrations has shown encouraging results, they require extensive data collection for training. Hence, decomposing long-horizon tasks into reusable primitive skills is a more efficient approach. To achieve so, we developed DexSkills, a novel supervised learning framework that addresses long-horizon dexterous manipulation tasks using primitive skills. DexSkills is trained to recognize and replicate a select set of skills using human demonstration data, which can then segment a demonstrated long-horizon dexterous manipulation task into a sequence of primitive skills to achieve one-shot execution by the robot directly. Significantly, DexSkills operates solely on proprioceptive and tactile data, i.e., haptic data. Our real-world robotic experiments show that DexSkills can accurately segment skills, thereby enabling autonomous robot execution of a diverse range of tasks.

This work presents a method for tactile classification of materials for virtual reality (VR) based robot teleoperation. In our system, a human-operator uses a remotely controlled robot-manipulator with an optical fibre-based tactile and proximity sensor to scan surfaces of objects in a remote environment. Tactile and proximity data and the robot's end-effector state feedback are used for the classification of objects' materials which are then visualized in the VR reconstruction of the remote environment for each object. Machine learning techniques such as random forest, convolutional neural and multi-modal convolutional neural networks were used for material classification. The proposed system and methods were tested with five different materials and classification accuracy of 90 % and more was achieved. The results of material classification were successfully exploited for visualising the remote scene in the VR interface to provide more information to the human-operator.

This article presents an impedance type ankle haptic interface for providing users with an immersive navigation experience in virtual reality (VR). The ankle platform, actuated by an electric motor with feedback control, enables the use of foot-tapping gestures to create a walking experience like a real one and to haptically render different types of walking terrains. Experimental studies demonstrated that the interface can be easily used to generate virtual walking and is capable of rendering terrains, such as hard and soft surfaces, and multi-layer complex dynamic terrains. The designed system is a seated-type VR locomotion interface, therefore allowing its user to maintain a stable seated posture to comfortably navigate a virtual scene.

A key aspect of Virtual Reality (VR) applications is the ability to move in the environment, which relies on the illusion of self-motion to create a good user experience. Self-motion has traditionally been studied in psychophysical studies in which a range of wording has been adopted to describe self-motion. However, it is not clear from current research whether the words used in self-motion studies match study participants’ own intuitions about the experience of self-motion. We argue that the terminology used in self-motion studies should be drawn from a participant perspective to improve validity. We undertook an online study involving VR self-motion and card-sorting with 50 participants to examine current self-motion terminology. We found that participants were not familiar with the concept of self-motion and that the virtual scene itself might suggest different terminology. We suggest how studies on motion perception in VR should be designed to better reflect participants’ vernacular.

Extended Reality (XR) technology - such as virtual and augmented reality - is now widely used in Human Computer Interaction (HCI), social science and psychology experimentation. However, these experiments are predominantly deployed in-lab with a co-present researcher. Remote experiments, without co-present researchers, have not flourished, despite the success of remote approaches for non-XR investigations. This paper summarises findings from a 30-item survey of 46 XR researchers to understand perceived limitations and benefits of remote XR experimentation. Our thematic analysis identifies concerns common with non-XR remote research, such as participant recruitment, as well as XR-specific issues, including safety and hardware variability. We identify potential positive affordances of XR technology, including leveraging data collection functionalities builtin to HMDs (e.g. hand, gaze tracking) and the portability and reproducibility of an experimental setting. We suggest that XR technology could be conceptualised as an interactive technology and a capable data-collection device suited for remote experimentation.

We explored rate mode control for virtual reality (VR) based robot teleoperation with constant and variable mapping of the human-operator’s joystick position to the speed (rate) of the robot’s end-effector. The variable mapping depended on the visual scale of the virtual reconstruction of the remote environment to the scale of the real remote environment. We demonstrated how the rate mode control and variable scaling based on the VR reconstruction scale can be efficiently used for seated VR based robot teleoperation when the operator’s arms are supported to reduce tiredness. The experimental study with five human participants demonstrated that variable mapping allowed participants to teleoperate the robot more effectively, by adjusting the VR visual scale albeit at a cost of increased perceived workload.

With the fast development of wearable IoT devices, their applications are becoming more and more pervasive, ranging from social networking, payment, and navigation to health and activity monitoring. The security of the communication between these devices is essential to protect the transmitted sensitive information from tampering and eavesdropping. With the integration of accelerometers into wearable IoT devices, the gait-based biometric cryptography technology has emerged as a data securing tool for wearables. This article proposes a lightweight noise-based group key generation method, which utilizes the noise signals imposed on the raw acceleration signals to generate an M-bit key with high randomness and bit generation rate. Moreover, a signed sliding window coding (SSWC)-based common feature extraction method was designed to extract the common feature for sharing the generated M-bit key among devices worn on different body parts. Finally, a fuzzy vault-based group key distribution system was implemented and evaluated using a public data set. The performed comprehensive analysis of the proposed key generation and distribution method proved that the binary keys generated via the introduced noise-based procedure have high entropy and can pass both the NIST and Dieharder statistical tests with high efficiency. The experimental results obtained prove the robustness of the proposed SSWC-based common feature extraction method in terms of the similarity and discriminability of intra- and inter-class features, respectively.

Remote characterisation of the environment during physical robot-environment interaction is an important task commonly accomplished in telerobotics. This paper demonstrates how tactile and proximity sensing can be efficiently used to perform automatic crack detection. A custom-designed integrated tactile and proximity sensor is implemented. It measures the deformation of its body when interacting with the physical environment and distance to the environment's objects with the help of fibre optics. This sensor was used to slide across different surfaces and the data recorded during the experiments was used to detect and classify cracks, bumps and undulations. The proposed method uses machine learning techniques (mean absolute value as feature and random forest as classifier) to detect cracks and determine their width. An average crack detection accuracy of 86.46% and width classification accuracy of 57.30% is achieved. Kruskal-Wallis results (p<; 0.001) indicate statistically significant differences among results obtained when analysing only force data, only proximity data and both force and proximity data. In contrast to previous techniques, which mainly rely on visual modality, the proposed approach based on optical fibres is suitable for operation in extreme environments, such as nuclear facilities in which nuclear radiation may damage the electronic components of video cameras.

This work describes a virtual reality (VR) based robot teleoperation framework which relies on scene visualization from depth cameras and implements human-robot and human-scene interaction gestures. We suggest that mounting a camera on a slave robot's end-effector (an in-hand camera) allows the operator to achieve better visualization of the remote scene and improve task performance. We compared experimentally the operator's ability to understand the remote environment in different visualization modes: single external static camera, in-hand camera, in-hand and external static camera, in-hand camera with OctoMap occupancy mapping. The latter option provided the operator with a better understanding of the remote environment whilst requiring relatively small communication bandwidth. Consequently, we propose suitable grasping methods compatible with the VR based teleoperation with the in-hand camera. Video demonstration: https://youtu.be/3vZaEykMS_E.

Video games have proved very valuable in rehabilitation technologies. They guide therapy and keep patients engaged and motivated. However, in order to realize their full potential, a good understanding is required of the players’ motor control. In particular, little is known regarding player behaviour in tasks demanding bimanual interaction. In this work, an experiment was designed to improve the understanding of such tasks. A driving game was developed in which players were asked to guide a differential wheeled robot (depicted as a rocket) along a trajectory. The rocket could be manipulated by using an Xbox controller’s triggers, each supplying torque to the corresponding side of the robot. Such a task is redundant, i.e. there exists an infinite number of input combinations to yield a given outcome. This allows the player to strategize according to their own preference. 10 participants were recruited to play this game and their input data was logged for subsequent analysis. Two different motor strategies were identified: an "intermittent" input pattern versus a "continuous" one. It is hypothesized that the choice of behaviour depends on motor skill and minimization of effort and error. Further testing is necessary to determine the exact relationship between these aspects.

This paper presents the development and evaluation of an MR compatible haptic interface for human motor control studies, which can be easily installed and removed from the scanner room. The interface is actuated by a powerful shielded DC motor located 2.1 m away from the 3T MR scanner. Rotational movements are transmitted to a subject's wrist through preloaded cable transmission which drives the handle unit. The handle of the interface is designed to be adjustable to different hands size, enabling comfortable and natural wrist movements. The nominal achievable wrist torque of the interface is up to 2Nm. The interface is easily transportable due to its design characteristics. A dynamic model of the interface is presented and identified for position and torque control modes. Phantom MR compatibility test in clinical environment showed that the interface is compatible with strong magnetic field and radio frequency emission and its operation does not affect the quality of MR images.

Vection refers to the illusion of self-motion when a significant portion of the visual field is stimulated by visual flow, while body is still. Vection is known to be strong for peripheral vision stimulation and relatively weak for central vision. In this paper, the results of an experimental study of central linear vection with and without vibrotactile feet stimulation are presented. Three types of vibratory stimuli were used: a sinusoidal signal, pink noise, and a chirp signal. Six subjects faced a screen looking at a looming visual flow that suggested virtual forward motion. The results showed that the sensation of self-motion happened faster and its intensity was the strongest for sinusoidal vibrations at constant frequency. For some subjects, a vibrotactile stimulus with an increasing frequency (a chirp) elicited as well a stronger vection. The strength of sensation of self-motion was the lowest in the cases when pink noise vibrations and no vibrotactile stimulation accompanied the visual flow stimulation. Possible application areas are mentioned.

Rate mode commanding together with obstacle based force feedback makes mobile robot teleoperation difficult to stabilize even without time-delay. This paper proposes a method for stable time-delayed teleoperation of a mobile robot. Rate mode teleoperation with three different types of force feedback is considered to develop generally applicable method. We reformulate mobile robot bilateral teleoperation architecture based on recently proposed Time Delayed Power Network framework. It allows clarifying ambiguous energy ports and makes it possible to implement Time Domain Passivity Approach in order to secure the system stability. Experimental results show the effectiveness of the proposed formulation for a mobile robot teleoperation with time-delay.

Despite the great diversity of teleoperator designs and applications, their underlying control systems have many similarities. These similarities can be exploited to enable inter-operability between heterogeneous systems. We have developed a network data specification, the Interoperable Telerobotics Protocol, that can be used for Internet based control of a wide range of teleoperators. In this work we test interoperable telerobotics on the global Internet, focusing on the telesurgery application domain. Fourteen globally dispersed telerobotic master and slave systems were connected in thirty trials in one twenty four hour period. Users performed common manipulation tasks to demonstrate effective master-slave operation. With twenty eight (93%) successful, unique connections the results show a high potential for standardizing telerobotic operation. Furthermore, new paradigms for telesurgical operation and training are presented, including a networked surgery trainer and upper-limb exoskeleton control of micro-manipulators.

This paper presents new feedback force rendering scheme for the bilateral teleoperation of mobile robot. Previous research indicated that the feedback force based on obstacle range information prevented accurate motion control of the mobile robot since human operator's commands were distorted by the feedback force. To solve this problem, a new force rendering approach with variable feedback gain is proposed. In proposed scheme, force feedback gain is adaptively tuned based on measured distances to the obstacle and time derivatives of the distances. Stability of the proposed bilateral teleoperation architecture was analyzed and the performance is proved by simulations. Results of simulation and experimental study proved that the quality of the mobile robot bilateral teleoperation with variable force feedback gain is significantly better than the conventional approach with constant feedback gain.