In the early days of the Open Source Robotics Foundation, a lesser-known project aimed to design an “app-able robot”, leading to the creation of the “Capabilities” package for the Robot Operating System (ROS). Over a decade later, formulating robot capabilities remains a significant technical hurdle in bringing robots from the lab into everyday life. This paper introduces Capabilities2, a successor to the original Capabilities package, now reimagined for ROS2. Capabilities2 enhances the original design by enabling advancements in skill-based control techniques and offering a more efficient, extensible framework for defining and utilising robot capabilities. We delve into its application in new real-world scenarios, with a particular focus on human-robot interactions and the deployment of collaborative mobile robots in human-centric environments. Capabilities2 addresses challenges in implementing intuitive, collaborative robots by introducing an abstracted database handler, an object-relational mapping for capability models, and a plugin architecture for capability execution. These features support dynamic capability representation, runtime adaptability, and integration with modern AI techniques for skill-based task planning. By providing a standardised yet flexible framework, Capabilities2 reduces the integration effort required to develop top-level controls for real-world scenarios, facilitating rapid development and deployment. Our contributions include the reimplementation of the Capabilities package in ROS2, enhancements to support contemporary robotic applications, and demonstrations of new use cases enabled by Capabilities2. We believe that Capabilities2 significantly advances the field of robotics by equipping developers with tools to create more capable, adaptable, and interactive robots. Capabilities2 is available at https://github.com/CollaborativeRoboticsLab/capabilities2

Social Presence (SP) is a critical factor in human-robot interaction (HRI), influencing how users perceive and engage with social robots. Traditionally, SP has been conceptualized by focusing on the user's subjective perception of a robot's social capabilities, which we label, Attributed Social Presence (ASP). However, this perspective overlooks the inherent capabilities of robots that can engender a sense of presence, which we argue can be termed, Intrinsic Social Presence (ISP). In this paper we explicitly explore and distinguish between ASP and ISP, highlighting the limitations of solely focusing on attributed user perceptions in HRI. We address the challenges associated with the conceptual ambiguity of SP, the difficulties in benchmarking due to inconsistent definitions, and the lack of research examining robot presence modulation. To overcome these issues, we propose reframing SP as a multidimensional construct encompassing three interconnected domains. By emphasizing ISP and providing a practical framework for its implementation, this paper aims to enhance the development of adaptable, responsive robots capable of meaningful engagement. This approach bridges the gap between rigid robotic behaviour and the fluid, adaptive presence characteristic of human interactions, emphasizing that genuine trust and connection are integral to the full functionality and effectiveness of social robots.

Attention serves as a critical antecedent to social presence, which fundamentally influences acceptance, trust, and overall interaction quality in human-robot interaction (HRI). This paper investigates the development of a gaze modulation framework that enables robots to strategically influence human attention through two complementary Q-learning-based modules: Gaze-Garnering Modulation (GGM) and Gaze-Avoidance Modulation (GAM). To measure gaze feedback, we introduce a novel metric—the Dynamic Gaze Engagement Index (DGEI)—that integrates attention ratio with stationary gaze entropy (SGE) to evaluate not just the quantity but also the quality of visual attention. This feedback allows the system to continuously adapt to each individual’s unique attentional patterns and thresholds, providing personalised interaction. In two experiments, 20 participants interacted with a Pepper robot that dynamically adjusted its behaviours (lights, movements, and voice volume) based on real-time gaze feedback. Results demonstrated that GGM significantly enhanced gaze engagement, fostering strong mutual interaction, while GAM effectively redirected attention when appropriate, with participants reporting lower perceived gaze engagement in this condition. Post-experiment questionnaires using the "Psycho-behavioural Interaction - Perceived Attentional Engagement" section of the Networked Minds Social Presence Inventory (NMSPI) revealed significant differences between conditions (t(18)=2.47, p=0.0238), validating the attention modulation by each module and corroborating the behavioural observations. These findings underscore the importance of adaptive robotic behaviours in facilitating dynamic and unobtrusive interactions.

In human-robot collaboration (HRC) settings, hand motion intention prediction (HMIP) plays a pivotal role in ensuring prompt decision-making, safety, and an intuitive collaboration experience. Precise and robust HMIP with low computational resources remains a challenge due to the stochastic nature of hand motion and the diversity of HRC tasks. This paper proposes a framework that combines hand trajectories and gaze data to foster robust, real-time HMIP with minimal to no training. A novel 3D vector field method is introduced for hand trajectory representation, leveraging minimum jerk trajectory predictions to discern potential hand motion endpoints. This is statistically combined with gaze fixation data using a weighted Naive Bayes Classifier (NBC). Acknowledging the potential variances in saccadic eye motion due to factors like fatigue or inattentiveness, we incorporate stationary gaze entropy to gauge visual concentration, thereby adjusting the contribution of gaze fixation to the HMIP. Empirical experiments substantiate that the proposed framework robustly predicts intended endpoints of hand motion before at least 50% of the trajectory is completed. It also successfully exploits gaze fixations when the human operator is attentive and mitigates its influence when the operator loses focus. A real-time implementation in a construction HRC scenario (collaborative tiling) showcases the intuitive nature and potential efficiency gains to be leveraged by introducing the proposed HMIP into HRC contexts. The opens-ource implementation of the framework is made available at https://github.com/maleenj/hmip_ros.git.

The main contribution of this paper is a novel Extended Kalman Filter (EKF) based localisation scheme that fuses two complementary approaches to outdoor vision based localisation. This EKF is aided by a front end consisting of two Convolutional Neural Networks (CNNs) that provide the necessary perceptual information from camera images. The first approach involves a CNN based extraction of information corresponding to artefacts such as curbs, lane markings, and manhole covers to localise on a vector distance transform representation of a binary image of these ground surface boundaries. The second approach involves a CNN based detection of common environmental landmarks such as tree trunks and light poles, which are represented as point features on a sparse map. Utilising CNNs to obtain higher level information about the environment enables this framework to avoid the typical pitfalls of common vision based approaches that use low level hand crafted features for localisation. The EKF framework makes it possible to deal with false positives and missed detections that are inevitable in a practical CNN, to produce a location estimate together with its associated uncertainty. Experiments using a Personal Mobility Device (PMD) driven in typical suburban streets are presented to demonstrate the effectiveness of the proposed localiser.

This paper presents a novel localisation framework based on an omnidirectional camera, targeted at outdoor urban environments. Bearing only information to persistent and easily observable high-level semantic landmarks (such as lamp-posts, street-signs and trees) are perceived using a Convolutional Neural Network (CNN). The framework utilises an information theoretic strategy to decide the best viewpoint to serve as an input to the CNN instead of the full 360° coverage offered by an omnidirectional camera, in order to leverage the advantage of having a higher field of view without compromising on performance. Environmental landmark observations are supplemented with observations to ground surface boundaries corresponding to high-level features such as manhole covers, pavement edges and lane markings extracted from a second CNN. Localisation is carried out in an Extended Kalman Filter (EKF) framework using a sparse 2D map of the environmental landmarks and Vector Distance Transform (VDT) based representation of the ground surface boundaries. This is in contrast to traditional vision only localisation systems that have to carry out Visual Odometry (VO) or Simultaneous Localisation and Mapping (SLAM), since low level features (such as SIFT, SURF, ORB) do not persist over long time frames due to radical appearance changes (illumination, occlusions etc) and dynamic objects. As the proposed framework relies on highlevel persistent semantic features of the environment, it offers an opportunity to carry out localisation on a prebuilt map, which is significantly more resource efficient and robust. Experiments using a Personal Mobility Device (PMD) driven in a representative urban environment are presented to demonstrate and evaluate the effectiveness of the proposed localiser against relevant state of the art techniques.