Wearable sensors enable continuous human activity monitoring for health, rehabilitation, and assistive applications. This study investigates the feasibility of a belt-mounted array of multi-placement Inertial Measurement Units (IMUs) for real-time fall detection and activity recognition. A deep learning framework based on Long Short-Term Memory (LSTM) networks is developed and compared against classical machine learning models, including Support Vector Machines (SVM), Random Forest, and XGBoost. The experimental setup employs a custom prototype integrating the Adafruit ICM-20948 IMU sensor across three different devices: a knee-mounted sensor and a waist-mounted sensor, along with the Huzzah32 microcontroller, utilizing Bluetooth Low Energy (BLE) for real-time data transmission. Experimental results show that the LSTM model achieves the highest recognition accuracy of 93.6% using data from a knee-mounted sensor, outperforming all traditional machine learning models such as Random Forest, SVM, and XGBoost. These findings underscore the potential of IMU-based wearable systems for reliable and portable fall detection, contributing to enhanced elderly home care and emergency response applications.

In this paper, we present a technique for recognising hand gestures using data obtained from wearable sensors. A Neural Network takes three input streams - accelerometer data, electromyography (EMG) data, and joint angle data - from sensors affixed to the user’s arm and estimates the gesture the user is performing based on the incoming data. Our proposed method handles each of the three input streams separately using multiple convolution and Long-Short Term Memory (LSTM) layers before concatenating them together and passing them through 3 layers of fully connected neurons. This work was trained on a subset of Database 2 (DB2) Exercise 3 from the Ninapro dataset, composed of data collected from 40 participants performing 23 gestures. This work achieved a classification accuracy of 82.11%. As this falls below the classification accuracy of established literature on similar data, we aim to improve its performance by training and validating the model with a larger quantity of training data.

Texture roughness perception is crucial for autonomous robots to perform manipulation, quality inspection, and material discrimination in unknown environments. This work proposes an approach to combine vibration and force data using the VibroTact sensor for texture roughness classification. Vibration and force data are first processed by CNN and ANN models, and then combined using a Bayesian framework. This approach is evaluated by recognizing 15 textures with different roughness (7 soft and 8 hard textures) using individual ANN and CNN models, and is compared against the Bayesian combination of both methods. Texture data is collected by mounting the VibroTact sensor on a robotic arm and using three sliding exploratory procedures (vertical, diagonal, and circular sliding). The texture roughness recognition results achieve 100% accuracy using the combined approach, which improves the performance of individual ANN and CNN models which range from 87.50% to 100% accuracy. The results also show that diagonal and vertical sliding are optimal for recognizing hard and soft textures, respectively. This approach demonstrates its potential for industrial robotics applications that require texture discrimination.

Peripheral nerve injury often impairs dorsiflexion, leading to asymmetrical gait and increased metabolic cost. While passive ankle-foot orthoses (AFOs) remain common in clinical use, their static, non-adaptive nature limits rehabilitation. Existing active orthoses face challenges such as unreliable gait transition detection, suboptimal actuation, poor generalization, and lack of open-sourcing. This work presents a smart, wearable interface featuring a reproducible design and Bowden cable transmission. A lightweight Bayesian framework enables robust real-time gait transition detection within robot operating system (ROS). The system employs phase-dependent control via a finite state machine, modulating ankle impedance based on gait events. Validation on three unseen users walking naturally demonstrated up to 25% dorsiflexion assistance and phase recognition accuracies of 99.21% (seen interfaces) and 98.75% (unseen), highlighting the system’s adaptability.

Gait phase recognition is vital for advancing assistive robotics, enabling phase-based assistance throughout the gait cycle. This article presents a real-time method using wearable sensors and computational methods for classifying the seven gait subphases. Current methods often struggle with accuracy on unseen subjects. Furthermore, walking speed variability, hardware complexity, and response time hinder robustness, portability, and real-time performance, respectively. A Bayesian method constructs posterior belief by selecting likely phase transition candidates heuristically and combining biomechanical signal knowledge with pattern recognition techniques. The approach is validated and benchmarked against prevailing deep learning (DL) methods using two datasets, each containing data from two inertial measurement units (IMUs) attached to the midshanks of test subjects. The first dataset includes six participants, while the second dataset includes ten, all walking at their comfortable speeds. Additionally, the method is validated in real time for nine test subjects walking at varying speeds (2.2–3.5 mph). The proposed method demonstrates strong robustness, achieving average steady accuracies of 98% and 97.4% for seen and unseen subjects, respectively, with an average runtime of 1.8 ms.

Recognition of gait phases holds great significance in the advancement of assistive robotic technologies. Assistive devices rely on phase-based control for automatic support throughout the gait cycle. Wearable sensors are instrumental in achieving gait phase recognition by providing streams of rich raw data. This paper focuses on the classification of the seven phases of the gait cycle by addressing key challenges in the literature. Existing classification approaches exhibit limited accuracy when applied to unseen data from unseen test subjects, highlighting robustness challenges. Portability and real-time performance are also impacted by challenges in hardware complexity and classifier response time. This paper introduces a hybrid approach that establishes a posterior belief by combining prior biomechanical signal knowledge, heuristics, and pattern recognition. The presented method is validated and benchmarked against Short-Long-Term-Memory (LSTM) model using an open-source dataset consisting of kinematic feedback from 4 inertial measurement units (IMU) attached to the lower limbs of 6 healthy subjects. The proposed approach demonstrates high robustness, achieving average online accuracy of 98.69% and 97.94% for seen and unseen subjects, respectively, with an average run time of 4 ms.

Measuring geometry of the printing road is key for detection of anomalies in 3D printing processes. Although commercial 3D printers can measure the extrusion height using various distance sensors, measuring of the width in real-time remains a challenge. This paper presents a visual in-situ monitoring system to measure width of the printing filament road in 2D patterns. The proposed system is composed of a printable shroud with embedded camera setup and a visual detection approach based on a two-stage instance segmentation method. Each of the segmentation and localization stages can use multiple computational approaches including Gaussian mixture model, color filter, and deep neural network models. The visual monitoring system is mounted on a standard 3D printer and validated with the measurement of printed filament roads of sub-millimeter widths. The results on accuracy and robustness reveal that combinations of deep models for both segmentation and localization stages have better performance. Particularly, fully connected CNN segmentation model combined with YOLO object detector can measure sub-millimeter extrusion width with 90 μm accuracy at 125 ms speed. This visual monitoring system has potential to improve the control of printing processes by the real-time measurement of printed filament geometry.

Data availability is a major obstacle to the successful application of artificial intelligence. In chemistry, pharmaceutical and materials science, generating labelled data on scale relies on expensive screening approaches that require major human and/or economic investments. Although industry uses sophisticated autonomous systems for such tasks, tight intellectual property concerns and strict confidentiality agreements limit data accessibility, thus restricting innovation and hindering the concept of ”open science”. Without automation, data collection is laborious and sample characterization often suffers from subjectivity introduced by the operator. This study tackles the challenge of pharmaceutical particle characterization by modifying a 3D printer to enable rapid, autonomous sample characterization using light microscopy. The system uses low-cost hardware and open-source software in a platform that is accessible and reproducible for the scientific community. As well as increasing throughput, this system uses deep learning to assign sample labels in real-time overcoming bias from subjective labelling, which hinders model performance. Different neural architectures were applied, including convolution and attention blocks, to maximize performance and demonstrate the transferability of the system. This autonomous system effectively characterized pharmaceutical crystal morphology, an area where subjective labelling and limited data have hindered applying machine learning models. By releasing this platform, researchers without access to sophisticated automation platforms can carry out larger in-house screening efforts to generate datasets for developing data-driven, intelligent models.

This work presents a framework for Human-Robot Collaboration (HRC) in assembly tasks that uses multimodal sensors, perception and control methods. First, vision sensing is employed for user identification to determine the collaborative task to be performed. Second, assembly actions and hand gestures are recognised using wearable inertial measurement units (IMUs) and convolutional neural networks (CNN) to identify when robot collaboration is needed and bring the next object to the user for assembly. If collaboration is not required, then the robot performs a solo task. Third, the robot arm uses time domain features from tactile sensors to detect when an object has been touched and grasped for handover actions in the assembly process. These multimodal sensors and computational modules are integrated in a layered control architecture for HRC collaborative assembly tasks. The proposed framework is validated in real-time using a Universal Robot arm (UR3) to collaborate with humans for assembling two types of objects 1) a box and 2) a small chair, and to work on a solo task of moving a stack of Lego blocks when collaboration with the user is not needed. The experiments show that the robot is capable of sensing and perceiving the state of the surrounding environment using multimodal sensors and computational methods to act and collaborate with humans to complete assembly tasks successfully.

Wearable haptic technology plays a key role to enhance the feeling of immersion in virtual reality, telepresence, telehealth and entertainment systems. This work presents a wearable fingertip capable of providing touch, sliding and vibration feedback while the user interacts with virtual objects. This multimodal feedback is applied to the human fingertip using an array of servo motors, a coin vibration motor and 3D printed components. The wearable fingertip uses a 3D printed cylinder that moves up and down to provide touch feedback, and rotates in left and right directions to deliver sliding feedback. The direction of movement and speed of rotation of the cylinder are controlled by the exploration movements performed by the user hand and finger. Vibration feedback is generated using a coin vibration motor with the frequency controlled by the type of virtual material explored by the user. The Leap Motion module is employed to track the human hand and fingers to control the feedback delivered by the wearable device. This work is validated with experiments for exploration of virtual objects in Unity. The experiments show that this wearable haptic device offers an alternative platform with the potential of enhancing the feeling and experience of immersion in virtual reality environments, exploration of objects and telerobotics.

This work presents a method for recognition of walking activities and prediction of gait periods using wearable sensors. First, a Convolutional Neural Network (CNN) is used to recognise the walking activity and gait period. Second, the output of the CNN is used by a Predicted Information Gain (PIG) method to predict the next most probable gait period while walking. The output of these two processes are combined to adapt the recognition accuracy of the system. This adaptive combination allows us to achieve an optimal recognition accuracy over time. The validation of this work is performed with an array of wearable sensors for the recognition of level-ground walking, ramp ascent and ramp descent, and prediction of gait periods. The results show that the proposed system can achieve accuracies of 100% and 99.9% for recognition of walking activity and gait period, respectively. These results show the benefit of having a system capable of predicting or anticipating the next information or event over time. Overall, this approach offers a method for accurate activity recognition, which is a key process for the development of wearable robots capable of safely assist humans in activities of daily living.

Based on hierarchical Multilayer Extreme Learning Machine (ML-ELM) and Fuzzy Logic theory (FL), in this paper a Multilayer Fuzzy Extreme Learning Machine (ML-FELM) has been developed with an application to active classification and transport of objects using an indoors Unmanned Aerial Vehicle (UAV). The learning approach that follows the proposed ML-FELM is a forward two-step hierarchical methodology. First, by stacking a number of Fuzzy Autoencoders (FAEs), input data is projected into a feature representation space. Each FAE is functionally equivalent to a Mamdani Fuzzy Logic System of type-1 (T1 FLS). Finally, in the second stage, features achieved by stacking a number of FAEs are classified by using a Fuzzy ELM (FELM) based on T1 FLS theory and ELM. To evaluate the effectiveness of the proposed ML-FELM, a number of other existing machine learning approaches were employed for the active classification and transport of four different geometrical objects. To further ensure the efficiency of the ML-ELM, a number of popular benchmark data sets for classification problems are also suggested. Based on our experimental results and compared to other deep learning strategies, the ML-FELM not only represents a fast machine learning approach, but also produces a high model accuracy for image classification.

In this paper, an evolutionary General Type-2 Radial Basis Function Neural Network (GT2-RBFNN) for trajectory planning in Remotely Operated underwater Vehicles (ROVs) is suggested. The GT2-RBFNN is used as a data-driven learning system to orient the current position of an ROV in underwater environments. To determine the parameters of GT2-RBFNN, Galactic Swarm Optimisation (GSO) was implemented. A BlueROV2 and a squared water container of 2.5m×2.5m×3.5m were employed to run all experiments. To control the ROV position, a sensory system that consists of a compass, a micro data sonar, a ping sonar and a pressure sensor was integrated. First, a Proportional Derivative fuzzy controller was implemented to control the depth and yaw positions of the ROV. Secondly, the GT2-RBFNN was applied to discriminate between two different types of contours, i.e. corners and walls in order to follow an obstacle-free trajectory. To compare the efficiency of the GT2-RBFNN, a number of learning techniques that are based on Extreme Learning Machine (ELM) and evolutionary optimisation were implemented. Based on our results, a high trade-off between model simplicity and low computational burden are provided by the GT2-RBFNN.

This paper presents a context-based approach for the recognition of transition between activities of daily living (ADLs) using wearable sensor data. A Bayesian method is implemented for the recognition of 7 ADLs with data from two wearable sensors attached to the lower limbs of subjects. A second Bayesian method recognises 12 transitions between the ADLs. The second recognition module uses both, data from wearable sensors and the activity recognised from the first Bayesian module. This approach analyses the next most probable transitions based on wearable sensor data and the context or current activity being performed by the subject. This work was validated using the ENABL3S Database composed of data collected from 7 ADLs and 12 transitions performed by participants walking on two circuits composed of flat surfaces, ascending and descending ramps and stairs. The recognition of activities achieved an accuracy of 98.3%. The recognition of transitions between ADLs achieved an accuracy of 98.8%, which improved the 95.3% accuracy obtained when the context or current activity is not considered for the recognition process. Overall, this work proposes an approach capable of recognising transitions between ADLs, which is required for the development of reliable wearable assistive robots.

In this paper, an active control method for visual object exploration and recognition with an unmanned aerial vehicle is presented. This work uses a convolutional neural network for visual object recognition, where input images are obtained with an unmanned aerial vehicle from multiple objects. The object recognition task is an iterative process actively controlled by a saliency map module, which extracts interesting object regions for exploration. The active control allows the unmanned aerial vehicle to autonomously explore better object regions to improve the recognition accuracy. The iterative exploration task stops when the probability from the convolutional neural network exceeds a decision threshold. The active control is validated with offline and real-time experiments for visual exploration and recognition of five objects. Furthermore, passive exploration is also tested for performance comparison. Experiments show that the unmanned aerial vehicle is capable to autonomously explore interesting object regions. Results also show an improvement in recognition accuracy from 88.14% to 95.66% for passive and active exploration, respectively. Overall, this work offers a framework to allow robots to autonomously decide where to move and look next, to improve the performance during a visual object exploration and recognition task.

A wearable ankle robot prototype for assistance to foot drop is presented in this work. This device is built with soft and hard materials and employs one inertial sensor. First, the ankle robot uses a high-level method, developed with a Bayesian formulation, for recognition of walking activities and gait periods. Second, a low-level method, with a proportional-integral-derivative controller (PID), controls the wearable device to operate in assistive and transparent modes. In an assistive mode, activated by the toe-off detection, the wearable device assists the human foot in dorsiflexion orientation to reduce the effect of foot drop abnormality. In a transparent mode, activated by the heel-contact detection, the robot device follows the movements performed by the human foot. The wearable prototype is validated with experiments, in simulation and real-time modes, for recognition of walking activity and control of assistive and transparent modes during walking. Experiments achieved 99.87% and 99.20% accuracies for recognition of walking activity and gait periods. Results also show the ability of the wearable robot to operate according to the gait period recognised during walking. Overall, this work offers a wearable robot prototype with the potential to assist the human foot during walking, which is important to allow subjects to recover their confidence and quality of life.

• High-level probabilistic identification of sit-to-stand. • High-level probabilistic tracking of the human body during the transition state. • Adaptable method for identification of sit-to-stand using one wearable sensor. • Autonomous decisions and actions for control of assistive robots. • Interaction of high- and low-level methods for control of assistive robots. Identification of human movements is crucial for the design of intelligent devices capable to provide assistance. In this work, a Bayesian formulation, together with a sequential analysis method, is presented for identification of sit-to-stand (SiSt) and stand-to-sit (StSi) activities. This method performs autonomous iterative accumulation of sensor measurements and decision-making processes, while dealing with noise and uncertainty present in sensors. First, the Bayesian formulation is able to identify sit, transition and stand activity states. Second, the transition state, divided into transition phases, is used to identify the state of the human body during SiSt and StSi. These processes employ acceleration signals from an inertial measurement unit attached to the thigh of participants. Validation of our method with experiments in offline, real-time and a simulated environment, shows its capability to identify the human body during SiSt and StSi with an accuracy of 100% and mean response time of 50 ms (5 sensor measurements). In the simulated environment, our approach shows its potential to interact with low-level methods required for robot control . Overall, this work offers a robust framework for intelligent and autonomous systems , capable to recognise the human intent to rise from and sit on a chair, which is essential to provide accurate and fast assistance.

This paper proposes a new General Type-2 Radial Basis Function Neural Network (GT2-RBFNN) that is functionally equivalent to a GT2 Fuzzy Logic System (FLS) of either Takagi-Sugeno-Kang (TSK) or Mamdani type. The neural structure of the GT2-RBFNN is based on the α-planes representation, in which the antecedent and consequent part of each fuzzy rule uses GT2 Fuzzy Sets (FSs). To reduce the iterative nature of the Karnik-Mendel algorithm, the Enhaned-Karnik-Mendel (EKM) type-reduction and three popular direct-defuzzification methods, namely the 1) Nie-Tan approach (NT), the 2) Wu-Mendel uncertain bounds method (WU) and the 3) Biglarbegian-Melek-Mendel algorithm (BMM) are used. Hence, this paper provides four different architectures of the GT2-RBFNN and their parametric optimisation. Such optimisation is a two-stage methodology that first implements an Iterative Information Granulation (IIG) approach to estimate the antecedent parameters of each fuzzy rule. Secondly, each consequent part and the fuzzy rule base of the GT2-RBFNN is optimised using an Adaptive Gradient Descent method (AGD) respectively. A number of popular benchmark data sets, the identification of a nonlinear system and the prediction of chaotic time series are considered. The reported comparative analysis of experimental results is used to evaluate the performance of the suggested GT2 RBFNN with respect to other popular methodologies.

Evolutionary Extreme Learning Machine (E-ELM) is frequently much more efficient than traditional gradient-based algorithms for the parameter identification of feedforward neural networks. In particular, E-ELM is usually faster and provides a higher trade-off between accuracy and model simplicity. For that reason, this paper shows that an E-ELM that is based on Particle Swarm Optimisation (PSO) and Extreme Learning machine (ELM) can be extended to the Interval Type-2 Radial Basis Function Neural Network (IT2-RBFNN) with a Karnik-Mendel type-reduction layer. To evaluate the efficiency of E-ELM, the IT2-RBFNN is used as an Interval Type-2 Fuzzy Logic System (IT2 FLS) for the modelling of two popular benchmark data sets and for the prediction of chaotic time series. According to our results, E-ELM applied to the IT2-RBFNN not only outperforms adaptive-gradient-based algorithms and provides a better generalisation compared to other existing IT2 fuzzy methodologies, but similarly to pure fuzzy models, the IT2-RBFNN is also able to preserve some model interpretation and transparency.

Autonomous exploration in robotics is a crucial feature to achieve robust and safe systems capable to interact with and recognize their surrounding environment. In this paper, we present a method for object recognition using a three-fingered robotic hand actively exploring interesting object locations to reduce uncertainty. We present a novel probabilistic perception approach with a Bayesian formulation to iteratively accumulate evidence from robot touch. Exploration of better locations for perception is performed by familiarity and novelty exploration behaviors, which intelligently control the robot hand to move toward locations with low and high levels of interestingness, respectively. These are active behaviors that, similar to the exploratory procedures observed in humans, allow robots to autonomously explore locations they believe that contain interesting information for recognition. Active behaviors are validated with object recognition experiments in both offline and real-time modes. Furthermore, the effects of inhibiting the active behaviors are analyzed with a passive exploration strategy. The results from the experiments demonstrate the accuracy of our proposed methods, but also their benefits for active robot control to intelligently explore and interact with the environment.

A robust strategy for recognition and prediction of gait events using wearable sensors is presented in this paper. The strategy adopted here uses a combination of two computational intelligence approaches: Adaptive Neuro-Fuzzy and Bayesian methods. Recognition of gait events is performed by a Bayesian method which iteratively accumulates evidence to reduce uncertainty from sensor measurements. Prediction of gait events is based on the observation of decisions and actions made over time by our perception system. An Adaptive Neuro-Fuzzy system evaluates the reliability of predictions, learns a weighting parameter and controls the amount of predicted information to be used by our Bayesian method. Thus, this strategy ensures the achievement of better recognition and prediction performance in both accuracy and speed. The methods are validated with experiments for recognition and prediction of gait events with different walking activities, using data from wearable sensors attached to lower limbs of participants. Overall, results show the benefits of our combined Adaptive Neuro-Fuzzy and Bayesian strategy to achieve fast and accurate decisions, but also to evaluate and adapt its own performance, making it suitable for the development of intelligent assistive and rehabilitation robots.

In this work, we present an adaptive perception method to improve the performance in accuracy and speed of a tactile exploration task. This work extends our previous studies on sensorimotor control strategies for active tactile perception in robotics. First, we present the active Bayesian perception method to actively reposition a robot to accumulate evidence from better locations to reduce uncertainty. Second, we describe the adaptive perception method that, based on a forward model and a predicted information gain approach, allows to the robot to analyse `what would have happened' if a different decision `would have been made' at previous decision time. This approach permits to adapt the active Bayesian perception process to improve the performance in accuracy and reaction time of an exploration task. Our methods are validated with a contour following exploratory procedure with a touch sensor. The results show that the adaptive perception method allows the robot to make sensory predictions and autonomously adapt, improving the performance of the exploration task.

In this paper, we present a Bayesian approach for perception of touch and control of robot emotion. Touch is an important sensing modality for the development of social robots, and it is used in this work as stimulus through a human-robot interaction. A Bayesian framework is proposed for perception of various types of touch. This method together with a sequential analysis approach allow the robot to accumulate evidence from the interaction with humans to achieve accurate touch perception for adaptable control of robot emotions. Facial expressions are used to represent the emotions of the iCub humanoid. Emotions in the robotic platform, based on facial expressions, are handled by a control architecture that works with the output from the touch perception process. We validate the accuracy of our system with simulated and real robot touch experiments. Results from this work show that our method is suitable and accurate for perception of touch to control robot emotions, which is essential for the development of sociable robots.

In this paper, we present a work on control of robot emotional expression using touch sensing. A tactile Bayesian framework is proposed for recognition of different types of touch gestures. We include a sequential analysis method that, based on the accumulation of evidence from tactile interaction, allows to achieve accurate results for recognition of touch. Input data to our method is obtained from touch sensing, which is an important modality for social robotics. Here, emotion in the robot platform are represented by facial expressions, that are handled by a developed control architecture. We validate our method with experiments on tactile interaction in simulated and real robot environments. Results demonstrate that our proposed method is suitable and accurate for control of robot emotions through interaction with humans using touch sensing. Furthermore, it is demonstrated the potential that touch provides as a non-verbal communication channel for the development of social robots capable to interact with humans.

In this paper, we present an intrinsic motivation approach applied to haptics in robotics for tactile object exploration and recognition. Here, touch is used as the sensation process for contact detection, whilst proprioceptive information is used for the perception process. First, a probabilistic method is employed to reduce uncertainty present in tactile measurements. Second, the object exploration process is actively controlled by intelligently moving the robot hand towards interesting locations. The active behaviour performed with the robotic hand is achieved by an intrinsic motivation approach, which permitted to improve the accuracy for object recognition over the results obtained by a fixed sequence of exploration movements. The proposed method was validated in a simulated environment with a Monte Carlo method, whilst for the real environment a three-fingered robotic hand and various object shapes were employed. The results demonstrate that our method is robust and suitable for haptic perception in autonomous robotics.

Motivated by the impact of superresolution methods for imaging, we undertake a detailed and systematic analysis of localization acuity for a biomimetic fingertip and a flat region of tactile skin. We identify three key factors underlying superresolution that enable the perceptual acuity to surpass the sensor resolution: 1) the sensor is constructed with multiple overlapping, broad but sensitive receptive fields; 2) the tactile perception method interpolates between receptors (taxels) to attain subtaxel acuity; and 3) active perception ensures robustness to unknown initial contact location. All factors follow from active Bayesian perception applied to biomimetic tactile sensors with an elastomeric covering that spreads the contact over multiple taxels. In consequence, we attain extreme superresolution with a 35-fold improvement of localization acuity (0.12 mm) over sensor resolution (4 mm). We envisage that these principles will enable cheap high-acuity tactile sensors that are highly customizable to suit their robotic use. Practical applications encompass any scenario where an end-effector must be placed accurately via the sense of touch.

In this work, we present an active tactile perception approach for contour following based on a probabilistic framework. Tactile data were collected using a biomimetic fingertip sensor. We propose a control architecture that implements a perception-action cycle for the exploratory procedure, which allows the fingertip to react to tactile contact whilst regulating the applied contact force. In addition' the fingertip is actively repositioned to an optimal position to ensure accurate perception. The method is trained off-line and then the testing performed on-line based on contour following around several different test shapes. We then implement object recognition based on the extracted shapes. Our active approach is compared with a passive approach, demonstrating that active perception is necessary for successful contour following and hence shape recognition.

In this paper, we propose that active perception will help attain autonomous robotics in unstructured environments by giving robust perception. We test this claim with a biomimetic fingertip that senses surface texture under a range of contact depths. We compare the performance of passive Bayesian perception with a novel approach for active perception that includes a sensorimotor loop for controlling sensor position. Passive perception at a single depth gave poor results, with just 0.2mm uncertainty impairing performance. Extending passive perception over a range of depths gave non-robust performance. Only active perception could give robust, accurate performance, with the sensorimotor feedback compensating the position uncertainty. We expect that these results will extend to other stimuli, so that active perception will offer a general approach to robust perception in unstructured environments.

Cooperation1is at the core of human social life. In this context, two major challenges face research on humanrobot interaction: the first is to understand the underlying structure of cooperation, and the second is to build, based on this understanding, artificial agents that can successfully and safely interact with humans. Here we take a psychologically grounded and human-centered approach that addresses these two challenges. We test the hypothesis that optimal cooperation between a naïve human and a robot requires that the robot can acquire and execute a joint plan, and that it communicates this joint plan through ecologically valid modalities including spoken language, gesture and gaze. We developed a cognitive system that comprises the human-like control of social actions, the ability to acquire and express shared plans and a spoken language stage. In order to test the psychological validity of our approach we tested 12 naïve subjects in a cooperative task with the robot. We experimentally manipulated the presence of a joint plan (vs. a solo plan), the use of task-oriented gaze and gestures, and the use of language accompanying the unfolding plan. The quality of cooperation was analyzed in terms of proper turn taking, collisions and cognitive errors. Results showed that while successful turn taking could take place in the absence of the explicit use of a joint plan, its presence yielded significantly greater success. One advantage of the solo plan was that the robot would always be ready to generate actions, and could thus adapt if the human intervened at the wrong time, whereas in the joint plan the robot expected the human to take his/her turn. Interestingly, when the robot represented the action as involving a joint plan, gaze provided a highly potent nonverbal cue that facilitated successful collaboration and reduced errors in the absence of verbal communication. These results support the cooperative stance in human social cognition, and suggest that cooperative robots should employ joint plans, fully communicate them in order to sustain effective collaboration while being ready to adapt if the human makes a midstream mistake.

In a series of papers, we have formalized an active Bayesian perception approach for robotics based on recent progress in understanding animal perception. However, an issue for applied robot perception is how to tune this method to a task, using: (i) a belief threshold that adjusts the speed-accuracy tradeoff; and (ii) an active control strategy for relocating the sensor e.g. to a preset fixation point. Here we propose that these two variables should be learnt by reinforcement from a reward signal evaluating the decision outcome. We test this claim with a biomimetic fingertip that senses surface curvature under uncertainty about contact location. Appropriate formulation of the problem allows use of multi-armed bandit methods to optimize the threshold and fixation point of the active perception. In consequence, the system learns to balance speed versus accuracy and sets the fixation point to optimize both quantities. Although we consider one example in robot touch, we expect that the underlying principles have general applicability.

In this work, we apply active Bayesian perception to angle and position discrimination and extend the method to perform actions in a sensorimotor task using a biomimetic fingertip. The first part of this study tests active perception off-line with a large dataset of edge orientations and positions, using a Monte Carlo validation to ascertain the classification accuracy. We observe a significant improvement over passive methods that lack a sensorimotor loop for actively repositioning the sensor. The second part of this study then applies these findings about active perception to an example sensorimotor task in real-time. Using an appropriate online sensorimotor control architecture, the robot made decisions about what to do next and where to move next, which was applied to a contour-following task around several objects. The successful outcome of this simple but illustrative task demonstrates that active perception can be of practical benefit for tactile robotics.

Recent advances in modeling animal perception has motivated an approach of Bayesian perception applied to biomimetic robots. This study presents an initial application of Bayesian perception on an iCub fingertip sensor mounted on a dedicated positioning robot. We systematically probed the test system with five cylindrical stimuli offset by a range of positions relative to the fingertip. Testing the real-time speed and accuracy of shape and position discrimination, we achieved sub-millimeter accuracy with just a few taps. This result is apparently the first explicit demonstration of perceptual hyperacuity in robot touch, in that object positions are perceived more accurately than the taxel spacing. We also found substantial performance gains when the fingertip can reposition itself to avoid poor perceptual locations, which indicates that improved robot perception could mimic active perception in animals.