Tools and other objects offer agents a range of potential actions, commonly referred to as affordances. Each tool is typically designed with a primary purpose in mind-like a hammer’s function to drive nails. However, tools can also serve purposes beyond their original design. These alternative uses represent secondary affordances, extending the tool’s utility beyond its primary intended function. While prior robotics research on affordance perception and learning has primarily focused on primary affordances, our work addresses the less-explored area of learning secondary tool affordances from human partners. Using the iCub robot equipped with three cameras, we observed humans performing actions on twenty objects using four different tools in ways that deviate from their primary purposes. For example, the iCub observed humans using rulers not for measuring but to push, pull, and move objects. In this setting, we constructed a dataset by taking pictures of objects before and after each action is executed. To model secondary affordance learning, we trained three neural networks (ResNet-18, ResNet-50, and ResNet-101) on three prediction tasks using these raw images as input: (1) identifying which tool was used to move an object, (2) predicting the tool with additional action category information, and (3) jointly predicting both the tool and action performed. Our results demonstrate that deep learning architectures enable the iCub robot to successfully predict secondary tool affordances, thereby paving the road for human-robot collaborative object manipulation involving complex affordances. Code and data from this study are available at https://github.com/BosongDing/second_affordance

Trust is an essential concept for human-human and human-robot interactions. Yet only a few studies have addressed this concept from a robot perspective -that is, forming robot trust in interaction partners. Our previous robot trust model relies on assessing the trustworthiness of the interaction partners based on the computational cognitive load incurred during the interactive task [1]. However, this model does not take into account the social markers indicative of trustworthiness, such as the gestures displayed by a human partner. In this study, we make a step toward this point by extending the model by integrating a social cue processing module to achieve social human-robot interaction. This new model serves as a novel social cognitive trust framework to enable the Pepper robot to evaluate the trustworthiness of its interaction partners based on both cognitive load (i.e., the cost of perceptual processing) and social cues (i.e., their gestures). For evaluating the efficacy of the framework, the Pepper robot with the developed model is put to interact with human partners who may take the roles of a reliable, unreliable, deceptive, or random suggestion providing partner. Overall, the results indicate that the proposed framework allows the Pepper robot to differentiate the guiding strategies of the partners by detecting deceptive partners and thus select a trustworthy partner in case of a free choice to perform the next task.

In this study, we extend our robot trust model into a multimodal setting in which the Nao robot leverages audio-visual data to perform a sequential multimodal pattern recalling task while interacting with a human partner who has different guiding strategies: reliable, unreliable, and random. Here, the humanoid robot is equipped with a multimodal auto-associative memory module to process audio-visual patterns to extract cognitive load (i.e., computational cost) and an internal reward module to perform cost-guided reinforcement learning. After interactive experiments, the robot associates a low cognitive load (i.e., high cumulative reward) yielded during the interaction with high trustworthiness of the guiding strategy of the partner. At the end of the experiment, we provide a free choice to the robot to select a trustworthy instructor. We show that the robot forms trust in a reliable partner. In the second setting of the same experiment, we endow the robot with an additional simple theory of mind module to assess the efficacy of the instructor in helping the robot perform the task. Our results show that the performance of the robot is improved when the robot bases its action decisions on factoring in the instructor assessment.

Forming trust in a biological or artificial interaction partner that provides reliable strategies and employing the learned strategies to scaffold another agent are critical problems that are often addressed separately in human-robot and robot-robot interaction studies. In this paper, we provide a unified approach to address these issues in robot-robot interaction settings. To be concrete, we present a trust-based affective computational account of scaffolding while performing a sequential visual recalling task. In that, we endow the Pepper humanoid robot with cognitive modules of auto-associative memory and internal reward generation to implement the trust model. The former module is an instance of a cognitive function with an associated neural cost determining the cognitive load of performing visual memory recall. The latter module uses this cost to generate an internal reward signal to facilitate neural cost-based reinforcement learning (RL) in an interactive scenario involving online instructors with different guiding strategies: reliable, less-reliable, and random. These cognitive modules allow the Pepper robot to assess the instructors based on the average cumulative reward it can collect and choose the instructor that helps reduce its cognitive load most as the trustworthy one. After determining the trustworthy instructor, the Pepper robot is recruited to be a caregiver robot to guide a perceptually limited infant robot (i.e., the Nao robot) that performs the same task. In this setting, we equip the Pepper robot with a simple theory of mind module that learns the state-action-reward associations by observing the infant robot’s behavior and guides the learning of the infant robot, similar to when it went through the online agent-robot interactions. The experiment results on this robot-robot interaction scenario indicate that the Pepper robot as a caregiver leverages the decision-making policies – obtained by interacting with the trustworthy instructor– to guide the infant robot to perform the same task efficiently. Overall, this study suggests how robotic-trust can be grounded in human-robot or robot-robot interactions based on cognitive load, and be used as a mechanism to choose the right scaffolding agent for effective knowledge transfer.

Multimodal information can significantly increase the perceptual capabilities of robotic agents, at the cost of a more complex sensory processing. This complexity can be reduced by employing machine learning techniques, provided that there is enough meaningful data to train on. This paper reports on creating novel datasets constructed by employing the iCub robot equipped with an additional depth sensor and color camera. We used the robot to acquire color and depth information for 210 objects in different acquisition scenarios. At the end, the results were large scale datasets that can be used for robot and computer vision applications: multisensory object representation, action recognition, rotation and distance invariant object recognition.

In this article we present implementation of Bacterial Foraging Optimization algorithm inspired search by multiple robots in an unknown area in order to find the region with highest chemical gas concentration as well as to build the chemical gas concentration map. The searching and map building tasks are accomplished by using mobile robots equipped with smart transducers for gas sensing called “KheNose”. Robots perform the search autonomously via bacterial chemotactic behavior. Moreover, simultaneously the robots send their sensor readings of the chemical concentration and their position data to a remote computer (a base station), where the data is combined, interpolated, and filtered to form an real-time map of the chemical gas concentration in the environment.