Tools and other objects offer agents a range of potential actions, commonly referred to as affordances. Each tool is typically designed with a primary purpose in mind-like a hammer’s function to drive nails. However, tools can also serve purposes beyond their original design. These alternative uses represent secondary affordances, extending the tool’s utility beyond its primary intended function. While prior robotics research on affordance perception and learning has primarily focused on primary affordances, our work addresses the less-explored area of learning secondary tool affordances from human partners. Using the iCub robot equipped with three cameras, we observed humans performing actions on twenty objects using four different tools in ways that deviate from their primary purposes. For example, the iCub observed humans using rulers not for measuring but to push, pull, and move objects. In this setting, we constructed a dataset by taking pictures of objects before and after each action is executed. To model secondary affordance learning, we trained three neural networks (ResNet-18, ResNet-50, and ResNet-101) on three prediction tasks using these raw images as input: (1) identifying which tool was used to move an object, (2) predicting the tool with additional action category information, and (3) jointly predicting both the tool and action performed. Our results demonstrate that deep learning architectures enable the iCub robot to successfully predict secondary tool affordances, thereby paving the road for human-robot collaborative object manipulation involving complex affordances. Code and data from this study are available at https://github.com/BosongDing/second_affordance

Despite recent developments that allowed neural networks to achieve impressive performance on a variety of applications, these models are intrinsically affected by the problem of overgeneralization, due to their partitioning of the full input space into the fixed set of target classes used during training. Thus it is possible for novel inputs belonging to categories unknown during training or even completely unrecognizable to humans to fool the system into classifying them as one of the known classes, even with a high degree of confidence. This problem can lead to security problems in critical applications, and is closely linked to open set recognition and 1-class recognition. This paper presents a novel way to compute a confidence score using the reconstruction error of denoising autoencoders and shows how it can correctly identify the regions of the input space close to the training distribution. The proposed solution is tested on benchmarks of 'fooling', open set recognition and 1-class recognition constructed from the MNIST and Fashion-MNIST datasets.