This study explores how humans and cognitive robots with different value systems and motivations understand each other's needs in free-form interactions. We developed a cognitive architecture that links sensing and perception to internal motivation and an intrinsic value system for determining actions. Inspired by young children's needs, this architecture includes three drives: learning, interaction, and recharging, each with varying dependence on the human partner. We aimed to assess how experimentally changing the importance of these drives within a fixed architecture affects interaction dynamics with human partners (acting as caregivers) and their understanding of the robot's needs. By adjusting the learning and interaction drives, we created two robot profiles: Playful, which prioritizes environmental exploration and playfulness to reduce boredom, and Social, which focuses on social interaction through touch and visual contact to increase comfort. Our findings show that changing the importance of these drives produces distinct behaviors and human perceptions. Robot behaviors matched their profiles, and participants adapted their responses accordingly. Participants identified and attributed distinct traits to each robot without knowing the specific profiles. Despite variability among human partners, the robots, especially the playful one, were generally well understood by most participants.

As hybrid interactions between humans and artificial agents become more prevalent, social skills are increasingly essential for autonomous systems. Beyond assisting in various tasks, robots are expected to understand human states and recognize that knowledge and perceptions of the world can differ, influencing overall behavior. This ability is closely tied to motivation, which plays a crucial role in driving autonomous agents’ actions. In this work, we explore the interaction between two intrinsically motivated cognitive autonomous robots with distinct profiles and preferences, utilizing Theory of Mind to infer each other’s motivations. We investigate the conditions under which they successfully collaborate to achieve mutual well-being and the circumstances that hinder cooperation. Our findings indicate that successful interactions emerge when at least one agent prioritizes helping others and when their profiles are aligned, leading to positive outcomes for both.

Human Activity Recognition (HAR) is an evolving field with applications in health monitoring, smart environments, exercise tracking, and human-computer interaction. HAR systems require models to adapt to dynamic and evolving data distributions, a challenge that traditional machine learning approaches often struggle to address, resulting in performance degradation over time. This paper introduces a framework for evaluating the application of meta-learning in continual learning scenarios within HAR. In the proposed framework, we employ Online Aware Meta-Learning (OML) and Model-Agnostic Meta-Learning (MAML-Rep) in a continual learning HAR scenario. These methods are evaluated for their ability to retain prior knowledge while efficiently adapting to new activities and data, addressing key challenges such as catastrophic forgetting and data imbalance. The framework also integrates robust preprocessing techniques, including data augmentation, to manage dataset variability. Experimental results highlight the superior adaptability and performance of OML across multiple HAR datasets, particularly in handling imbalanced data, validating the efficacy of meta-learning strategies efficacy for continual learning in HAR.

With the widespread adoption of deep learning, reinforcement learning (RL) has experienced a dramatic increase in popularity, scaling to previously intractable problems, such as playing complex games from pixel observations, sustaining conversations with humans, and controlling robotic agents. However, there is still a wide range of domains inaccessible to RL due to the high cost and danger of interacting with the environment. Offline RL is a paradigm that learns exclusively from static datasets of previously collected interactions, making it feasible to extract policies from large and diverse training datasets. Effective offline RL algorithms have a much wider range of applications than online RL, being particularly appealing for real-world applications, such as education, healthcare, and robotics. In this work, we contribute with a unifying taxonomy to classify offline RL methods. Furthermore, we provide a comprehensive review of the latest algorithmic breakthroughs in the field using a unified notation as well as a review of existing benchmarks' properties and shortcomings. Additionally, we provide a figure that summarizes the performance of each method and class of methods on different dataset properties, equipping researchers with the tools to decide which type of algorithm is best suited for the problem at hand and identify which classes of algorithms look the most promising. Finally, we provide our perspective on open problems and propose future research directions for this rapidly growing field.

Object affordance learning is the ability to process information about objects and how to use them. Embedding this knowledge in robots is an essential step for the development of intelligent and truly autonomous agents. This work proposes the development of a framework for learning affordances for robotic manipulation. The proposed approach was implemented as a reinforcement function in a network with a Soft Actor-Critic (SAC) algorithm and trained in simulation with a humanoid robot. Among different affordance complexities (touching and grabbing the object), the results show a rate of up to 95% correctness in the best scenario, with the agent properly performing all desired actions. Such results suggest that it is possible to define reward functions representing the object’s affordances for different objects.

The possibility of robots imitating reference movements performed by experts recently emerged in the Machine Learning context. Based on Deep Reinforcement Learning (DRL), this process focuses on observing a reference movement policy and its adaptation to a robot with a similar body scheme. In the humanoid robots domain, the massive availability of videos on the internet holds the potential to provide reference movements for virtually any task performed by humans. However, 3D pose estimation algorithms based on videos are currently subject to failure due to several practical situations (poor image framing, low video quality, joints occlusions and mismatch, and so on) and typically require applying a complex methodology. This paper presents SoccerKicks, a new dataset that provides 3D reference movements of humans performing dead ball kicks (penalty and foul) obtained from reference videos suitable for use in the robotics soccer domain. In this work we describe: i) the methodology adopted for the videos selection; ii) the algorithms chosen to perform the 2D and 3D pose estimation based on the videos; iii) the evaluation of the algorithms performance; iv) the annotation on these videos and the reference movements provided. Our dataset is publicly available at https://github.com/larocs/SoccerKicks.

This paper proposes a comparison between different neural network models, using multilayer perceptron (MLPs) and recurrent neural network (RNN) models, for predicting Parkinson's disease electromyography (EMG) signals, to anticipate resulting resting tremor patterns. The experimental results indicate that the proposed models can adapt to different frequencies and amplitudes of tremor, and provide reasonable predictions for both EMG envelopes and EMG raw signals. Therefore, one could use these models as input for a control strategy for functional electrical stimulation (FES) devices used on tremor suppression, by dynamically predicting and improving FES control parameters based on tremor forecast.

Visual saliency is an important component of attention. It helps animals survive and can also be used by computer vision applications to filter out irrelevant information from high volumes of data. In this work, we present a new convolutional neural network designed for detecting visual saliency, with architecture and data pre-processing methods specific for the task of visual saliency detection. Experiments carried out with the MIT300 benchmark presented state-of-the-art performance and a parameter reduction of 3/4 compared to similar models.

Usually embodied in real environments, robots are expected to perceive and act in their surrounding world in a human-like fashion [1], through perception, reasoning, planning and decision making processes. Although high computational power is available nowadays, the complexity of real scenes - continuous, partially unknown and usually unpredictable - can not be taken for granted. To actively act in this environment, robots are fully equipped with an enormous amount of sensors, overwhelming them with an immense amount of data that can not fully be processed and that constantly changes across time and space. To overcome this problem, the natural human filter - Attention - could be used as inspiration. This paper proposes an architecture that supports a transfer of domain from visual models to a robotics domain that uses sensors such as range scanners and sonars. Furthermore, it discusses the possibility of using multiple sensors to define multiple features. The experiments were performed in a simulated high fidelity environment and results have shown that the model proposed can account for detecting salient stimuli according to the modeled features.

Most of the techniques proposed for problems involving mobile robots are specified in terms of optimal control of Markov decision processes (MDPs). However, the state space dimension explosion makes such tabular MDP-based solutions unfeasible. As an alternative to this, a planning technique based on sparse sampling (SSA) of simulated instances of a MDP model has been suggested. Because the execution time of this algorithm is exponential on the level of an exploration tree and on the number of samplings to be generated, this paper proposes a technique where leaves null-values in the SSA algorithm are substitute by meaningful values, acquired from any of the following approaches: 1) a simple environment reward distribution; 2) a standard reinforcement learning algorithm, and 3) a compact representation on a coarse state discretization for generating initial estimates of the action values. The experiments carried out showed that such information-based variants of SSA lead quickly to better results than the original technique.

The information available to robots in real tasks is widely distributed both in time and space, requiring the agent to search for relevant information. In this paper, we implement a solution that uses qualitative and quantitative knowledge to turn robot tasks able to be treated by reinforcement learning (RL) algorithms. The steps of this procedure include: 1) to decompose the overall task into smaller ones, using abstraction and macro-operators, thus achieving a discrete action space; 2) to use observation functions of the environment - here called features - to achieve both time and state space discretisation; 3) to use quantitative knowledge to design controllers that are able to solve the subtasks; 4) to learn the coordination of these behaviours using RL, more specifically Q-learning. The approach was verified on an increasingly complex set of robot tasks using a Khepera robot simulator. Two approaches for space discretisation were used, one based on features and the other on states. The learned policies over these two models were compared to a predefined hand-crafted one. It was found that the learned policy over the state-based discretisation leads quickly to good results, although it can not be applied to complex tasks, where the state space representation becomes computationally unfeasible.