While appearance-based gaze estimation has achieved significant improvements in accuracy and domain adaptation, the fairness of these systems across different demographic groups remains largely unexplored. To date, there is no comprehensive benchmark quantifying algorithmic bias in gaze estimation. This paper presents the first extensive evaluation of fairness in appearance-based gaze estimation, focusing on ethnicity and gender attributes. We establish a fairness baseline by analyzing state-of-the-art models using standard fairness metrics, revealing significant performance disparities. Furthermore, we evaluate the effectiveness of existing bias mitigation strategies when applied to the gaze domain and show that their fairness contributions are limited. We summarize key insights and open issues. Overall, our work calls for research into developing robust, equitable gaze estimators. To support future research and reproducibility, we publicly release our annotations, code, and trained models at: github.com/ akgulburak/gaze-estimation-fairness.

While zero-shot appearance-based 3D gaze estimation offers significant cost-efficiency by directly mapping RGB images to gaze vectors, its reliability in Human-Robot Interaction (HRI) settings remains uncertain. Existing benchmarks frequently overlook fundamental HRI conditions, such as dynamic camera viewpoints and moving targets in video. Furthermore, current cross-dataset evaluations often suffer from a complexity gap, where methods trained on diverse datasets are tested on significantly smaller and less varied sets, failing to assess true robustness. To bridge these gaps, we introduce Gaze4HRI, a large-scale dataset (50+ subjects, 3,000+ videos, 600,000+ frames) designed to evaluate state-of-the-art performance against critical HRI variables: illumination, headgaze conflict, as well as the motion of camera and gaze target in video. Our benchmark reveals that all evaluated methods fail in at least one condition, identifying steeply-downward gaze as a universal failure point. Notably, PureGaze trained on the ETH-X-Gaze dataset uniquely maintains resilience across all other conditions. These results challenge the recent focus in the literature on complex spatial-temporal modeling and Transformer-based architectures. Instead, our findings suggest that extensive data diversity, as exemplified by the ETH-X-Gaze dataset, serves as the primary driver of zero-shot robustness in unconstrained environments, while resilienceenhancing frameworks, such as PureGaze's self-adversarial loss for gaze feature purification, provide a substantial further improvement. Ultimately, this study establishes a rigorous benchmark that provides practical guidelines for practitioners as well as reshaping future research. The dataset and codes are available at https://gazeforhri.github.io.

Achieving fluent flocking, similar to those observed in birds and fish, on robotic swarms in a desired direction while avoiding obstacles using onboard sensing and computation remains a challenge. In a previous study (Önür et al, Proc. of ANTS'2022), we proposed a predictive flocking model as a computationally efficient method to generate smoother and more robust motion of the swarm. In this study, we extend this model to achieve safe flocking in cluttered environments in the presence of faulty robots that get immobilized during flocking. Systematical evaluation of the model in simulation with different swarm sizes and different faulty robot ratios has shown that safe flocking can be achieved even when 40% of the robots malfunction during flocking. Finally, we validate the model on a swarm of five micro quadcopters using only onboard range and bearing sensors and computation in a distributed manner without any communication11Videos of the experiments are available here: https://www.youtube.com/playlist?list=PLY04vZs6xGr8U5Y8KWMD056ZRjaUVMh63..

In this paper, we propose a Model Predictive Control (MPC) based distributed formation control method for a multi-robot system (MRS) that would move them among dynamic obstacles to a desired goal position. Specifically, after formulating the formation control, as a distributed version of MPC, we propose and evaluate three information-sharing schemes within the MRS; namely sharing (i) positions, (ii) complete predicted trajectories, and (iii) exponentially-sampled predicted trajectories. Using a simplified kinematic model for robots, we conducted systematic simulation experiments in (a) scenarios, where the robots are instructed to switch places, as one of the most challenging forms of formation changes, and in (b) scenarios where robots are instructed to reach a goal, within environments containing dynamic obstacles. In a set of systematic experiments conducted in simulation and with mini quadcopters, we have shown that sharing of exponentially-sampled trajectories (as opposed to positions, or complete trajectories) among the robots provides near-optimal paths while decreasing the required computation cost and communication bandwidth. Surprisingly, in the presence of noise, sharing exponentially-sampled trajectories among the robots decreased the variance in the final paths. The proposed method is demonstrated on a group of Crazyflie quadcopters.

The rise of simulation environments has enabled learning-based approaches for assembly planning, which is otherwise a labor-intensive and daunting task. Assembling furniture is especially interesting since furniture are intricate and pose challenges for learning-based approaches. Surprisingly, humans can solve furniture assembly mostly given a 2D snapshot of the assembled product. Although recent years have witnessed promising learning-based approaches for furniture assembly, they assume the availability of correct connection labels for each assembly step, which are expensive to obtain in practice. In this paper, we alleviate this assumption and aim to solve furniture assembly with as little human expertise and supervision as possible. To be specific, we assume the availability of the assembled point cloud, and comparing the point cloud of the current assembly and the point cloud of the target product, obtain a novel reward signal based on two measures: Incorrectness and incompleteness. We show that our novel reward signal can train a deep network to successfully assemble different types of furniture. Code and networks available here: https://github.com/METU-KALFA/AssembleRL.

In this paper, we investigate how collaborative robots, or cobots, typically composed of a robotic arm and a gripper carrying out manipulation tasks alongside human coworkers, can be enhanced with HRI capabilities by applying ideas and principles from character animation. To this end, we modified the appearance and behaviors of a cobot, with minimal impact on its functionality and performance, and studied the extent to which these modifications improved its communication with and perceptions by human collaborators. Specifically, we aimed to improve the Appeal of the robot by manipulating its physical appearance, posture, and gaze, creating an animal-like character with a head-on-neck morphology; to utilize Arcs by generating smooth trajectories for the robot arm; and to increase the lifelikeness of the robot through Secondary Action by adding breathing motions to the robot. In two user studies, we investigated the effects of these cues on collaborator perceptions of the robot. Findings from our first study showed breathing to have a positive effect on most measures of robot perception and reveal nuanced interactions among the other factors. Data from our second study showed that, using gaze cues alone, a robot arm can improve metrics such as likeability and perceived sociability.

The studies on mirror neurons observed in monkeys indicate that recognition of other's actions activates neural circuits that are also responsible for generating the very same actions in the animal. The mirror neuron hypothesis argues that such an overlap between action generation and recognition can provide a shared worldview among individuals and be a key pillar for communication. Inspired by these findings, this paper extends a learning by demonstration method for online recognition of observed actions. The proposed method is shown to recognize and generate different reaching actions demonstrated by a human on a humanoid robot platform. Experiments show that the proposed method is robust to both occlusions during the observed actions as well as variances in the speed of the observed actions. The results are successfully demonstrated in an interactive game with the iCub humanoid robot platform.

Behavioural and neuroscience studies have shown that observing objects activates affordances, evoking motor responses. The aim of the present study is twofold. First, we intend to investigate whether children are sensitive to the distinction between neutral/graspable (affordances) and dangerous objects. Second, we aim to verify whether children's responses are modulated also by the agent who is interacting with the objects (human hand contrasted with robot hand, and male hand contrasted to female hand). We conducted an experiment on school-age children using a priming paradigm: a prime given by a hand or a control object was followed by graspable or dangerous objects. Children were required to categorize them into artefacts or natural objects by pressing two keys on a keyboard. Our results clearly showed that children are able to distinguish between neutral and dangerous objects: the latter produced an interference effect. In addition, we demonstrated that children are sensitive to the difference between actions performed by biological and non-biological agents: responses were faster when the prime was a grasping hand of a human compared to control stimuli. Results were interpreted in terms of gradient of vulnerability (female hand induced the most inhibition, while robot hand induced the least one) and of motor resonance (resonance is higher when the similarity between the hand prime and the participant's hand is higher).

Human infants practice their initial, seemingly random arm movements for transforming them into voluntary reaching and grasping actions. With the developing perceptual abilities, infants further explore their environment using the behavior repertoire they have developed, and learn causality relations in the form of affordances, which they use for goal satisfaction and motor planning. This study proposes and implements a developmental progression on a robotic system mimicking the aforementioned infant development stages: An anthropomorphic robot hand with one basic action of swing-hand and the palmar reflex (i.e. the enclosure of the fingers upon contact) at its disposal, executes swing-hand action targeted to a salient object with different hand speeds. During the executions, it monitors the changes in its sensors, automatically forming behavior primitives such as `grasp', `hit', `carry-object' and `drop' by segmenting and differentiating the initial swing-hand action. The study then focuses on one of these behaviors, namely grasping, and shows how further practice allows the robot to learn affordances of more complex objects, which can be further used to make plans to achieve desired goals using the discovered behavior repertoire.

In this paper, we propose a method that enables a robot to learn not only the existence of affordances provided by objects, but also the behavioral parameters required to actualize them, and the prediction of effects generated on the objects in an unsupervised way. In a previous study, it was shown that through self-interaction and self-observation, analogous to an infant, an anthropomorphic robot can learn object affordances in a completely unsupervised way, and use this knowledge to make plans in its perceptual space. This paper extends the affordances model proposed in that study by using parametric behaviors and including the behavior parameters into affordance learning and goal-oriented plan generation. Furthermore, for handling complex behaviors and complex objects (such as execution of precision grasp on a mug), the perceptual processing is improved by using a combination of local and global features. Finally, a hierarchical clustering algorithm is used to discover the affordances in non-homogenous feature space. In short, object affordances for object manipulation are discovered together with behavior parameters based on the monitored effects.

In this paper, we use the notion of affordances, proposed in cognitive science, as a framework to propose a developmental method that would enable a robot to ground symbolic planning mechanisms in the continuous sensory-motor experiences of a robot. We propose a method that allows a robot to learn the symbolic relations that pertain to its interactions with the world and show that they can be used in planning. Specifically, the robot interacts with the objects in its environment using a pre-coded repertoire of behaviors and records its interactions in a triple that consist of the initial percept of the object, the behavior applied and its effect, defined as the difference between the initial and the final percept. The method allows the robot to learn object affordance relations which can be used to predict the change in the percept of the object when a certain behavior is applied. These relations can then be used to develop plans using forward chaining. The method is implemented and evaluated on a mobile robot system with limited object manipulation capabilities. We have shown that the robot is able to learn the physical affordances of objects from range images and use them to build symbols and relations that can be used in making multi-step predictions about the affordances of objects and achieve complex goals.

In this paper, we demonstrate that simple interactions with objects in the environment leads to a manifestation of the perceptual properties of objects. This is achieved by deriving a condensed representation of the effects of actions (called effect prototypes in the paper), and investigating the relevance between perceptual features extracted from the objects and the actions that can be applied to them. With this at hand, we show that the agent can categorize (i.e., partition) its raw sensory perceptual feature vector, extracted from the environment, which is an important step for development of concepts and language. Moreover, after learning how to predict the effect prototypes of objects, the agent can categorize objects based on the predicted effects of actions that can be applied on them.

In this paper, we propose a model for the self-organized aggregation of a swarm of mobile robots. Specifically, we use a simple probabilistic finite state automata (PFSA) based aggregation behavior and analyze its performance using both a point-mass and a physics-based simulator and compare the results against the predictions of the model. The results show that the probabilistic model predictions match simulation results and PFSA-based aggregation behaviors with fixed probabilities are unable to generate scalable aggregations in low robot densities. Moreover, we show that the use of a leave probability that is inversely proportional to the square of the neighbor count (as an estimate of aggregate size) does not improve the scalability of the behavior.

"Developmental robotics" proposes that, instead of trying to build a robot that shows intelligence once and for all, what one must do is to build robots that can develop. These robots should be equipped with behaviors that are simple but enough to bootstrap the system. Then, as the robot interacts with its environment, it should display increasingly complex behaviors. In this paper, we propose such a development scheme for a mobile robot. J.J. Gibson's concept of "affordances" and a formalization of this concept provides the basis of this development scheme. We show that an autonomous robot can start with pre-coded primitive behaviors, and as it executes its behaviors randomly in an environment, it can learn the affordance relations between the environment and its behaviors. We then present two ways of using these learned structures, in achieving more complex, intentional behaviors. In the first case, the sequencing of these primitive behaviors are such that new more complex behaviors emerge. In the second case, the robot makes a "blending" of its pre-coded primitive behaviors to create new behaviors.

We are interested in how the concept of affordances can affect our view to autonomous robot control, and how the results obtained from autonomous robotics can be reflected back upon the discussion and studies on the concept of affordances. In this paper, we studied how a mobile robot, equipped with a 3D laser scanner, can learn to perceive the traversability affordance and use it to wander in a room tilled with spheres, cylinders and boxes. The results showed that after learning, the robot can wander around avoiding contact with non-traversable objects (i.e. boxes, upright cylinders, or lying cylinders in certain orientation), but moving over traversable objects (such as spheres, and lying cylinders in a rollable orientation with respect to the robot) rolling them out of its way. We have shown that for each action approximately 1% of the perceptual features were relevant to determine whether it is afforded or not and that these relevant features are positioned in certain regions of the range image. The experiments are conducted both using a physics-based simulator and on a real robot.

In this paper, we studied how a mobile robot equipped with a 3D laser scanner can start from primitive behaviors and learn to use them to achieve goal-directed behaviors. For this purpose, we propose a learning scheme that is based on the concept of "affordances", where the robot first learns about the different kind of effects it can create in the environment and then links these effects with the perception of the initial environment and the executed primitive behavior. It uses these learned relations to create certain effects in the environment and achieve more complex behaviors.

When one attempts to use artificial evolution to develop behaviors for a swarm robotic system, he is faced with decisions to be made regarding the parameters of the evolution. In this paper, aggregation behavior is chosen as a case, where performance and scalability of aggregation behaviors of perceptron controllers that are evolved for a simulated swarm robotic system are systematically studied with different parameter settings. Four experiments are conducted varying some of the parameters, and rules of thumb are derived, which can be of guidance to the use of evolutionary methods to generate other swarm robotic behaviors.

In this study, a systematic analysis of probabilistic aggregation strategies in swarm robotic systems is presented. A generic aggregation behavior is proposed as a combination of four basic behaviors: obstacle avoidance, approach, repel, and wait. The latter three basic behaviors are combined using a three-state finite state machine with two probabilistic transitions among them. Two different metrics were used to compare performance of strategies. Through systematic experiments, how the aggregation performance, as measured by these two metrics, change 1) with transition probabilities, 2) with number of simulation steps, and 3) with arena size, is studied.