Socially Assistive Robotics (SAR) has shown promise in supporting emotion regulation for neurodivergent children. Recently, there has been increasing interest in leveraging advanced technologies to assist parents in co-regulating emotions with their children. However, limited research has explored the integration of large language models (LLMs) with SAR to facilitate emotion co-regulation between parents and children with neurodevelopmental disorders. To address this gap, we developed an LLM-powered social robot by deploying a speech communication module on the MiRo-E robotic platform. This supervised autonomous system integrates LLM prompts and robotic behaviors to deliver tailored interventions for both parents and neurodivergent children. Pilot tests were conducted with two parent-child dyads, followed by a qualitative analysis. The findings reveal MiRo-E’s positive impacts on interaction dynamics and its potential to facilitate emotion regulation, along with identified design and technical challenges. Based on these insights, we provide design implications to advance the future development of LLM-powered SAR for mental health applications.

Parent involvement in children’s homework has emerged as an important component of their education. However, this involvement can generate tension between parents and children, particularly when coupled with a heavy workload, potentially exacerbating stress levels in children. Some children desire to express and share their stress with their parents. However, limited emotional regulation abilities and a lack of stress awareness often hinder effective stress management and communication by children. To address this, we designed a stress biofeedback system named Stress Diffuser, comprising a wearable biosensor with interactive embodied devices, that display the child’s stress during homework sessions. The system intends to manage children’s stress by enhancing awareness among both the child and the parent, and influencing their educational strategies and the way of interaction. We conducted experiments with eight families, including primary school children aged eight to eleven and their parents. The results of our quantitative and qualitative analysis indicated that all participants enhanced their stress awareness while using Stress Diffuser during homework. Most parents adjusted their supervisory strategies and interaction styles in response to their children’s stress data. The study sheds light on children’s needs for a designed agent to communicate their stress and regulate their emotions. Furthermore, the study explores the use of interactive biofeedback technology in homework settings with parent involvement. It reveals a favorable influence on facilitating stress management in children and on enhancing the quality of supervision and interaction between children and parents.

Research has shown the potential of social robots to support learning in science, technology, and language. We contribute to this field by exploring how robots can support music learning. We report on a within-subjects experiment where 50 young learners practiced the piano in the presence of a robot assuming a non-evaluative and a self-assessment enhancing role implemented in a Wizard-of-Oz fashion. We examined whether the robot can make piano practice more fun, and whether initiating self-assessment to support self-regulated learning is a useful strategy for the robot. We collected quantitative self-report data to assess fun, learning, interest, engagement, and effort. We found a direct positive effect of fun on learning in the context of musical instrument practice. Path modeling showed a positive influence of having fun on learners' attitudes, interests, and learning outcomes in music education, particularly with the self- assessment robot role exhibiting superiority.

The increasing demand for medical services in hospitals has sparked interest in exploring alternative methods to support children with functional articulation disorders. Online speech games have emerged as a promising avenue to motivate children to engage in speech therapy. This study investigates the impact of gamification strategies on children's motivation within the context of speech rehabilitation games. Four distinct game prototypes were developed, with a specific emphasis on stimulating children's motivation to speak. Two sets of experiments involving 48 participants were conducted to assess the influence of (1) time-limitation and (2) interactive imitation objects on children's motivation. The results revealed that time limitation significantly increased motivation, while the effect of imitation objects on motivation was not statistically significant. These findings offer valuable insights into designing effective speech games for children. By leveraging gamification strategies in online speech games, we can address the motivation challenges faced by children with functional articulation disorders and potentially enhance the efficiency of speech therapy interventions.

Social robots exist in various forms but are still not spreading widely to the societal contexts they are envisioned for, such as educational settings. There appears to be a gap between research and practice: A lot of research already demonstrates promising results for using social robots in various domains, but this promise has yet to be realized in “the real world”. The aim of this paper is to form a systematic understanding of the potential and challenges of social robots in the domain of education and learning, which is an intensively researched application domain for social robots. We conducted a meta-review of recent literature reviews (published in 2018-2022), using the PRISMA method. We analyzed 12 review papers that met the defined inclusion criteria by extracting the potential benefits and challenges presented in these reviews. We identify six benefits, five challenges, and six recommendations for future research on social robots in education and learning. These findings emphasize the potential and developments needed to realize the potential of social robots in educational contexts.

Stress-related health problems in children have increased in recent years, resulting in significant negative physical and mental impacts on children’s daily lives. This systematic review explores the potential of embodied technologies, such as robots, smart wearables, and the Internet of Things (IoT), as tools for managing stress in children. The goal of this systematic review is to identify the design opportunities of embodied technologies in stress management, by looking for answers in terms of different technologies, users, issues, and challenges addressed in the 91 selected papers. Through the frequency and thematic analysis, we identified six main challenges and eight design opportunities for embodied technologies. Where there are gaps and opportunities in research, we propose to focus on connectivity and active sensing through connected objects, by exploring the potential of the Internet of Robotic Things (IoRT) as an M-health solution for providing real-time and personalized stress detection and interventions for children in various daily life settings.

There is growing interest in psychological interventions using socially assistive robots to mitigate distress and pain in the pediatric population. This work seeks to address the deficit in understanding of what features and functionality young children and their parents desire to help with pain management by using co-design, a common approach to exploring participants' imaginations and gathering design requirements. To close this gap, we carried out a co-design workshop involving seven families (with children aged between 4–6 and their parents) to understand their expectations and design preferences for a robot designed for pain management in children. Data were collected from surveys, video and audio recordings, interviews, and field notes. We present the robot prototypes constructed during the workshops and derive several preferences of the children (e.g, zoomorphic shape, distractors and emotional expressions as behaviors). Additionally, we report methodological insights regarding the involvement of young children and their parents in the co-design process. Based on the findings of this co-design study, we discuss personalization as a possible design concept for future child-robot interaction development.

In social robotics, where people and robots interact in a social context, robot personality design is critical. Through voice, words, gestures, and nonverbal clues, social robots with expressive behaviors can display human-like actions, and the robot’s personality will ensure consistency. This research aims to create robot personalities expressed only by nonverbal cues. Differently from existing studies that test expressive behaviors with non-specialized participants, we look at how and why human movement analysts perceive distinct personalities in robots (introvert vs. extrovert) based on the robot’s movement and other dynamic features, such as joint position, head, and torso position, voice pitch, speed, and so on. We report the findings of a thematic analysis of the data obtained during a focus group with movement analysis experts who watched Pepper robot behaviors designed to be extrovert and introvert. Our findings lead to new guidelines for designing different robot movement features, including body symmetry, personality trait consistency, and social cue congruence during an interaction, all emphasized by the movement analyzers. Finally, we summarize the design principles for extrovert and introvert robot behaviors based on the combined findings of the focus group data analysis and literature review.

In music education, staying motivated has always been a crucial goal, especially for children. Earlier research has demonstrated how social robots could help improve motivation and increase the performance of children practicing music pieces. As important stakeholders, the attitudes and expec- tations of parents and music teachers are important factors that affect children’s and their own acceptance of applying social robots in music education. In this study, a survey was created and used to assess parents’ and music teachers’ attitudes, expectations, and intentions related to using social robots in music education. The survey results suggested that parents and music teachers expect robots to act as assistive and motivating agents, give positive feedback, and schedule and structure practice sessions. Moreover, parents expect a social robot to evaluate student performance, while music teachers focus more on positive encouragement. The biggest concerns of these stakeholders are whether the robots can convey the emotional aspects of music and demonstrate how to play the instrument. The intention of using robots is positively correlated with positive attitudes towards robots in general and the positive attitude towards robots specifically in the context of music education. The current study contributes insights into parents’ and music teachers’ attitudes and expectations as determinants of their acceptance of of social robots in music education, and provides a method for measuring these attitudes and expectations.

Background: Where self-report is unfeasible or observations are difficult, physiological estimates of pain are needed. Methods: Pain-data from 30 healthy adults were gathered to create a database of physiological pain responses. A model was then developed, to analyze pain-data and visualize the AI-estimated level of pain on a mobile app. Results: The initial low precision and F1-score of the pain classification algorithm were resolved by interpolating a percentage of similar data. Discussion: This system presents a novel approach to assess pain in noncommunicative people with the use of a sensor sock, AI predictor and mobile app. Performance analysis and the limitations of the AI algorithm are discussed.

Recent emotion recognition models, most of them being based on strongly supervised deep learning solutions, are rather successful in recognizing instantaneous emotion expressions. However, when applied to continuous interactions, these models show a weaker adaptation to a person-specific and long-term emotion appraisal. In this article, we present an unsupervised neural framework that improves emotion recognition by learning how to describe continuous affective behavior of individual persons. Our framework is composed of three self-organizing mechanisms: (1) a recurrent growing layer to cluster general emotion expressions, (2) a set of associative layers, acting asaffective memoriesto model specific emotional behavior of individual persons, (3) and an online learning layer which provides contextual modeling of continuous emotion expressions. We propose different learning strategies to integrate all three mechanisms and to improve the performance on arousal and valence recognition of the OMG-Emotion dataset. We evaluate our model with a series of experiments ranging from ablation studies assessing the different contributions of each neural component to an objective comparison with state-of-the-art solutions. The results from the evaluations show a good performance on emotion recognition of continuous emotions on monologue videos. Furthermore, we discuss how the model self-regulates the interplay between generalized and personalized emotion perception and how this influences the model’s reliability when recognizing unseen emotion expressions.

One of the most effective ways to improve quality of life in dementia is by exposing people to meaningful activities. The study of engagement is crucial to identify which activities are significant for persons with dementia and customize them. Previous work has mainly focused on developing assessment tools and the only available model of engagement for people with dementia focused on factors influencing engagement or influenced by engagement. This article focuses on the internal functioning of engagement and presents the development and testing of a model specifying the components of engagement, their measures, and the relationships they entertain. We collected behavioral and physiological data while participants with dementia (N = 14) were involved in six sessions of play, three of game-based cognitive stimulation and three of robot-based free play. We tested the concurrent validity of the measures employed to gauge engagement and ran factorial analysis and Structural Equation Modeling to determine whether the components of engagement and their relationships were those hypothesized. The model we constructed, which we call the ENGAGE-DEM, achieved excellent goodness of fit and can be considered a scaffold to the development of affective computing frameworks for measuring engagement online and offline, especially in HCI and HRI.

The papers in this special section focus on the interaction with artificial intelligence (AI) systems using human-centered applications. AI methods are being applied to numerous areas, including medicine, security, transportation, industry, smart homes and cities, business, social sciences, and psychology. AI is currently a part of our daily lives. People interact continuously with AI: it is inside houses, computers, mobile phones, and applications. AI can make predictions and give suggestions for movies, songs, or future purchases based on our previous choices. It affects the society and economy. People are fascinated by AI in the ways it improves and facilitates human life (improving health care and discharging workers from heavy or dangerous jobs). People are also concerned with AI’s implementation risks, such as ethical, security, and privacy issues. There are also concerns that AI machines may replace humans in various activities. AI researchers and practitioners have been facing these issues and further research is needed to design technical and regulatory applicable solutions. This special issue (SI) investigates a broad range of issues deriving from human interaction with AI. We encouraged interdisciplinary and multidisciplinary contributions toward understanding how AI could improve human life in various fields.

Almost all older people with dementia have progressive communication difficulties, which lead to increased social isolation and negative emotions. Thus, providing communication assistance for them is essential. This paper explores the feasibility of using social robots to assist older people with dementia in their face-to-face communication with others. We designed the behavior of a humanoid Pepper robot and made a Wizard of Oz prototype that the robot can serve as a personal memory assistant. The robot stores personal information for older people and assists in their communication through voice and screen display. In a video-based study with 88 participants, we investigated the effects of this assistive robot from a third-person observer perspective. Data were collected and analyzed using both three-way MANCOVAs for quantitative analysis and conventional content analysis for qualitative data. The results revealed that, by providing memory support, the robot significantly improved the observer's perceptions of an older person with dementia, including her perceived communication ability and performance, and personal image. Meanwhile, the communication is perceived to be significantly more effective when the robot assisted an older person. The willingness of others to communicate with more senior people also increased accordingly. Based on these findings, we present guidelines that may inform the design and development of communication assistant robots for older people with dementia.

Introducing robots in healthcare facilities and homes may reduce the workload of healthcare personnel while providing the users with better and more available services. It may also contribute to interactions that are engaging and safe against transmitting contagious diseases for senior adults. A major challenge in this regard is to design and adapt the robot’s behavior based on the requirements and preferences of the different users. In this paper, we report a conducted use study on how people perceive different kinds of robot encounters. We had two groups of target users: one with senior residents at a care center and another with young students at a university, which would be representative for the visitors and care volunteers in the facility. Several common scenarios have been created to evaluate the perception of the robot’s behavior by the participants. Two sets of questionnaires were used to collect feedback on the behavior and the general perception of the users about the robot´s different styles of behavior. An exploratory analysis of the effect of age shows that the age of the targeted user group should be considered as one of the main criteria when designing the social parameters of a care robot, as seniors preferred slower speed and closer distance to the robot. The results can contribute to improving a future robot’s control to better suit users from different generations.

The application of social robots has recently been explored in various types of educational settings including music learning. Earlier research presented evidence that simply the presence of a robot can influence a person's task performance, confirming social facilitation theory and findings in human-robot interaction. Confirming the evaluation apprehension theory, earlier studies showed that next to a person's presence, also that person's social role could influence a user's performance: the presence of a (non-) evaluative other can influence the user's motivation and performance differently. To be able to investigate that, researchers need the roles for the robot which is missing now. In the current research, we describe the design of two social roles (i.e., evaluative role and non-evaluative role) of a robot that can have different appearances. For this, we used the SocibotMini: A robot with a projected face, allowing diversity and great flexibility of human-like social cue presentation. An empirical study at a real practice room including 20 participants confirmed that users (i.e., children) evaluated the robot roles as intended. Thereby, the current research provided the robot roles allowing to study whether the presence of social robots in certain social roles can stimulate practicing behavior and suggestions of how such roles can be designed and improved. Future studies can investigate how the presence of a social robot in a certain social role can stimulate children to practice.

As developments in artificial intelligence and robotics progress, more tasks arise in which humans and robots need to collaborate. With changing levels of complementarity in their capabilities, leadership roles will constantly shift. The research presented explores how people adapt their behavior to initiate or accommodate continuous leadership shifts in human-robot collaboration and how this influences trust and understanding. We conducted an experiment in which participants were confronted with seemingly conflicting interests between robot and human in a collaborative task. This was embedded in a physical navigation task with a robot on a leash, inspired by the interaction between guide dogs and blind people. Explicit and implicit feedback factors from the task and the robot partner proved to trigger humans to reconsider when to lead and when to follow, while the outcome of this differed across participants. Overall the participants evaluated the collaboration more positively over time, while participants who took the lead more often valued the collaboration more negatively than other participants.

The wellbeing of people with dementia in long-term care facilities is hindered, as they spend most of their time alone with little engagement in meaningful activities and an absence of pleasant sensory stimulation. We designed an interactive system called LiveNature that adopts a novel combined approach involving an ambient display unit and an interactive robotic sheep, to offer long-term access and to engage people with dementia in long-term care facilities in rewarding experiences. LiveNature aims to provide holistic multi-sensory engagement to provoke positive emotions, increase social bonding, and restore attentiveness and communication. The design was implemented within a Dutch nursing home. An evaluation of the user experience and the effectiveness of the design was conducted in a real-life setting with nine participants, five family members, two caregivers and four volunteers, using observational rating scales and semi-structured interviews. The results of the rating scales and the findings from qualitative data showed evidence of enhanced positive engagement.

To advance the research area of social robotics, it is important to understand the effect of different social cues on the perceived social agency to a robot. This paper evaluates three sets of verbal and nonverbal social cues (emotional intonation voice, facial expression and head movement) demonstrated by a social agent delivering several messages. A convenience sample of 18 participants interacted with SociBot, a robot that can demonstrate such cues, experienced in sequence seven sets of combinations of social cues. After each interaction, participants rated the robot's social agency (assessing its resemblance to a real person, and the extent to which they judged it to be like a living creature). As expected, adding social cues led to higher social agency judgments; especially facial expression was connected to higher social agency judgments.

Previous research has shown that robot-mediated therapy may be effective in improving different mental or physical conditions, but this effectiveness strongly depends on how well the therapy can be translated to robot training. The goal of this study is to assist the end-users such as occupational and rehabilitation therapists to create without help of technical professional therapy-specific and sensory-enabled scenarios for the robotic assistant for use in an unstructured environment. The Cognitive Dimension of Notations framework was applied to assess the usability of the programming interface and the Cyclomatic complexity method was used to evaluate the complexity of the created robot scenarios. Eleven therapists with a mean age of 39 years working in the care for persons with visual-and-intellectual disabilities participated. The results show good usability of the interface, as measured via the CDN framework and the cyclomatic complexity analysis showed an increased complexity of the created by the occupational and rehabilitation therapist's scenarios. The participants did not request for very specifically defined behaviors for the robot, and therefore descriptions in natural text can be successfully used for robot programming.

Applications of social robotics in different domains such as education, healthcare, or as companions to people living alone, often entail that robots will act as persuasive agents. However, persuasive attempts can give rise to psychological reactance where people have negative thoughts and emotions that limit adherence to the persuader. To understand the phenomenon of reactance to robotic persuaders, we investigate the effect of social cues of an artificial agent on psychological reactance and compliance. Participants in a laboratory experiment played a decision-making game in which persuasive attempts were delivered in one of three forms: as a persuasive-text, spoken by a social robot (the SocibotTM) displaying minimal social cues, or by the same robot displaying enhanced social cues. Our results suggest that a social robot with minimal social cues invokes the lowest reactance. Remarkably, exploratory analyses indicate cross-gender effects (between robot and user) upon invoking lower psychological reactance and female participants have higher compliance than male participants.

This study examines patterns of coordinated gaze between a child and a robot (NAO) during a card matching game, `Memory'. Dyadic gaze behavior like mutual gaze, gaze following and joint attention are indications both of child's engagement with the robot and of the quality of child-robot interaction. Eighteen children interacted with a robot tutor in two settings. In the first setting, the robot tutor gave clues to assist children in finding the matching cards, and in the other setting, the robot tutor only looked at the participants during the play. We investigated the coordination between child and robots' gaze behaviors. We found that more occurrences of mutual gaze and gaze following made the children aware of the gaze hints given by the robot and improved the efficacy of the robot tutor as a helping agent. This study, therefore, provides guidelines for gaze behaviors design to enrich child-robot interaction in a tutoring context.

In search for better technological solutions for education, we adapted a principle from economic game theory, namely that giving a help will promote collaboration and eventually long-term relations between a robot and a child. This principle has been shown to be effective in games between humans and between humans and computer agents. We compared the social and cognitive engagement of children when playing checkers game combined with a social strategy against a robot or against a computer. We found that by combining the social and game strategy the children (average age of 8.3 years) had more empathy and social engagement with the robot since the children did not want to necessarily win against it. This finding is promising for using social strategies for the creation of long-term relations between robots and children and making educational tasks more engaging. An additional outcome of the study was the significant difference in the perception of the children about the difficulty of the game – the game with the robot was seen as more challenging and the robot – as a smarter opponent. This finding might be due to the higher perceived or expected intelligence from the robot, or because of the higher complexity of seeing patterns in three-dimensional world.

Previous research has suggested that robot-mediated therapy is effective in the treatment of children with Autism Spectrum Disorder (ASD), but not all robots seem equally appropriate for this purpose. We investigated in an exploratory study whether a robot’s intonation (monotonous vs. normal) and bodily appearance (mechanical vs. humanized) influence the treatment outcomes of Pivotal Response Treatment (PRT) sessions for children with ASD. The children (age range 4–8 years) played puzzle games with a robot which required communication with the robot. The treatment outcomes were measured in terms of both task performance and affective states. We have found that intonation and bodily appearance have an effect on children’s affective states but not on task performance. Specifically, humanized bodily appearance leads to more positive affective states in general and a higher degree of interest in the interaction than mechanical bodily appearance. Congruence between bodily appearance and intonation triggers a higher degree of happiness in children with ASD than incongruence between these two factors.

To understand the behavior of moving entities in a given environment, one should be capable of predicting their motion, that is, to model their dynamics. In a setting where different behaviors can arise, one can assume that each of them corresponds to different motivational states of observed entities. Here, those motivations are understood as goal positions or spots where entities seek to arrive. To build prediction models based on that idea, we present an unsupervised method to estimate motivational spots actively. Additionally, we use the output of such process to refine an adaptive system modeling the dynamics of inferred hidden causes of observed data. The whole method uses deep variational methods, and particularly, the network estimating motivations is trained through dynamic programming. Results show that modeling the dynamics of entities can be better achieved by integrating information about motivational spots. Notably, a network modeling the dynamics converges faster through the incorporation of information about motivations.

This paper presents a method for observational learning in autonomous agents. A formalism based on deep learning implementations of variational methods and Bayesian filtering theory is presented. It is explained how the proposed method is capable of modeling the environment to mimic behaviors in an observed interaction by building internal representations and discovering temporal and causal relations. The method is evaluated in a typical surveillance scenario, i.e., perimeter monitoring. It is shown that the vehicle learns how to drive itself by simultaneously observing its surroundings and the actions taken by a human driver for a given task. That is achieved by embedding knowledge regarding perception-action couplings in dynamic representational states used to produce action flows. Thereby, representations link sensory data to control signals. In particular, the representational states associate visual features to stable action concepts such as turning or going straight.

This paper presents a study that analyzes the effects of robots' gaze hints on children's performance in a card-matching game. We conducted a within-subjects study, in which children played a card matching game "Memory" in the presence of a robot tutor in two sessions. In one session, the robot gave hints to help the child find matching cards by looking at the correct match and, in the other session, the robot only looked at the child and did not give them any help. Children performance was measured using the number of tries and overall time used to complete the game. Our findings show that the use of gaze hints (help condition) made the matching task significantly easier and that children used significantly fewer tries than without help.

In many future social interactions between robots and humans, robots may need to convince people to change their behavior. People may dislike and resist such persuasive attempts, a phenomenon known as psychological reactance. This paper examines how reactance, measured in terms of negative cognitions and feelings of anger, is affected by the persuading agent's social agency cues and the level of controlling language used. Participants played a decision-making game in which a persuasive agent attempted to influence their choices exhibiting high or low controlling language, and three different levels of social agency. Results suggest that controlling language will lead to increased reactance when the persuasive agent does not exhibit social cues. Surprisingly, reactance is not affected by controlling language in the same way when the persuading agent is a social robot exhibiting social cues.

The study of engagement is central to improve the quality of care and provide people with dementia with meaningful activities. Current assessment techniques of engagement for people with dementia rely exclusively on behavior observation. However, novel unobtrusive sensing technologies, capable of tracking psychological states during activities, can provide us with a deeper layer of knowledge about engagement. We compared the engagement of persons with dementia involved in two playful activities, a game-based cognitive stimulation and a robot-based free play, using observational rating scales and electrodermal activity (EDA). Results highlight significant differences in observational rating scales and EDA between the two activities and several significant correlations between the items of observational rating scales of engagement and affect, and EDA features.

This paper investigates how drones could be integrated into the context of sports, boxing in particular. The goal of this project is to design a drone application that allows direct and embodied interaction. The sport of boxing provides a very interesting setting, because the intimidating and dangerous appearance of the drone could be beneficial when it is used as a boxing opponent. A concept of a drone box application was developed and a pilot experiment was performed to compare drone against human boxing with human against human boxing. It was found that even in a simple exercise, the present day drone technology wasn't fast and precise enough to provide a fluent sparring experience. A positive outcome was that participants did complete and enjoy the exercise in both cases. Present day drones could be used for boxing exercises in which no or minimum impact is required.

Gaze is an important nonverbal cue in human - human communication, for example, in communicating direction of attention. Therefore, presumably being able to understand and provide gaze cues is an important aspect in robot's interactive behavior. While there is considerable progress, as regards the design of social gaze cues for robots, there is little that has been done to examine the ability of humans to read and accept help signals from a robot's gaze. In this study, we examine how people perceive gaze cues and head angles directed towards different target positions on a table when human and NAO robot are sitting against each other as in board game scenarios. From the results, we show that when the head pitch angle is higher (24±2) and the depth is less, approximately 20 cm from the robot, participants detected the positions with good accuracy. Unexpectedly, the locations on the left of the robot were detected with lower accuracy. In conclusion, we discuss the implications of this research for design of interaction settings between human and a robot that is intended for social and educational support.

End-user programming for robots is becoming an increasingly important topic since robots are being introduced into a wide variety of domains. We propose a design of a web based programming interface that makes it possible for end-users with different backgrounds to program robots using natural language. We used the cognitive dimensions framework to compare the usability of the newly created and the currently employed programming interfaces. The results showed that domain specialists are able to make robot programs more quickly and pleasantly with the proposed interface than with an existing one. Another important finding is that without physical simulation of the robot behaviours, the end-users do not feel confident enough to develop their scenarios in a realistic setting.

Abstract To utilise the knowledge gained from highly specialised domains as autism therapy to robot‐based interactive training platforms, an innovative design approach is needed. We present the process of content creation and co‐design of LEGO therapy for children with autism spectrum disorders performed by a humanoid robot. The co‐creation takes place across the disciplines of autism therapy, and behavioural robotics, and applies methods from design and human–robot interaction, in order to connect state‐of‐the‐art developments in these disciplines. We designed, carried out and analyzed a pilot and final experiment, in which a robot mediated LEGO therapy between pairs of children was mediated by a robot over the course of 10 to 12 sessions. The impact of the training on the children was then analysed from a clinical and human–robot interaction perspective. Our major findings are as follows: first, game‐based robot scenarios in which the game continues over the sessions opened possibilities for long‐term interventions using robots and led to a significant increase in social initiations during the intervention in natural settings; and second, including dyadic interactions between robot and child within triadic games with robots has positive effects on the children's engagement and on creating learning moments that comply with the chosen therapy framework.

This paper proposes a method for interpretation of the emotions detected in facial expressions in the context of the events that cause them. The method was developed to analyze the video recordings of facial expressions depicted during a collaborative game played as a part of the Mars-500 experiment. In this experiment, six astronauts were isolated for 520 days in a space station to simulate a flight to Mars. Seven time-dependent components of facial expressions were extracted from the video recordings of the experiment. To interpret these dynamic components, we proposed a mathematical model of emotional events. Genetic programming was used to find the locations, types, and intensities of the emotional events as well as the way the recorded facial expressions represented reactions to them. By classification of different statistical properties of the data, we found that there are significant relations between the facial expressions of different crew members and a memory effect between the collective emotional states of the crew members. The model of emotional events was validated on previously unseen video recordings of the astronauts. We demonstrated that both genetic search and optimization of the parameters improve the accuracy of the proposed model. This method is a step toward automating the analysis of affective expressions in terms of the cognitive appraisal theory of emotion, which relies on the dependence of the expressed emotion on the causing event.

Detection of emotions of a crowd is a new research area, which has never, to our knowledge, been accounted for research in previous literature. A bio-inspired model for representation of emotional patterns in crowds has been demonstrated. Emotions have been defined as evolving patterns as part of a dynamic pattern of events. This model has been developed to detect emotions of a crowd based on the knowledge from a learned context, psychology and experience of people in crowd management. The emotions of multiple people making a crowd in any surveillance environment are estimated by sensors signals such as a camera and are being tracked and their behavior is modeled using bio-inspired dynamic model. The behavior changes correspond to changes in emotions. The proposed algorithm involves the probabilistic signal processing modelling techniques for analysis of different types of behavior, interaction detection and estimation of emotions. The emotions are recognized by the expectation of temporal occurrences of causal events modeled by Gaussian mixture model. The model has been evaluated using the simulated behavioral model of a crowd.

Research in uptake and actual use of robots in socially assistive tasks is rapidly growing. However, practical applications lack behind due to the enormous effort to create meaningful behaviours. This paper describes a rapid prototyping framework for robot-assisted training of children with Autism Spectrum Disorder (ASD). The main goal of this research is to provide a framework which translates the knowledge from the evidenced Pivotal Response Training to end-user tools, that allow therapists to program/adapt a training program mediated by a robot in order to use it in therapies. The overall structure of the intervention is based on Activity Theory which makes it easy to properly arrange robot actions and decisions. We appended a general end-user robot programming tool with PRT therapy-specific training structures which can be adapted with ease to create almost limitless learning opportunities utilizing a range of training scenarios or games. Pilot tests with children with ASD were performed to assess whether the robot assisted intervention created by this framework is ready for practical use. These showed that only minor adaptations were needed to increase the fluency of the robot-child interaction.

Robot assisted therapy for patients with autism is promising, but there is a need for well designed studies that combine expert knowledge from the field of robotics and autism. Here, an iterative, participatory design process of robot assisted Pivotal Response Training (PRT) for autism therapy is presented. The scenarios for the robot assisted PRT intervention were created using a scenario based design approach through intensive collaboration between robot engineer and therapist. The developed scenarios were then represented in a hierarchical model using Activity Theory. Based on that model, the PRT intervention was implemented by the end-user programming software and evaluated with a child with autism. The scenario based design and the use of Activity Theory framework considerably speeded up the scenario creation process and the redesign time needed between the pilots compared to our previous experiments.

This paper explores new possibilities for social interaction between a human user and a robot with an abstract shape. The social interaction takes place by simulating behaviors such as submissiveness and dominance and analyzing the corresponding human reactions. We used an object that has no resemblance with human features in its shape or expression mode, in order to exclude the effect of these features on the human behavior. An intelligent walk-in closet was made to behave either dominantly or submissively using lighting effects. The behaviors of the closet were rated by participants using the Bem Sex Role Inventory in a pilot study, resulting in the selection of one submissive and one dominant lighting behavior for the closet. Participants' personality was measured using the Social Dominance Orientation questionnaire. These data were then compared to measurements of user satisfaction and feelings of dominance, arousal, and valence after scenario completion. A surprising effect was revealed as participants with a dominant personality reported feeling submissive to a dominant system, while in comparison, persons with a submissive personality felt more dominant in the same condition. Furthermore, it was found that a submissive system was generally more preferred by users. We draw a careful conclusion that people interact differently with systems that show human-like attitudes, than they would in response to similar attitude expressed by other person. These findings need to be investigated further with dominant/submissive nonverbal behaviors that are then simulated on a humanoid robot.

\n Contains fulltext :\n 130083.pdf (Publisher’s version ) (Open Access)\n

Imitation learning is a promising way to learn new behavior in robotic multiagent systems and in human-robot interaction. However, imitating agents should be able to decide autonomously which behavior, observed in others, is interesting to copy. This paper shows a method for extraction of meaningful chunks of information from a continuous sequence of observed actions by using a simple recurrent network (Elman Net). Results show that, independently of the high level of task-specific noise, Elman nets can be used for learning through prediction a reoccurring action patterns, observed in another robotic agent. We conclude that this primarily robot to robot interaction study can be generalized to human-robot interaction and show how we use these results for recognizing emotional behaviors in human-robot interaction scenarios. The limitations of the proposed approach and the future directions are discussed. © 2010 Wiley Periodicals, Inc.

This paper provides a framework for recording, analyzing and modeling of 3 dimensional emotional movements for embodied game applications. To foster embodied interaction, we need interfaces that can develop a complex, meaningful understanding of intention—both kinesthetic and emotional—as it emerges through natural human movement. The movements are emulated on robots or other devices with sensory-motor features as a part of games that aim improving the social interaction skills of children. The design of an example game platform that is used for training of children with autism is described since the type of the emotional behaviors depends on the embodiment of the robot and the context of the game. The results show that quantitative movement parameters can be matched to emotional state of the embodied agent (human or robot) using the Laban movement analysis. Emotional movements that were emulated on robots using this principle were tested with children in the age group 7–9. The tests show reliable recognition on most of the behaviors.

Products can support user groups in social interaction and/or physical play in various ways. Depending on the requirements and needs of specific user groups and contexts of use, different approaches are applied to create successful designs. This special issue contains papers that describe designs for children, adults and elderly for sports, home and outdoor contexts. The papers in this issue explore how technology can contribute to enhancing user experiences in terms of social interaction and physical activities. Knowledge from very diverse research areas, such as, social psychology, persuasive technology, child development, human–robot interaction, and creativity is used as an inspiration source for the various projects.

We describe the development of a multi-agent platform and adequate games that aim to stimulate social behavior of autistic children. User tests with two games, one with emerging patterns and another with goals and dramatic elements were compared. The results show that most of the children recognized the dramatic elements, which makes us believe that by longer exposure and proper guidance autistic children might be tough social skills. Test results are described quantitatively and qualitatively.

This paper features the design process, the outcome, and preliminary tests of an interactive toy that expresses emergent behavior and can be used for behavioral training of autistic children, as well as for an engaging toy for every child. We exploit the interest of the autistic children in regular patterns and order to stimulate their motivational, explorative and social skills. As a result we have developed a toy that consists of undefined number of cubes that express emergent behavior by communicating with each other and changing their colors as a result of how they have been positioned by the players. The user tests have shown increased time of engagement of the children with the toy in comparison with their usual play routines, pronounced explorative behavior and encouraging results with improvement of turn taking interaction.

Many cells in the primary visual cortex respond differently when a stimulus is placed outside their classical receptive field (CRF) compared to the stimulus within the CRF alone, permitting integration of information at early levels in the visual processing stream that may play a key role in intermediate-level visual tasks, such a perceptual pop-out [Knierim JJ, van Essen DC (1992) J Neurophysiol 67(5):961–980; Nothdurft HC, Gallant JL, Essen DCV (1999) Visual Neurosci 16:15–34], contextual modulation [Levitt JB, Lund JS (1997) Nature 387:73–76; Das A, Gilbert CD (1999) Nature 399:655–661; Dragoi V, Sur M (2000) J Neurophysiol 83:1019–1030], and junction detection [Sillito AM, Grieve KL, Jones HE, Cudiero J, Davis J (1995) Nature 378:492–496; Das A, Gilbert CD (1999) Nature 399:655–661; Jones HE, Wang W, Sillito AM (2002) J Neurophysiol 88:2797–2808]. In this article, we construct a computational model in programming environment TiViPE [Lourens T (2004) TiViPE—Tino’s visual programming environment. In: The 28th Annual International Computer Software & Applications Conference, IEEE COMPSAC 2004, pp 10–15] of orientation contrast type of cells and demonstrate that the model closely resembles the functional behavior of the neuronal responses of non-orientation (within the CRF) sensitive 4Cβ cells [Jones HE, Wang W, Sillito AM (2002) J Neurophysiol 88:2797–2808], and give an explanation of the indirect information flow in V1 that explains the behavior of orientation contrast sensitivity. The computational model of orientation contrast cells demonstrates excitatory responses at edges near junctions that might facilitate junction detection, but the model does not reveal perceptual pop-out.