Human activity recognition (HAR) has become a key technology for improving the quality of life for individuals with special needs, such as older adults and children with Down Syndrome (CwDS). This study presents a novel application of HAR systems to detect directive behaviors in parent-child interactions, focusing on parents of CwDS during educational activities. Using video data, we developed two models: a Convolutional Neural Network (CNN) and a hybrid CNN-LSTM model, to recognize subtle cues such as physical proximity and verbal interactions. The CNN3D model achieved over 90% accuracy in detecting approach behaviors and around 65% for verbal expressions. The CNN-LSTM model outperformed CNN3D in classifying verbal expressions, achieving over 68% accuracy. These results highlight the potential of deep learning classifiers for analyzing subtle parent-child interactions, offering valuable insights into parent-child dynamics and contributing to the development of assistive tools for studying educational settings.

In this work, we aim at identifying the speaker in interactions between mothers and children with Down syndrome (DS) using audio. We collected audio from a session in which children with DS solved puzzles, and their mothers were by their side. We generated a dataset by manually annotating human speech activity and non-speech. We used machine learning to perform four experiments, including individual and generalized models achieving on average F1-scores of 0.74 (five-class model) and 0.84 (two-class) for individual models and 0.69 (five-class) and 0.82 (two-class) for the generalized models. Our results can be helpful to behavioral researchers, therapists, and those interested in better understanding how mother–child interactions unfold in naturalistic settings.

Abstract Museums have employed video games in their exhibitions to foster visitors’ participation and improve their learning process through a positive experience. In this work, we present Aventura Marina, a serious video game inspired by escape room game mechanics, composed of an app and an authoring tool, which aims to promote visitors’ engagement with the new exhibition Sea Hall of the Caracol Museo de Ciencias, a Mexican museum in the northwest of Mexico. To design Aventura Marina, we followed a user-centered design approach, based on the design thinking methodology. We evaluated the high-fidelity prototype of the game and the low-fidelity prototype of the authoring tool with museum staff and potential visitors. Our results indicate that Aventura Marina has the potential to promote visitors’ engagement with the Sea Hall exhibition as participants indicated having a positive experience, recognized that they were more interested in and less bored by the topics of the exhibition and that they were eager to play the game again. In relation to the authoring tool, museum staff mentioned how this project could be implemented in other halls of the museum. Finally, we discuss the improvements for a future iteration of the prototypes and how this kind of technology can complement the content of museum exhibitions.

Abstract Down syndrome is considered the most common genetic cause of intellectual disability. People with intellectual disabilities have deficits in intellectual functioning such as attention, reasoning and academic learning. According to the American Association on Intellectual and Developmental Disabilities and the Centers for Disease Control and Prevention, intellectual disability encompasses only a person’s intellectual functioning; however, it can co-exist with other developmental disabilities, such as Down syndrome. People with Down syndrome may have intellectual disabilities in the mild to moderate range. Because of this, they exhibit some cognitive and emotional characteristics of intellectual disability. Among the emotional characteristics of these populations are difficulties in recognizing some emotions, which triggers anxiety and depression problems. The literature proposes using serious video games to support different skills of people with Down syndrome and intellectual disabilities. Some of these works have shown how these serious video games support the development of different skills of these populations. However, they have not been designed to support their emotional awareness. Likewise, technology is aimed at supporting emotion recognition and identification, but most of these have been aimed at other populations, such as people with autism. This paper presents the evaluation process of the design of Emotion4Down: a serious video game to support emotional awareness. We conducted this evaluation in three stages involving people with intellectual disabilities and Down syndrome.

Among the characteristics of people with Down syndrome is the difficulty in identifying and expressing their own emotions. This difficulty stems from the lack of development of their emotional awareness which can cause them anxiety, and depression when they are exposed to significant situations in their lives, and they do not have the skills to express their emotions.

Children with autism have fine-motor coordination problems, which have a significant impact on the execution of activities of daily living. To practice their fine-motor coordination skills, they attend occupational therapy, which involves repetitive motor activities. However, sometimes traditional therapy can be perceived by children as tedious and boring. On the other hand, games-based therapy has shown to be effective and more fun. In this sense, gesture-based video games, digital games with gestural interaction as a game controller, might have the potential to support the fine-motor coordination skills of children with autism. In this paper, we present the design of a gesture-based video game to support the fine-motor coordination skills of children with autism, with the goal to contribute to the proper execution of activities of daily living, required for achieving independent living; and the process followed in the design.

Voice User Interfaces (VUIs) are growing in popularity. However, even the most current VUIs regularly cause frustration for their users. Very few studies exist on what people do to overcome VUI problems they encounter, or how VUIs can be designed to aid people when these problems occur. In this paper, we analyze empirical data on how users (n=12) interact with our VUI calendar system, DiscoverCal, over three sessions. In particular, we identify the main obstacle categories and types of tactics our participants employ to overcome them. We analyzed the patterns of how different tactics are used in each obstacle category. We found that while NLP Error obstacles occurred the most, other obstacles are more likely to frustrate or confuse the user. We also found patterns that suggest participants were more likely to employ a "guessing" approach rather than rely on visual aids or knowledge recall.

We present the development and evaluation of an online learning system for people with developmental disabilities (DD) of all ages in collaboration with Imagine! and Hope Services, two not-for-profit organizations that provide care services to people with DD. The system was implemented as an HTML5-based web application for iPad. It includes activities that aim to support and improve the process through which people with DD of all ages reinforce basic skills such as recognizing numbers, letters, money, shapes, and colors. User evaluations suggest that a system such as ours will be: 1) helpful in supporting people with DD reinforce basic skills, and 2) well-received by users.

Motor coordination problems are common in different developmental disorders including autism and dyspraxia. Gross and fine motor coordination skills are critical to the appropriate motor coordination development that is relevant to support individuals' independence. Exergames are a good tool to help children practice motor skills as they find them engaging. In this work, we present how FroggyBobby an exergame designed for practicing gross motor coordination skills, can be extended to combine gross and fine motor exercises for supporting children with motor problems to practice self-care activities that require motor coordination.