Demographic shifts have led to an increase in mild cognitive impairment (MCI), and this study investigates the effects of cognitive training (CT) and reminiscence therapy (RT) conducted by humans or socially assistive robots (SARs) on prefrontal cortex activation in elderly individuals with MCI, aiming to determine the most effective therapy-modality combination for promoting cognitive function. This pilot study employs a randomized control trial (RCT) design. Additionally, the study explores the efficacy of Reminiscence Therapy (RT) in comparison to Cognitive Training (CT). Eight MCI subjects, with a mean age of 70.125 years, were randomly assigned to “human-led” or “SAR-led” groups. Utilizing Functional Near- infrared Spectroscopy (fNIRS) to measure oxy-hemoglobin concentration changes in the dorsolateral prefrontal cortex (DLPFC), the study found no significant differences in the effects of human-led and SAR-led cognitive training on DLPFC activation. However, distinct patterns emerged in memory encoding and retrieval phases between RT and CT, suggesting the impacts of these interventions on brain activation in the context of MCI.

Laughter serves as a subtle social signal in human interaction, playing an essential role in expressing emotions and facilitating social connections. However, laughter comes in various forms and is usually accompanied by different non-verbal expressions, such as facial expressions and gestures. These accompanying factors can significantly influence the effect of laughter in diverse contexts, thus complicating the research on laughter, especially in understanding its role in social dynamics. Consequently, endowing robots with the ability to appropriately use laughter in interactions with humans is still a big challenge. Our current study focuses on the effect of robot laughter on robot humor expression. Our objective is to investigate whether and how two factors, the type of laughter and the robot laughter gesture, impact the overall humor performance. In this study, we selected four types of laughter (sarcastic, joyful, embarrassed, and relieved laughter) from a laughter corpus based on four specific types of jokes (Affiliative, Aggressive, Self-enhancing, and Self-defeating). For each type of laughter, we designed distinct robot gestures. During the humor performance, the robot NAO delivered jokes accompanied by matching or mismatching laughter, with or without corresponding gestures. To enhance the quantity and diversity of experimental data, we conducted an online survey utilizing recordings of the robot’s humor performance. The experimental findings indicate that when the robot’s laughter matches the type of humor in the joke, participants rate the humor performance significantly higher compared to situations where there is a mismatch. Additionally, the results confirm the positive impact of robot laughter gestures on humor performance.

Desktop-size robots are becoming commonly used in many fields such as caring, education and entertainment. But factors that affect human perception of desktop-size robots and their interaction are still unclear. The study examined the impact of robot behavior and appearance on the Robot Social Attributes Scale (RoSAS) during a virtual human-robot interaction study. The results showed that robots with human-like behavior were perceived more positively than those with random behavior, but the appearance of the robot did not have a significant impact on perception. The findings suggest that the design of humanlike behavior should be prioritized in future HRI studies and robot design. Real-world experiments are also recommended to verify the findings for the application of desktop-size robots in the healthcare field.

Abstract Human distance estimation is essential in many vital applications, specifically, in human localisation-based systems, such as independent living for older adults applications, and making places safe through preventing the transmission of contagious diseases through social distancing alert systems. Previous approaches to estimate the distance between a reference sensing device and human subject relied on visual or high-resolution thermal cameras. However, regular visual cameras have serious concerns about people’s privacy in indoor environments, and high-resolution thermal cameras are costly. This paper proposes a novel approach to estimate the distance for indoor human-centred applications using a low-resolution thermal sensor array. The proposed system presents a discrete and adaptive sensor placement continuous distance estimators using classification techniques and artificial neural network, respectively. It also proposes a real-time distance-based field of view classification through a novel image-based feature. Besides, the paper proposes a transfer application to the proposed continuous distance estimator to measure human height. The proposed approach is evaluated in different indoor environments, sensor placements with different participants. This paper shows a median overall error of $$\pm 0.2$$ ± 0.2 m in continuous-based estimation and $$96.8\%$$ 96.8 % achieved-accuracy in discrete distance estimation.

To support the independent living of older adults in their own homes, it is essential to identify their abnormal behaviors before triggering an automated alert system. Existing normal vision sensing approaches to detect human falls in the activities of daily living (ADL) experienced acceptability issues due to outstanding privacy concerns when they are deployed in personal environments. Besides, false alerts (false-positive) fall detection has not been addressed thoroughly in systems that report abnormal human behaviors as emergency alerts to the information support. This article proposes a novel human-in-the-loop fall detection approach in the ADLs using a low-resolution thermal sensor array. The motivation for enabling a human interactive model, fall detection confirmation, is to influence resource efficiency by reducing false-positive alerts while keeping the false-negative fall predictions as low as possible. The proposed approach is based on the motion sequence classification of human movements using a recurrent neural network. The proposed approach is evaluated with comprehensive experiments using different learning techniques, users, and domestic environment conditions. This article shows a performance accuracy of 99.7% to detect human falls from various typical ADLs.

Human-centric applications of a single thermal sensor array (TSA) have performed extremely well in many areas. However, most of these works have not yet reached the real applicability stage of the Internet of Things (IoT) applications. The main limitation of deploying such systems on a large scale is the challenge of fusing multiple TSAs to cover a wide inspection area, e.g., smart homes, hospitals, and many other domestic environments. On the other hand, objects that appear in the low-resolution thermal images acquired from TSA have low intraclass variations and high interclass similarities, making the identification of the overlapping regions through matching a comparable template image in multiple images very difficult. This article proposes a motion-based approach to fuse multiple TSAs and learn the domestic environment layout to enable further human-centred IoT applications to run in the cloud. Besides, a privacy improvement on utilizing these sensors in IoT applications is proposed. The proposed approach is evaluated with comprehensive experiments on different sensor placements and domestic environment conditions. This article shows an average performance of 92.5% accuracy using various machine learning techniques and use case scenarios.

There is a growing demand for efficient and privacy-preserving intelligent solutions in a multi-occupancy environment. This paper proposes a non-contact scheme for occupancy estimation using an infrared thermal sensor array, which has the advantages of low-cost, low-power, and high-performance capabilities. The proposed scheme offers an accurate human heat segmentation technique that extracts human body temperature from a noisy environment. It is shown that the proposed system can detect the empty occupancy state after utilising the segmentation technique with an accuracy of 100%. By using adaptive boosting, it is shown that the system is capable of measuring the non-empty occupancy with an overall accuracy of 98.2%.

Recurrent Neural Network (RNN) have been used for sequence-related learning tasks, such as language and action, in the field of cognitive robotics. Gated mechanisms used in LSTM and GRU perform well in remembering long-term dependency. But to better mimic the neural dynamics in cognitive processes, the Multiple Time-scales (MT) RNN uses a hierarchical organization of memory updates which is similar to human cognition. Since the MT feature is typically used with a vanilla RNN or different gated mechanisms, its effect on the updates and training is still not fully uncovered. Therefore, we conduct a comparative experiment on two MT recurrent neural network models, i.e. the Multiple Time-Scale Recurrent Neural Network (MTRNN) and the Multiple Time-Scale Gated Recurrent Unit (MTGRU), for action sequence learning in robotics. The experiment shows that the MTRNN model can be used in learning tasks with low requirements for learning of long-term dependency due to its low computation. On the other hand, the MTGRU model is appropriate for learning the longterm dependency. Furthermore, because of the duplicated feature of the MT and the GRU feature, we also propose a simplified MTGRU model, named Multiple Time-scale SingleGate Recurrent Unit (MTSRU) which could reduce computational cost while it achieves the similar performance as the original version.

This paper describes experiments to learn laughter co-occurrences with dialogue contributions. The dialogue data belongs to the special type of First Encounter Dialogues where the interlocutors meet each other for the first time and where laughter mainly functions as a sign of politeness or relief of embarrassment. The earlier studies have shown that there is a correlation between the speaker's utterance content (topic) and non-verbal communication (laughter and body movement) while in this paper we seek to learn the correlations via a neural model. The results show that there seems to be a weak correlation in our data.

The predictive processing (PP) hypothesizes that the predictive inference of our sensorimotor system is encoded implicitly in the regularities between perception and action. We propose a neural architecture in which such regularities of active inference are encoded hierarchically. We further suggest that this encoding emerges during the embodied learning process when the appropriate action is selected to minimize the prediction error in perception. Therefore, this predictive stream in the sensorimotor loop is generated in a top-down manner. Specifically, it is constantly modulated by the motor actions and is updated by the bottom-up prediction error signals. In this way, the top-down prediction originally comes from the prior experience from both perception and action representing the higher levels of this hierarchical cognition. In our proposed embodied model, we extend the PredNet Network, a hierarchical predictive coding network, with the motor action units implemented by a multi-layer perceptron network (MLP) to modulate the network top-down prediction. Two experiments, a minimalistic world experiment, and a mobile robot experiment are conducted to evaluate the proposed model in a qualitative way. In the neural representation, it can be observed that the causal inference of predictive percept from motor actions can be also observed while the agent is interacting with the environment.

The abstraction tasks are challenging for multi-modal sequences as they require a deeper semantic understanding and a novel text generation for the data. Although the recurrent neural networks (RNN) can be used to model the context of the time-sequences, in most cases the long-term dependencies of multi-modal data make the back-propagation through time training of RNN tend to vanish in the time domain. Recently, inspired from Multiple Time-scale Recurrent Neural Network (MTRNN) [1], an extension of Gated Recurrent Unit (GRU), called Multiple Time-scale Gated Recurrent Unit (MTGRU), has been proposed [2] to learn the long-term dependencies in natural language processing. Particularly it is also able to accomplish the abstraction task for paragraphs given that the time constants are well defined. In this paper, we compare the MTRNN and MTGRU in terms of its learning performances as well as their abstraction representation on higher level (with a slower neural activation). This was done by conducting two studies based on a smaller dataset (two-dimension time sequences from non-linear functions) and a relatively large data-set (43-dimension time sequences from iCub manipulation tasks with multi-modal data). We conclude that gated recurrent mechanisms may be necessary for learning long-term dependencies in large dimension multi-modal data-sets (e.g. learning of robot manipulation), even when natural language commands was not involved. But for smaller learning tasks with simple time-sequences, generic version of recurrent models, such as MTRNN, were sufficient to accomplish the abstraction task.