People living with dementia (PwD) in residential care settings often exhibit responsive behaviors, with agitation being the most common behavior. Automated systems to detect agitation events have the potential to improve patient care by helping to track symptoms and their response to interventions. We conducted a study from 2017 to 2019 in which we collected a novel multi-modal sensor dataset from 20 PwD living in a dementia care unit. Each participant wore an Empatica E4 watch for a maximum period of up to 60 days, leading to a large dataset worth 600 days. This wearable device collects raw acceleration, blood volume pulse, electro-dermal activity, and skin temperature data. The data are annotated with the start and end times of agitation events. Our previous analyses have shown that agitation behavior can be detected with a high area under receiver operating characteristic curve. We are now releasing this novel dataset for the research community to advance research in the field. In this article, we describe the study details, protocol used for annotation, improved agitation labelling, signal processing steps, and feature extraction approach. We present a new baseline on this dataset that can fuel new research in this important area of research.

This paper presents a novel home-based reminder system designed to enhance communication and support for people living with dementia and their caregivers. Leveraging machine learning and data analytics, this system aims to provide behavioral insights that tailor the care process, promoting autonomy and independence in daily living for older adults. The research emphasizes a co-collaborative, user-informed design approach, ensuring that the technology aligns with the needs and preferences of its users. Beyond the project’s scope, this study explores the landscape of assistive technologies (ATs) for older adults, discussing the potential of artificial intelligence (AI) to revolutionize healthy aging. It highlights the importance of ethical considerations, such as privacy and user autonomy, in the development of technological solutions. The objective is to advance the dialogue on employing emerging technologies to promote sustained healthy aging, with this study serving as both a model and a broader investigation into AI's potential within assistive solutions.

Mode confusion occurs when the driver of an automated vehicle (AV) is confused about the active operating mode of the AV and therefore, their responsibilities as the driver. Mode confusion is a serious safety concern, especially for cohorts who are less familiar with AVs and/or who are more likely to have poorer situational awareness during automated driving such as older adults. In this article, we propose a design framework for driver state monitoring systems that can potentially be used to detect older drivers’ mode confusion by inferring drivers’ perceived AV mode using gaze behaviour data. As a proof-of-concept for an AV with two modes, the efficacy of the proposed framework is tested by applying it on a gaze behaviour dataset collected from 29 older drivers (65+) during simulated non-automated and simulated fully automated drives. The proposed framework utilizes classification models trained on features extracted from the gaze behaviour data. Among 25 features, the mRMR (maximum relevance minimum redundancy) feature ranking framework ranked our proposed feature of weighted static gaze entropy as having the highest relevance with the driver’s perceived AV modes while having the least redundancy with the rest of the selected features. An ensemble stacking model achieved the highest classification performance with an average accuracy of 73% and an average AUC score of 80%. The results indicate that gaze behaviour features can distinguish between the driving scenarios of automated and non-automated as perceived by the drivers. While the dataset does not include confirmed instances of driver’s mode confusion and therefore, the framework testing provides preliminary results towards a proof of concept, this work provides a foundational model for future studies in which actual data from confirmed mode confusions are intentionally introduced or measured. In turn, this study can inform future designs of driver state monitoring systems aimed to detect and mitigate the safety risks of driver’s mode confusions in automated vehicles.

Unlike traditional hierarchical controllers for robotic leg prostheses and exoskeletons, continuous systems could allow persons with mobility impairments to walk more naturally in real-world environments without requiring high-level switching between locomotion modes. To support these next-generation controllers, we developed a new system called KIFNet (Kinematics and Image Fusing Network) that uses lightweight and efficient deep learning models to continuously predict the leg kinematics during walking. We tested different sensor fusion methods to combine kinematics data from inertial sensors and computer vision data from smart glasses and found that adaptive instance normalization achieved the lowest RMSE predictions for knee and ankle joint kinematics. We also deployed our model on an embedded device. Without inference optimization, our model was 20 times faster than the previous state-of-the-art and achieved 20% higher prediction accuracies, and during some locomotor activities like stair descent, decreased RMSE up to 300%. With inference optimization, our best model achieved 125 FPS on an NVIDIA Jetson Nano. These results demonstrate the potential to build fast and accurate deep learning models for continuous prediction of leg kinematics during walking based on sensor fusion and embedded computing, therein providing a foundation for real-time continuous controllers for robotic leg prostheses and exoskeletons.

Convolutional neural networks trained using supervised learning can improve visual perception for human-robot walking. These advances have been possible due to largescale datasets like ExoNet and StairNet - the largest open-source image datasets of real-world walking environments. However, these datasets require vast amounts of manually annotated data, the development of which is time consuming and labor intensive. Here we present a novel semi-supervised learning system (ExoNet-SSL) that uses over 1.2 million unlabelled images from ExoNet to improve training efficiency. We developed a deep learning model based on mobile vision transformers and trained the model using semi-supervised learning for image classification. Compared to standard supervised learning (98.4%), our ExoNet-SSL system was able to maintain high prediction accuracy (98.8%) when tested on previously unseen environments, while requiring 35% fewer labelled images during training. These results show that semi-supervised learning can improve training efficiency by leveraging large amounts of unlabelled data and minimize the size requirements for manually annotated images. Future research$\text{will}$focus on model deployment for onboard real-time inference and control of human-robot walking.

Persons with dementia and their care partners have been found to adapt their own technological arrangements using commercially available information and communication technologies (ICTs). Yet, little is known about these processes of technology appropriation and how care practices are impacted. Adopting a relational perspective of care, we longitudinally examined how four family care networks appropriated a new commercial ICT service into their existing technological arrangements and care practices. Cross-case analysis interpreted collaborative appropriation to encompass two interrelated processes of creating and adapting technological practices and negotiating and augmenting care relationships . Four driving forces were also proposed: motivating meanings that actors ascribe to the technology and its use; the learnability of the technology and actors’ resourcefulness ; the establishment of responsive and cooperative care practices ; and the qualities of empathy and shared power in care relationships . The importance of technological literacy, learning, meaning-making, and the nature and quality of care relationships are discussed. Future work is urged to employ longitudinal and naturalistic approaches, and focus design efforts on promoting synergistic care relationships and care practices.

Socially assistive robots can autonomously provide activity assistance to vulnerable populations, including those living with cognitive impairments. To provide effective assistance, these robots should be capable of displaying appropriate behaviors and personalizing them to a user's cognitive abilities. Our research focuses on the development of a novel robot learning architecture that uniquely combines learning from demonstration ( LfD ) and reinforcement learning ( RL ) algorithms to effectively teach socially assistive robots personalized behaviors. Caregivers can demonstrate a series of assistive behaviors for an activity to the robot, which it uses to learn general behaviors via LfD . This information is used to obtain initial assistive state-behavior pairings using a decision tree. Then, the robot uses an RL algorithm to obtain a policy for selecting the appropriate behavior personalized to the user's cognition level. Experiments were conducted with the socially assistive robot Casper to investigate the effectiveness of our proposed learning architecture. Results showed that Casper was able to learn personalized behaviors for the new assistive activity of tea-making, and that combining LfD and RL algorithms significantly reduces the time required for a robot to learn a new activity.

The early detection of dementias such as Alzheimer's disease can in some cases reverse, stop, or slow cognitive decline and in general greatly reduce the burden of care. This is of increasing significance as demographic studies are warning of an aging population in North America and worldwide. Various smart homes and systems have been developed to detect cognitive decline through continuous monitoring of high risk individuals. However, the majority of these smart homes and systems use a number of predefined heuristics to detect changes in cognition, which has been demonstrated to focus on the idiosyncratic nuances of the individual subjects, and thus, does not generalize. In this paper, we address this problem by building generalized linear models of home activity of older adults monitored using unobtrusive sensing technologies. We use inhomogenous Poisson processes to model the presence of the recruited older adults within different rooms throughout the day. We employ an information theoretic approach to compare the generalized linear models learned, and we observe significant statistical differences between the cognitively intact and impaired older adults. Using a simple thresholding approach, we were able to detect mild cognitive impairment in older adults with an average area under the ROC curve of 0.716 and an average area under the precision-recall curve of 0.706 using activity models estimated over a time window of 12 weeks.

This study applies mixture-model clustering to spatiotemporal gait parameters in order to characterize the pathological gait pattern and to generate a composite measure indicative of overall gait performance. Gait data from 68 adults with stroke (age: 61.5 ± 13.6 years) and 20 healthy adults (age: 28.8 ± 7.1 years) were used in this study. Participants performed three passes across a GAITRite mat at different time points following stroke (poststroke adults only). Mixture-model clustering grouped participants' gait patterns based on their spatiotemporal gait features including symmetry, speed, and variability. Mixture-models with different covariance matrix parameterizations and numbers of clusters were examined. The selected clustering model successfully categorized participants' spatiotemporal gait data into three clinically meaningful groups. Based on the clustering results, gait speed, and variability measures varied across the three groups. Individuals in Group 1 are all symmetric and had the fastest and lowest gait velocity and variability, respectively. As expected, healthy participants were assigned to Group 1. All gait parameters were at an intermediate level in Group 2 and worse condition in Group 3. Moreover, resulting cluster centers were in line with previously published clinical studies on gait. In addition to clustering, each individual was given an indexed membership (ranged 0-1) to each of three groups. These indexed memberships were proposed as a single measure to encompass information about multiple gait parameters (symmetry, speed, and variability) and as a measure that is sensitive and responsive to improvement or deterioration and rehabilitation over time.

In recent years, we have witnessed a rapid surge in assisted living technologies due to a rapidly aging society. The aging population, the increasing cost of formal health care, the caregiver burden, and the importance that the individuals place on living independently, all motivate development of innovative-assisted living technologies for safe and independent aging. In this survey, we will summarize the emergence of 'ambient-assisted living" (AAL) tools for older adults based on ambient intelligence paradigm. We will summarize the state-of-the-art AAL technologies, tools, and techniques, and we will look at current and future challenges.

The ratio of healthcare professionals to care recipients is dropping at an alarming rate, particularly for the older population. It is estimated that the number of persons with Alzheimer's disease, for example, will top 100 million worldwide by the year 2050 [Alzheimer's Disease International 2009]. It will become harder and harder to provide needed health services to this population of older adults. Further, patients are becoming more aware and involved in their own healthcare decisions. This is creating a void in which technology has an increasingly important role to play as a tool to connect providers with recipients. Examples of interactive technologies range from telecare for remote regions to computer games promoting fitness in the home. Currently, such technologies are developed for specific applications and are difficult to modify to suit individual user needs. The future potential economic and social impact of technology in the healthcare field therefore lies in our ability to make intelligent devices that are customizable by healthcare professionals and their clients, that are adaptive to users over time, and that generalize across tasks and environments. A wide application area for technology in healthcare is for assistance and monitoring in the home. As the population ages, it becomes increasingly dependent on chronic healthcare, such as assistance for tasks of everyday life (washing, cooking, dressing), medication taking, nutrition, and fitness. This article will present a summary of work over the past decade on the development of intelligent systems that provide assistance to persons with cognitive disabilities. These systems are unique in that they are all built using a common framework, a decision-theoretic model for general-purpose assistance in the home. In this article, we will show how this type of general model can be applied to a range of assistance tasks, including prompting for activities of daily living, assistance for art therapists, and stroke rehabilitation. This model is a Partially Observable Markov Decision Process (POMDP) that can be customized by end-users, that can integrate complex sensor information, and that can adapt over time. These three characteristics of the POMDP model will allow for increasing uptake and long-term efficiency and robustness of technology for assistance.

Many older adults with cognitive impairment are excluded from powered wheelchair use because of safety concerns. This leads to reduced mobility, and in turn, higher dependence on caregivers. In this paper, we describe an intelligent wheelchair that uses computer vision and machine learning methods to provide adaptive navigation assistance to users with cognitive impairment. We demonstrate the performance of the system in a user study with the target population. We show that the collision avoidance module of the system successfully decreases the number of collisions for all participants. We also show that the wayfinding module assists users with memory and vision impairments. We share feedback from the users on various aspects of the intelligent wheelchair system. In addition, we provide our own observations and insights on the target population and their use of intelligent wheelchairs. Finally, we suggest directions for future work.

This paper describes the development of a tool that assists arts therapists working with older adults with dementia. Participation in creative activities is becoming accepted as a method for improving quality of life. This paper presents the design of a novel tool to increase the capacity of creative arts therapists to engage cognitively impaired older adults in creative activities. The tool is a creative arts touch-screen interface that presents a user with activities such as painting, drawing, or collage. It was developed with a user-centered design methodology in collaboration with a group of creative arts therapists. The tool is customizable by therapists, allowing them to design and build personalized therapeutic/goal-oriented creative activities for each client. In this paper, we evaluate the acceptability of the tool by arts therapists (our primary user group). We perform this evaluation qualitatively with a set of one-on-one interviews with arts therapists who work specifically with persons with dementia. We show how their responses during interviews support the idea of a customizable assistance tool. We evaluate the tool in simulation by showing a number of examples, and by demonstrating customizable components.

Older adults with dementia often cannot remember how to complete activities of daily living and require a caregiver to aid them through the steps involved. The use of a computerized guidance system could potentially reduce the reliance on a caregiver. This paper examines the design and preliminary evaluation of a planning system that uses Markov decision processes (MDPs) to determine when and how to provide prompts to a user with dementia for guidance through the activity of handwashing. Results from the study suggest that MDPs can be applied effectively to this type of guidance problem. Considerations for the development of future guidance systems are presented.

This paper discusses the use of computer vision in pervasive healthcare systems, specifically in the design of a sensing agent for an intelligent environment that assists older adults with dementia during an activity of daily living. An overview of the techniques applied in this particular example is provided, along with results from preliminary trials completed using the new sensing agent. A discussion of the results obtained to date is presented, including technical and social issues that remain for the advancement and acceptance of this type of technology within pervasive healthcare.