Abstract This work presents a real-time system for tracking multiple object in the context of meal preparation when using the Cognitive Orthosis for CoOKing (COOK). This system is called SafeCOOK. It aims to provide more capabilities to detect some dangerous situations that the current system does not consider. For example, it can locate a utensil or other kitchen object that has been left on the cooking surface of the stove while a meal is being prepared. This system uses a hybrid method based on YOLO and KCF to detect, track and drop cooking utensils as they enter and leave the cooking area, and is capable of monitoring an entire cooktop in real-time with a single camera. The software has been implemented on an embedded platform in the smart stove and has been added to it. The system produces good segmentation and tracking results at a frame rate of 1 to 4 frames per second, as demonstrated in extensive experiments using video sequences under different conditions.

In the last decade, advances in the field of ambient assisted living (AAL) have changed the way services can be provided in smart homes. New possibilities are now offered for addressing complex interaction problems between the technology and users with special needs. Within this context, this study addresses human-computer interactions for cognitive assistance to people with Traumatic Brain Injury (TBI). Interdisciplinary research combining computer science and occupational therapy was conducted to model the interaction between an AAL system (AALS) and a person with cognitive impairments due to TBI residing in an AAL environment. Cognitive assistance is modelled as an interactive exchange where an AALS spreads assistance cues that should induce appropriate behaviours from an assisted person as responses. After an assistance cue is delivered, the AALS should evaluate the user reaction and, stop the interaction if the intended reaction is observed or resume by adapting the assistance. To do so, evidence-based cognitive rehabilitation and speech acts are used to model the interaction content i.e., assistive cues and user feedback, while an ontology formalizes the semantics of this knowledge in a computer readable format. A behavioural model based on behaviour trees informed by the ontological model is then used to enable the cognitive assistant to plan the sequence of cues to be delivered adaptively depending on the circumstances, the assistance goal and the user's behaviour. To show that the proposed interaction model can help an AALS to provide adaptive assistance to people with cognitive impairments, it is exemplified on a cooking safety assistant designed for people with TBI.

Abstract User-centered design (UCD) facilitates the creation of technologies that are specifically designed to answer users’ needs. This paper presents the first step of a UCD using a persona, a fictitious character representing the targeted population, which in this case is people having sustained a traumatic brain injury (TBI). The persona is used to better understand the possible interactions of a TBI population with a prototype of a technology that we wish to develop, namely the Cognitive Orthosis for coOKing (COOK). COOK is meant to be an assistive technology that will be designed to promote independence for cooking within a supported-living residence. More specifically, this paper presents the persona’s creation methodology based on the first four phases of the persona’s lifecycle. It also describes how the persona methodology served as a facilitator to initiate an interdisciplinary collaboration between a clinical team and a computer science team. Creation of personas relied on a clinical model (Disability Creation Process) that contextualized the needs of this population and an evaluation tool [Instrumental Activities of Daily Living (IADL) Profile] that presented a wide range of cognitive assistance needs found in this same population. This paper provides an in-depth description of some of the most frequent everyday difficulties experienced by individuals with TBI as well as the persona’s abilities, limitations and social participation during the realization of IADL, and an evaluation of the manifestations of these difficulties during IADL performance as represented through scenarios. The interdisciplinary team used the persona to complete a first description of the interactions of a persona with TBI with COOK. This work is an attempt at offering a communication tool, the persona, to facilitate interdisciplinary research among diverse disciplines who wish to learn to develop a common language, models and methodologies at the beginning of the design process.

Ambient Assisted Living (AAL) environments encompass technical systems and the Internet of Things (IoT) tools to support seniors in their daily routines. They aim to enable seniors to live independently and safely for as long as possible when faced declining physical or cognitive capacities. This work presents the design, development and deployment of an AAL system in the context of smart cities. The proposed architecture is based on microservices and software components. We examined the requirements and specifications of AAL systems in smart homes, in efforts to describe and evaluate how they would be transposable in the case of smart cities. The system has been tested and evaluated in the laboratory; it has been deployed in real life settings within city and is still in use by five elderly people.

The aging of the world population is accompanied by a substantial increase in neurodegenerative disorders, such as dementia. Early detection of mild cognitive impairment (MCI), a clinical diagnostic that comes with an increased chance to develop dementias, could be an essential condition for promoting quality of life and independent living, as it would provide a critical window for the implementation of early pharmacological and nonpharmacological interventions. This systematic review aims to investigate the current state of knowledge on the effectiveness of smart home sensors technologies for the early detection of MCI through the monitoring of everyday life activities. This approach offers many advantages, including the continuous measurement of functional abilities in ecological environments. A systematic search of publications in MEDLINE, EMBASE, and CINAHL, before November 2017, was conducted. Seventeen studies were included in this review. Thirteen studies were based on real-life monitoring, with several sensors installed in participants' actual homes, and four studies included scenario-based assessments, in which participants had to complete various tasks in a research lab apartment. In real-life monitoring, the most used indicators of MCI were walking speed and activity/motion in the house. In scenario-based assessment, time of completion, quality of activity completion, number of errors, amount of assistance needed, and task-irrelevant behaviors during the performance of everyday activities predicted MCI in participants. Despite technological limitations and the novelty of the field, smart home technologies represent a promising potential for the early screening of MCI and could support clinicians in geriatric care.

The aging of the population will bring changes on home care for elders and gerontechnologies provide alternatives for aging in place. This article presents a concept of a smart home that reminds the resident the activities he has planned on an interactive calendar. The goal is to evaluate the feasibility of the activities recognition and to study the acceptability of such a device. A proof of concept has been carried out in the DOMUS laboratory smart home to evaluate how people react when facing activity reminders. Twelve adults executed sixteen activities that were part of a one-hour morning routine. The participants were recalled for the activities they have not performed on time. The recognition program Pradha presents a fairly good accuracy rate (73%), as the learning phase is based on activities that have been performed by other people who have not been taking part of this experiment. The participants relied on the vocal reminders and expressed satisfaction after using an interactive calendar that remembers important activities.

The Assistive living technologies provide good results for the support of specific activities transforming a home into a smart home. In this paper, we present a personalized ambient support system for elderly suffering from Alzheimer's dementia and nighttime wandering. Our goal is to help the person stay at home as long as possible and regain a regular circadian cycle while providing more comfort to the caregiver. The intervention proceeds in two phases. During the monitoring phase, the system determines the resident profile based on nighttime routines. Data is gathered from sensors dispatched in the smart home, coupled with physiological data obtained from worn sensors. Data is then classified to determine engine rules that will provide assistance to the resident to satisfy his needs. In the second phase, assistance is provided to the person by triggering rules depending on the activities occurring during night. It offers a calm environment with music and visual icons to soothe the person then encourage it to return to bed. The system is installed at the Alzheimer's home using wireless technologies. Multiple heterogeneous technologies are put in common to achieve it. Reliabilities and robustness tests were carried out in a 4 1/2 room apartment for 3 months with over 3.78 million collected data tuples. These tests have established three clusters of activities necessary for the recognition of nighttime wandering activities. This helped start an ongoing experiment in homes.

At DOMUS for the past 14 years, several cognitive orthotics were designed and implemented and evaluated, most of the time using participatory design. The resulting set of orthotics can support a wide variety of activities of daily living (ADL) to foster autonomy at home for people with cognitive impairments, e.g. medication, meal preparation, or budget. DOMUS benefits from a rich and versatile research infrastructure to design, implement, and evaluate such orthotics, namely a smart apartment on the campus, a living lab in an alternative housing unit for people with traumatic brain injury (TBI), seniors' residences, personal residences (apartments and houses). To different extents, all these places can be considered as living labs. In this paper, building on our extensive experience, we first show that participative design is the best-suited methodology for developing cognitive orthotics in living labs. Clinical researchers, caregivers, and end users are involved from the start. This ensures that design is user driven and that assistive technologies will satisfy users needs. Then we propose a classification of living labs according to the levels of control one can have on which and how much technology is deployed, on how space is organized and how it may vary from experiment to experiment, and on how the progression and execution of a scenario can be constrained or not. For each category of living labs, we discuss their different yet complementary characteristics, highlighting their pros and cons. For instance, our smart apartment provides tight control over technology, space, and execution of predefined scenarios. Accordingly evaluations then provide a lot of useful and reliable information about technology and the ability for people to use it, but less on acceptance of the technology and its integration at one's real home in her daily life habits.

This paper presents a novel model for user profile construction using causal relationships. Causal relationships are extracted from behavior sequences to build user profiles. Our model first discovers significant patterns by adapting a new sequence clustering algorithm, and then discovers pattern associations using normalized mutual information (NMI). Causal relationships between significant patterns are then extracted using the transfer entropy approach. These relationships are used to construct causal graphs of activities, to generate the user profile. In extensive experiments on a variety of datasets, we empirically demonstrate that these causality-based profiles yield a significant increase in performance on activity prediction.

This paper presents a practical and novel model for behavioral user profile construction using causal relationships. Causal relationships are extracted from behavior sequences for building user profiles. Our model discovers significant patterns from behavior sequences, then it discovers patterns associations using normalized mutual information. Causal relationships between significant patterns are then identified using the transfer entropy approach. We empirically demonstrate that these causality-based profiles accurately describe users profiles and allow developing practical Ubicomp applications.

This paper presents an approach for recognition of Activities of Daily Living (ADLs) in smart environments. Our approach is based on the frequent pattern mining principle to extract frequent patterns in the datasets collected from different sensors disseminated in a smart environment. In contrast with existing intrusive activity recognition approaches that have been proposed in the literature, where the datasets are basically composed of audio-visual or images files recorded during experiments, our approach is fully non-intrusive and it is based on the analysis of event sequences collected from heterogenous sensors. Our approach consists of two main phases, (1) frequent pattern mining to extract frequent patterns, and (2) activity recognition using a mapping function between the extracted frequent patterns and the activity models. We show through experiments how our approach accurately recognizes tasks as well as activities and outperforms the HMM model.

Designing human machine interfaces that respect the ergonomic norms and following rigorous approaches constitutes a major concern for computer systems designers. The increased need on easily accessible and usable interfaces leads researchers in this domain to create methods and models that make it possible to evaluate these interfaces in terms of utility and usability. Two different approaches are currently used to evaluate human machine interfaces, empirical approaches that require user involvement in the interface development process, and analytical approaches that do not associate the user during the interface development process. This paper presents a study of user performance on two principal tasks of the contextual assistant’s interface, developed in the context of smart homes, to assist persons with cognitive disabilities. We use three different methods to analyze and evaluate this interface, focusing basically on time of execution. Two of the models developed are based on cognitive models, which are ACT-R and GOMS and the third one is based on the Fitts’ Law model. The results show that, all models give a good prediction of user performance, even if the cognitive models show better accuracy of the user performance. Furthermore, they provide a better insight into cognitive abilities required to interact with the interface.

International audience