Assistive navigation for blind or visually impaired (BVI) individuals is of significance to extend their mobility and safety in traveling, enhancing their employment opportunities and fostering personal fulfillment. Conventional research is mainly based on robotic navigation approaches through localization, mapping, and path planning frameworks. They require heavy manual annotation of semantic information in maps and its alignment with sensor mapping. Inspired by the fact that we human beings naturally rely on language instruction inquiry and visual scene understanding to navigate in an unfamiliar environment, this paper proposes a novel vision-language model-based approach for BVI navigation. It does not need heavy-labeled indoor maps and provides a Safe and Efficient E-Wayfinding (SeeWay) assistive solution for BVI individuals. The system consists of a scene-graph map construction module, a navigation path generation module for global path inference by vision-language navigation (VLN), and a navigation with obstacle avoidance module for real-time local navigation. The SeeWay system was deployed on portable iPhone devices with cloud computing assistance for the VLN model inference. The field tests show the effectiveness of the VLN global path finding and local path re-planning. Experiments and quantitative results reveal that heuristic-style instruction outperforms direction/detailed-style instructions for VLN success rate (SR), and the SR decreases as the navigation length increases.

Traditional manual wheelchairs have a fixed seat with no movement or angle adjustment, which can seriously affect the user's comfort and greatly limit user experience. However, the electric wheelchair relies on strong intelligence and automatic features; it can not only realize the multidegree freedom adjustment of the human body and the seat but also has a rich and powerful man–machine control interface, which greatly facilitates and improves the user experience. This study upgraded a Permobil C400-powered wheelchair with multisensor data fusion technology to enrich its terrain recognition, tipping stability, and comfortability prediction. The tipping stability modeling of the wheelchair dummy system is carried out using multibody dynamics and vibration mechanics to obtain the tipping stability limit and the comfort evaluation of the wheelchair vibration acceleration on the human body during travel. Based on the elevation mapping method, the wheelchair can estimate the terrain from the local point of view at any point in time. At the same time, the RGB-D depth camera is connected to the robot operating system (ROS) system, and the open-source algorithm package RTAB-MAP is used to complete the MAP construction and collect the 3-D point-cloud terrain data. Then, the real 3-D terrain files are generated through the point-cloud stitching technology for stability simulation of the wheelchair–human system. The tipping stability and comfort indexes of the wheelchair–human system when passing over different physical terrains can be obtained. The experimental results show that the IMU data located on the human chest agree well with the simulation analysis data and are suitable for a variety of complex real-terrain conditions, verifying the accuracy of the wheelchair–human system dynamics model and the feasibility of the simulation analysis process. Thus, this modeling and simulation method can predict wheelchair stability and user comfortability well and ensure a high-performance experience.

There has been increased development of assistive robots for the home, along with empirical studies assessing cultural differences on user perception. However, little attention has been paid to cultural differences with respect to non- humanoid, multi-robot interactions in homes or otherwise. In this exploratory paper, we investigate how cultural differences may impact users’ preferences and perceived usefulness of a multi-robot system by creating an interactive online survey and considering variables often absent in HRI studies. We introduce our multi-robot design and survey construction, and report results evaluated across 191 young adult participants from China, India, and the USA. We find significant effects of culture on both participants’ preferences and perceived usefulness of the system between India and China or the USA, but not between China and the USA. We also find effects of culture on perceived usefulness to be partially mediated by participant preferences. Our findings reinforce the importance of considering cultural differences in designing domestic multi-robotic assistants.

An inevitable new frontier for the CHI community is the development of complex, larger-scale, cyber-physical artifacts where advancements in design, computing and robotics converge. Presented here is a design exemplar: the Assistive, Robotic Table (ART), the key component of our envisioned home suite of networked, robotic furnishings for hospitals and homes, promoting wellbeing and independent living. We begin with the motivations for ART, and present our iterative, five-phase, participatory design-and-evaluation process involving clinicians at a rehabilitation hospital, focusing here on the final usability study. From our wide-ranging design-research activities, which may be characterized as research through design, we found ART to be promising but also challenging. As a design exemplar, ART offers invaluable lessons to the CHI community as it comes to design larger-scale, cyber-physical artifacts cultivating interactions across people and their surroundings that define places of social, cultural and psychological significance.

Recognition of human gestures is an active area of research integral for the development of intuitive human-machine interfaces for ubiquitous computing and assistive robotics. In particular, such systems are key to effective environmental designs that facilitate aging in place. Typically, gesture recognition takes the form of template matching in which the human participant is expected to emulate a choreographed motion as prescribed by the researchers. A corresponding robotic action is then a one-to-one mapping of the template classification to a library of distinct responses. In this paper, we explore a recognition scheme based on the growing neural gas (GNG) algorithm that places no initial constraints on the user to perform gestures in a specific way. Motion descriptors extracted from sequential skeletal depth data are clustered by GNG and mapped directly to a robotic response that is refined through reinforcement learning. A simple good/bad reward signal is provided by the user. This paper presents results that show that the topology-preserving quality of GNG allows generalization between gestured commands. Experimental results using an automated reward are presented that compare learning results involving single nodes versus results involving the influence of node neighborhoods. Although separability of input data influences the speed of learning convergence for a given neighborhood radius, it is shown that learning progresses toward emulation of an associative memory that maps input gesture to desired action.

Healthcare is becoming more digital and technological, but healthcare environments have not yet become embedded with digital technologies to support the most productive (physical) interaction between medical patients, clinical staff and the physical artifacts that surround and envelop them. This shortcoming is an opportunity for the architecture and robotics communities to interface with each other and the everyday users of healthcare environments. Our extended lab focused ten weeks on sketching in hardware a robotic, patient-room ecosystem we call home+ with the help of clinicians at the Roger C. Peace Rehabilitation Hospital of the Greenville Hospital System University Medical Center [GHS]. This early prototyping effort represents our vision for the larger robotic patient room, and identifies opportunities for more focused work on an Assistive Robotic Table (ART).

While high-technology has become pervasive in hospitals, domestic environments remain essentially low-tech and conventional, despite the care needs of an aging population wishing to age in place. In response, an interdisciplinary team - robotics engineer, architect, human factors psychologist and gerontologist - are designing, constructing, field testing, and evaluating comforTABLE, an intelligent environment for aging in place. comforTABLE is designed to increase the quality of life of both healthy individuals as well as persons with impaired mobility by intelligently supporting the physical organization of their immediate environment. While comforTABLE features intelligent behavior and robotic elements, comforTABLE aims to help people do things for themselves. This paper introduces the motivations for comforTABLE, presents its three intelligent, networked components and describes scenarios of how the system might operate in domestic situations.