Early mobilization is a structured protocol designed to facilitate motor recovery in intensive care unit (ICU) patients with ICU-acquired weakness. This process is typically implemented by an interdisciplinary team of nurses, physical therapists, and other healthcare professionals. However, its application is often constrained by the patients' critical conditions, limited mobility, and the challenges of coordinating care within resource-intensive ICU environments. In this study, we developed a patient-centered virtual reality (VR) exergame through an interdisciplinary design process involving clinicians and therapists, tailored to the constraints of critical care. The exergame incorporates progressive mobility levels that mirror early mobilization practices, and includes an embodied avatar to provide guidance and motivation. Using Meta Quest 3 body tracking, the system captures and visualizes patients' movements, thereby providing motivational engagement and quantifiable mobility metrics. We evaluated the exergame in two stages: a dual-user study involving healthy participants and healthcare professionals or students (N = 13), and a subsequent study with cardiothoracic ICU patients (N = 18) to assess feasibility, design validity, and clinical acceptance. Across both studies, participants reported high enjoyment and engagement without discomfort or stress. Furthermore, patients demonstrated increases in movement speed, range of motion, and workspace volume of the upper body across game levels. Physiological monitoring further indicated that the exergame elicited exertion without inducing excessive cardiovascular responses. These findings highlight the feasibility of VR exergames as a clinically acceptable and engaging adjunct to early mobilization in critical care, offering a novel pathway to improve rehabilitation outcomes for ICU patients.

External ventricular drain (EVD) is a common, yet challenging neurosurgical procedure of placing a catheter into the brain ventricular system that requires prolonged training for surgeons to improve the catheter placement accuracy. In this article, we introduce NeuroLens, an Augmented Reality (AR) system that provides neurosurgeons with guidance that aids them in completing an EVD catheter placement. NeuroLens builds on prior work in AR-assisted EVD to present a registered hologram of a patient's ventricles to the surgeons, and uniquely incorporates guidance on the EVD catheter's trajectory, angle of insertion, and distance to the target. The guidance is enabled by tracking the EVD catheter. We evaluate NeuroLens via a study with 33 medical students and 9 neurosurgeons, in which we analyzed participants' EVD catheter insertion accuracy and completion time, eye gaze patterns, and qualitative responses. Our study, in which NeuroLens was used to aid students and surgeons in inserting an EVD catheter into a realistic phantom model of a human head, demonstrated the potential of NeuroLens as a tool that will aid and educate novice neurosurgeons. On average, the use of NeuroLens improved the EVD placement accuracy of the year 1 students by 39.4%, of the year 2$-$-4 students by 45.7%, and of the neurosurgeons by 16.7%. Furthermore, students who focused more on NeuroLens-provided contextual guidance achieved better results, and novice surgeons improved more than the expert surgeons with NeuroLens's assistance.

In ophthalmology, retinal laser therapy is a treatment for retinopathy that requires the use of magnifying lens to treat damaged regions of retinal landmarks, hence creating challenges of inverted magnified images and requiring prolonged training. Augmented Reality (AR) can benefit clinicians during retinal laser therapy by guiding them with retinal landmark holograms and contextual information. Though recent developments in AR magnification show that a direct overlay of the magnified scenes can be achieved, retinal laser therapy requires high precision and visual acuity while maintaining the visual perception of the rest of the environment. Therefore, we demonstrate an AR-based selective magnification system that provides contextual and visualization-based guidance to clinicians. An edge-computing architecture is developed for detecting and matching the feature points between the magnified image and color fundus image of the retina to identify the magnified region of retinal landmarks. We showcase how our AR guidance system can assist clinicians during retinal laser therapy.

External ventricular drain (EVD) is a common, yet challenging neurosurgical procedure of placing a catheter into the brain ventricular system that requires prolonged training for surgeons to improve the catheter placement accuracy. In this paper, we introduce NeuroLens, an Augmented Reality (AR) system that provides neurosurgeons with guidance that aides them in completing an EVD catheter placement. NeuroLens builds on prior work in AR-assisted EVD to present a registered hologram of a patient’s ventricles to the surgeons, and uniquely incorporates guidance on the EVD catheter’s trajectory, angle of insertion, and distance to the target. The guidance is enabled by tracking the EVD catheter. We evaluate NeuroLens via a study with 33 medical students, in which we analyzed students’ EVD catheter insertion accuracy and completion time, eye gaze patterns, and qualitative responses. Our study, in which NeuroLens was used to aid students in inserting an EVD catheter into a realistic phantom model of a human head, demonstrated the potential of NeuroLens as a tool that will aid and educate novice neurosurgeons. On average, the use of NeuroLens improved the EVD placement accuracy of year 1 students by 39.4% and of the year 2–4 students by 45.7%. Furthermore, students who focused more on NeuroLens-provided contextual guidance achieved better results.

Sensing and Augmented Reality (AR) can benefit a wide range of applications that involve the use of magnifying lenses. Recent developments in AR magnification provide a direct overlay of the magnified scenes in AR. However, instrumentation tasks that require high precision and visual acuity need to selectively magnify a region of interest while maintaining the visual perception of the rest of the environment. In this demo, we present AR-Magnifier, an AR magnification system through feature detection and matching. We propose a general framework based on an edge-computing architecture that can be applied to various types of instrumentation tasks. A pipeline is developed for detecting feature points and computing the homography matching to identify the magnified region of an object. We showcase how selective magnification in AR through sensing can assist the user in complex instrumentation tasks by providing visualization-based guidance.

Smart devices have entered all spheres of modern living, from monitoring the steps we walk to managing refrigerator inventory, ushering in the dawn of a new urban experience. The kitchen is the heart of the home; a place to share, care for and nurture the family unit, but also a place seeing the greatest impact from the introduction of smart devices. The smart sensing and remote control capability of smart appliances have enabled great physical convenience for users but have had less impact on cognitive conveniences. While such devices can sensewhatthey are working with, they fail to understandwhothey are working for, leaving much of the burden of trivial planning and decision making to humans with less personalized services. Hence, we introduce TupperwareEarth, a knowledge-based ontological semantic network for the “Internet of Kitchen Things” with the aim of reducing physical as well as cognitive loads of humans in cooking tasks. Also, we present a testbed for exploring kitchen innovation and validating the effectiveness of TupperwareEarth that combines intelligent kitchen storage containers, Smart Tupperware, and existing smart kitchen appliances through an Internet of Things network and a user-friendly front-end interface, Tuppy. Using this testbed, the quantitative user studies show a 33% reduction in average food preparation time and qualitative user surveys show that 75% of the users observed a significant reduction in cognitive loads, thereby validating the cognitive conveniences granted by TupperwareEarth.

This paper describes the optimization of a neuromorphic architecture for printable organic neurons as part of an ongoing project to develop Form + Function 4-D Printing. The previously proposed architecture prioritizes simplicity and massive redundancy for a printable analog neural network consisting of only one transistor plus memristors for synapses, per neuron, but sacrifices negative synaptic weights. This paper demonstrates an optimization technique to minimize the insertion of inverting amplifiers to realize a minimal approximating set of negative weights. This helps to develop a compact, printable and accurate neuromorphic computer to bring new function to the 3-D printing of form in multi-functional robotic materials. An example robotic skin is developed with the ability to compute the centroid of touch "compiled into the skin" to an average accuracy of 9.19% in comparison to an unconstrained Artificial Neural Network (ANN). The presented soft robotic skin is fabricated by hand using conventional silicon components, but serves as a proof-of-concept for radical new capabilities in Form + Function 4-D Printing.