Half a century since the concept of a cyborg was introduced, digital cyborgs, enabled by the spread of wearable robotics, are the focus of much research in recent times. We introduce JIZAI ARMS, a supernumerary robotic limb system consisting of a wearable base unit with six terminals and detachable robot arms controllable by the wearer. The system was designed to enable social interaction between multiple wearers, such as an exchange of arm(s), and explore possible interactions between digital cyborgs in a cyborg society. This paper describes the JIZAI ARMS’ design process, an interdisciplinary collaboration between human augmentation researchers, product designers, a system architect, and manufacturers, to realize a technically complex system while considering the aesthetics of a digital cyborg. We also provide an autobiographical report of our first impressions of using the JIZAI ARMS and use our findings to speculate on a model of potential social interactions between digital cyborgs.

This paper describes a case study wherein we realized a novel remote shopping experience by using T-Leap, a wearable telepresence system with an integrated 360° camera, to connect a supermarket in the city center with elderly participants living far from the store. The participants of our case study operated T-Leap via a PC/tablet and remotely joined a surrogate shopper to explore the supermarket and shop for products. In this study, we revealed that a real-time 360° view provides opportunities for accidental, serendipitous purchasing activities, despite being difficult for the elderly participants to control. Reflecting on our findings, we discuss the design of future remote shopping services from an HCI perspective. Our design implications will contribute to assisting people who have difficulties regularly shopping for their daily needs, like our participants, have rich, in-store experiences that have never been provided by existing online shopping services.

While floral tributes are commonly used for the public commemoration of victims of disasters, war, and other accidents, flowers in vases color everyday life. In this research, these features of flowers are intertwined with the recent phenomenon of online memorials to develop a virtual floral tribute concept that includes physical rituals. We designed SenseVase, a smart vase to detect flowers placed in it, and a 3DCG Virtual Memorial that illustrates floral tributes given by people using SenseVases at home. This paper describes how we developed our design concept by reviewing previous literature and social aspects, and presents a video illustrating the concept. To validate the current concept, we interviewed several experts knowledgeable in public commemorations, virtual and online communities, and the floral business. Through a discussion of our findings from the design process and interviews, we propose a new direction for how HCI technology can contribute to public commemoration in addition to personal memorialization.

Funerals are irreplaceable events, especially for bereaved family members and relatives. However, the COVID-19 pandemic has prevented many people worldwide from attending their loved ones’ funerals. The authors had the opportunity to assist one family faced with this predicament by webcasting and recording funeral rites held near Tokyo in June, 2020. Using our original 360-degree Telepresence system and smartphones running Zoom, we enabled the deceased’s elder siblings to remotely attend the funeral and did our utmost to make them feel present in the funeral hall. Despite the webcasting via Zoom contributing more to their remote attendances than our system, we discovered thoughtful findings which could be useful for designing remote funeral attendances. From the findings, we also discuss how HCI designers can contribute to this highly sensitive issue, weaving together knowledge from various domains including techno-spiritual practices, thanato-sensitive designs; and other religious and cultural aspects related to death rituals.

Makeup (i.e., cosmetics) has long been used to transform not only one’s appearance but also their self-representation. Previous studies have demonstrated that visual transformations can induce a variety of effects on self-representation. Herein, we introduce Digital Speech Makeup (DSM), the novel concept of using voice conversion (VC) based auditory feedback to transform human self-representation. We implemented a proof-of-concept system that leverages a state-of-the-art algorithm for near real-time VC and bone-conduction headphones for resolving speech disruptions caused by delayed auditory feedback. Our user study confirmed that conversing for a few dozen minutes using the system influenced participants’ speech ownership and implicit bias. Furthermore, we reviewed the participants’ comments about the experience of DSM and gained additional qualitative insight into possible future directions for the concept. Our work represents the first step towards utilizing VC to design various interpersonal interactions, centered on influencing the users’ psychological state.

Haptic information significantly improves human awareness of objects in virtual reality. One way of presenting this information is via encountered-type haptic feedback. An advantage of encounter type feedback is that it enables physical interaction with virtual environments without the need for specialized haptic devices on the hand. Additionally, encountered-type haptics are known for being able to provide high quality contact feedback to the user. However, such systems are typically designed to be grounded (i.e., fixed to the floor). As such, they typically have bounded workspace and a limited range of possible applications. In this work, we present a novel, wearable approach to presenting a user with encountered-type haptic feedback. We realize this feedback using a wearable robotic limb that holds a plate where the user might interact with their environment. An appropriate location for the plate is determined by a novel haptic solver while control of the arm is made possible using motion trackers. The system was designed to be stable, for presenting consistent haptic feedback, while also being safe and lightweight for wearability. By making the feedback system wearable, we enable the presentation of stiff feedback while maintaining the spatial freedom and unbounded workspace of natural hand interaction. Herein, we present the design of the novel system, mechanical and safety considerations when designing a wearable encountered-type system, and an evaluation of the system. A technical evaluation of the implemented system showed that the system provides a stiffness over 25 N/m and slant angle errors under 3°. Three user studies show the limitations of haptic slant perception in humans and the quantitative and qualitative effectiveness of the current prototype system. We conclude the paper by discussing various potentialapplications and possible improvements that could be made to the system.

This paper describes a field study conducted with our system, T-Leap, a telepresence system connecting one person (the Viewer), situated indoors, with multiple destinations (the Nodes), that roam outdoors. Here, each Node is a person wearing a module that includes a 360-degree camera and a microphone-speaker. Through our study, we demonstrate that T-Leap enables the Viewer to perform various interactions with the Nodes including being helped by them, collaborating with them, and guiding them. These interactions were demonstrated through three studies completing different tasks: 1) Nodes purchasing souvenirs for the Viewer, 2) Nodes finding objects in the park, and 3) Viewer guiding Nodes to purchase things. The studies were primarily conducted with Taiwanese locals and Japanese visitors in Taipei. Throughout the studies, we found that T-Leap worked especially well for mediating communication between a Viewer with local knowledge acting as a guide and several Nodes who were being guided. To conclude the paper, we broadly discuss our findings, the lessons we learned from our field study, and present recommendations for the future development of mobile and wearable telepresence systems.

Locating a mobile target, untrackable in real-time, is pertinent to numerous time-critical applications, such as wilderness search and rescue. This paper proposes a hybrid approach to this dynamic problem, where both static and mobile sensors are utilized for the goal of detecting a target. The approach is novel in that a team of robots utilized to deploy a static-sensor network also actively searches for the target via on-board sensors. Synergy is achieved through: 1) optimal deployment planning of static-sensor networks and 2) optimal routing and motion planning of the robots for the deployment of the network and target search. The static-sensor network is planned first to maximize the likelihood of target detection while ensuring (temporal and spatial) unbiasedness in target motion. Robot motions are, subsequently, planned in two stages: 1) route planning and 2) trajectory planning. In the first stage, given a static-sensor network configuration, robot routes are planned to maximize the amount of spare time available to the mobile agents/sensors, for target search in between (just-in-time) static-sensor deployments. In the second stage, given robot routes (i.e., optimal sequences of sensor delivery locations and times), the corresponding robot trajectories are planned to make effective use of any spare time the mobile agents may have to search for the target. The proposed search strategy was validated through extensive simulations, some of which are given in detail here. An analysis of the method's performance in terms of target-search success is also included.

Time-phased sensor-network deployment refers to the delivery of a set of sensors to their predetermined locations at exact times by a fleet of vehicles. Applications for such network deployments include wilderness search and rescue (WiSAR) and wildfire monitoring, where desirable resource management would imply allowing the vehicles to perform other tasks between deliveries. The goal of this paper is, thus, to formulate and solve a vehicle-routing problem (VRP) for such just-in-time time-phased sensor-network deployments. The proposed optimization method for the modified VRP outlined herein has two primary novelties: 1) the consideration of spare time as the objective function and 2) the use of a targeted local-search (LS) method. The spare-time objective function was formulated to address the uniqueness of the modified routing problem at hand. The targeted LS algorithm, on the other hand, was developed to tangibly improve the efficiency of the search for the optimal values of the chosen objective function. The proposed vehicle-route-planning method was validated via a range of simulated WiSAR scenarios, some of which are included herein. The robustness of the method to variations in problem parameters was also investigated.

This paper discusses the time-phased deployment of wireless sensor networks, applied to surveillance areas growing in time. The focus herein is on the planning of the time-efficient delivery of static sensors to their designated nodes, given a network configuration. The novelty of the proposed strategy is in that it determines optimal delivery plans for spatio-temporally constrained static-sensor networks using multi-robot teams. The proposed sensor delivery planning strategy starts with an already determined (optimal) network plan specified by sensor placement locations (i.e., nodes) and deployment times. Thus, the goal at hand is to determine the optimal routes for the robots delivering the sensors to their intended locations just-in-time. The travel routes are, thus, determined to maximize spare time for the robots between the nodes. The problem is similar to the multiple travelling salesperson problem, but, with temporal constraints. Namely, sensors must be delivered to their designated nodes at designated (optimized) times in order to maintain the optimal deployment of the network configuration. Furthermore, the strategy is designed to be adaptive to new information that can become available during the search for the mobile target, allowing for replanning of the sensor network (i.e., new sensors locations and new deployment times). Numerous simulated experiments were conducted to validate the proposed strategy.

Millirobots have increasingly become popular over the past several years, especially for swarm-behavior studies, allowing researchers to run experiments with a large number of units in limited workspaces. However, as these robots have become smaller in size, their sensory capabilities and battery life have been reduced. A number of these have also been customized, with few off-the shelf components, exhibiting integral (i.e., non-modular) designs. In response to the above concerns, this paper presents a novel open-source millirobot with a modular design based on the use of easily sourced elements and off-the-shelf components. The proposed milli-robot-Toronto (mROBerTO), is a 16×16 mm2robot with a variety of sensors (including proximity, IMU, compass, ambient light, and camera). mROBerTO is capable of formation control using an IR emitter and detector add-on. It can also communicate via Bluetooth Smart, ANT+, or both concurrently. It is equipped with an ARM processor for handling complex tasks and has a flash memory of 256 KB with over-the-air programming capability.