This study evaluates software and hardware implementations based on the Cyber-Physical Firewall (CPFW) framework, which provides a flexible and generic access control mechanism for regulating malicious analog signals targeting cyber-physical systems. We describe the CPFW framework design and the implementation strategies for both software and hardware. The software implementation was performed on a Raspberry Pi, whereas the hardware implementation was performed on a Zybo Z7-10 SoC board. We evaluate the characteristics of each implementation, particularly for audio signals, and discuss the differences that arise between software and hardware implementations, along with the selection of an appropriate approach based on specific requirements. The evaluation results demonstrate that the software implementation of the CPFW framework has an overhead of 3.219 ms, whereas the hardware implementation has an overhead of 310 ns, indicating a difference in overhead of \(10^{4}\) . The hardware implementation resulted in only a 5.88% increase in resource utilization owing to the addition of CPFW circuits, indicating that the added circuit size is practical for real-world applications.

In this paper, we develop a novel behavioral biometric recognition framework, BlinkAuth, that takes advantage of a user's blinking. BlinkAuth utilizes electrooculogram (EOG) data, (i.e., the electric potential difference between the corneal and retinal sides of the eye), and applies a machine-learning model to achieve user recognition. BlinkAuth works with devices like smart glasses and VR headsets and can be used simultaneously in activities such as driving or cooking. Using JINS MEME, a glasses-type wearable device that can measure EOG, we collected EOG data from 31 participants under various conditions and evaluated the recognition accuracy of BlinkAuth. The results demonstrate that BlinkAuth can achieve high accuracy as a behavioral biometric recognition with an average AUC of 95.8% and an average EER of 9.28%. We developed a system for implementing BlinkAuth for real-time recognition and evaluated the time required for the recognition process and the system's usability with the System Usability Scale (SUS). The results show an overall processing time of approximately 0.6 seconds, including the data measurement time, and an average SUS score of 82.50, which indicates high usability equivalent to rank A in the standard criteria for interpreting SUS scores. Six extensive user experiments and 17 evaluation perspectives reveal that BlinkAuth is highly robust to environmental changes, such as skin moisture and makeup, participant actions, and eye strain conditions, as well as to attacks that imitate the target's blinking.

This research represents the initial stages of our efforts to establish a digital library run by visually impaired individuals. We use the Action Research (AR) methodology, which involves researchers and community members collaborating to solve real-world problems. Our study focuses on documenting the process of creating a non-profit organization (NPO), building a supportive community, and gathering insights through interviews with key stakeholders. Specifically, the lead author established an NPO registered under Article 37, Paragraph 3 of the Japanese Copyright Act and partnered with the National Diet Library to upload accessible e-books. The NPO expanded its network through local business events, university collaborations, and social entrepreneur communities. Interviews underscored a decline in Braille library volunteers and severe delays in accessibility resources for minority languages like Mongolian. These findings will shape our future efforts to improve accessibility and self-directed learning for visually impaired individuals.

Autonomous systems that rely on object recognition are susceptible to the unique vulnerability of phantom attacks. In these scenarios, adversaries exploit the system by projecting sophisticated deceptive illusions that cause confusion between real objects and their virtual shadows. Despite the growing consensus on the importance of this threat, previous research has lacked comprehensive and quantitative assessments. In an effort to address this research gap, we first methodically investigated the success rates of attacks at various projection distances and angles. Following this baseline assessment, we conducted targeted experiments on two different setups: a black-box approach using the commercial DJI Mavic Air drone with its ActiveTrack feature, and a white-box approach using the open-source Tello drone integrated with YOLOv3 object recognition. These real-world evaluations clearly demonstrated the effectiveness of the phantom attacks. Considering the identified vulnerabilities, we developed DeGhost, a deep learning framework capable of distinguishing real entities from their projected counterparts. To ensure a holistic understanding of its performance, we projected phantoms using different types of projectors onto various surfaces such as concrete, screens, white cloth, white walls, whiteboards, and wooden boards. DeGhost was then evaluated against a range of SoTA object detectors, including the YOLO series, Faster R-CNN, and CenterNet. Our results underscored the ability of DeGhost to detect these phantom attacks with high accuracy, as evidenced by an AUC of 0.998, an FNR of 0.013, and an FPR of 0.018. In addition, the incorporation of an advanced Fourier technique enhanced the robustness of the model. This study not only illuminates the feasibility of the attack but also offers practical security countermeasures for emerging autonomous technologies.

When playing musical instruments, deaf and hard-of-hearing (DHH) people typically sense their music from the vibrations transmitted by the instruments or the movements of their bodies while performing. Sensory substitution devices now exist that convert sounds into light and vibrations to support DHH people’s musical activities. However, these devices require specialized hardware, and the marketing profiles assume that standard musical instruments are available. Hence, a significant gap remains between DHH people and their musical performance enjoyment. To address this issue, this study identifies end users’ preferred gestures when using smartphones to emulate the musical experience based on the instrument selected. This gesture elicitation study applies 10 instrument types. Herein, we present the results and a new taxonomy of musical instrument gestures. The findings will support the design of gesture-based instrument interfaces to enable DHH people to more directly enjoy their musical performances.

This work developed a new security framework named Cyber-Physical Firewall (CPFW), which provides a generic and flexible access control mechanism for regulating the malicious analog signals that target cyber-physical system (CPS) devices. This framework enables the defeat of various attacks that make use of malicious analog signals against CPS devices; e.g., stealth voice command injection attack using ultrasonic waves or adversarial examples, or attacks to crash drones in flight using malicious sound waves.

A voice assistant (VA) is a platform that provides users with a wide range of services via interaction with a voice application using verbal commands. Since the VA application is deployed in the cloud, its behavior is not transparent to the user, which raises privacy concerns. In this study, we developed a framework called ChatterBox, which attempts to analyze VA applications via extensive continuous interaction, to understand their behavior. ChatterBox is capable of parsing and generating dialogues by utilizing natural language processing approach. It can also parse application-level messages to understand how a VA app acquires personal information. ChatterBox supports English and Japanese, which are completely different languages, and can extract more than twice as many dialogues from VA applications compared to SkillExplorer, a state-of-the-art VA dialogue analysis system. Based on analyses of English and Japanese VA applications using ChatterBox, we revealed that 5–15% of VA applications collect personal information or recorded user identifiers in a non-transparent manner, and 76–94% applications collected personal information without providing appropriate privacy policies. In light of these findings, we discuss the implementation of a highly transparent VA application platform.

Deaf and hard of hearing (DHH) people have limited access to auditory input, so they mainly receive visual information during online meetings. In recent years, the usability of a system that visualizes the ongoing topic in a conference has been confirmed, but it has not been verified in a remote conference that includes DHH people. One possible reason is that visual dispersion occurs when there are multiple sources of visual information. In this study, we introduce “Word Cloud for Meeting,” a system that generates a separate word cloud for each participant and displays it in the background of each participant’s video to visualize who is saying what. We conducted an experiment with seven DHH participants and obtained positive qualitative feedback on the ease of recognizing topic changes. However, when the topic changed in a sequence, it was found to be distracting. Additionally, we discuss the design implications for visualizing topics for DHH people in online meetings.

In this study, we conduct the first study on the analysis of voice assistant (VA) apps. We first collect the metadata of VA apps from the VA app directory and analyze them. Next, we call VA apps by the corresponding voice commands and examine how they identify users by analyzing the responses from the apps. We found that roughly half of the VA apps performed user identification by some means. We also found that several apps aim to acquire personal information such as birth date, age, or the blood type through voice conversations. As such data will be stored in the cloud, we need to have a mechanism to ensure that an end-user can check/control the data in a usable way.

We propose a novel attack named "Audio Hotspot Attack'', which performs an inaudible malicious voice command attack, targeting voice assistance systems, e.g., smart speakers or in-car navigation systems. This attack leverages directional sound beams generated from parametric loudspeakers, which emit AM-modulated ultrasounds that will be self-demodulated in the air. It can succeed in the attack on a long distance (2--4 meters in a small room and 10+ meter in a long hallway). To evaluate the feasibility of the attack, we performed extensive in-lab experiments and a user study involving 20 participants. The results demonstrate that the attack is feasible in a real-world setting.