The perception of pitch and engagement in melodic singing significantly contribute to the appreciation and comprehension of music. Notably, for hard-of-hearing individuals, perceiving pitch and singing play crucial roles as effective aids in auditory rehabilitation. However, the limited perception of pitch, stemming from the constraints of hearing devices, poses challenges in the learning process of recognizing and vocalizing melodies. In this study, we developed a wearable vibrotactile glove, TTS+, designed to convert musical tones into the corresponding vibrotactile patterns, complemented by an associated training application. After ten training sessions with our system, the participants exhibited notable improvements in pitch perception and the capacity to reproduce melodies through singing compared with their pre-test baseline without vibrotactile feedback. Beyond quantitative outcomes, findings from user experiences obtained through user feedback during training and post-test interviews offer valuable perspectives and constraints for our study, which may be further investigated in future studies.

The pandemic has contributed to the increased digital content development for remote experiences. Notably, museums have begun creating virtual exhibitions using 360-videos, providing a sense of presence and high level of immersion. However, 360-video content often uses a linear timeline interface that requires viewers to follow the path decided by the video creators. This format limits viewers’ ability to actively engage with and explore the virtual space independently. Therefore, we designed a map-based video interface, Viewer2Explorer, that enables the user to perceive and explore virtual spaces autonomously. We then conducted a study to compare the overall experience between the existing linear timeline and map interfaces. Viewer2Explorer enhanced users’ spatial controllability and enabled active exploration in virtual museum exhibition spaces. Additionally, based on our map interface, we discuss a new type of immersion and assisted autonomy that can be experienced through a 360-video interface and provide design insights for future content.

We present ExpanStick, a novel thumbstick that augments video game interaction using the force that the user applies at the physical limit of the gamepad. Gamers frequently manipulate a thumbstick leaning on the limit, or physical boundary around it, either to apply maximum input or to secure physical support, which makes the contact force an easily accessible input channel. The ExpanStick preserves the form factor and usability of the thumbstick, while expanding its input space. We built the ExpanStick and inserted it in a commercial gamepad as a proof-of-concept prototype. Using the prototype, we conducted a design workshop with console gamers to design the ExpanStick game interaction. Through demonstration of the interactions, we found the early use experience with the ExpanStick is positive and well accepted by the game players.

This study aims to clarify the effect of non-speech sound augmentation (i.e., everyday and instrumental sounds) on outdoor play for children, where has been lacking in empirical examination. In a within-subject observational study, sixteen children (ages 10-11) were divided into four equally sized groups and equipped with SoundWear, which is a wearable bracelet that allowed them to explore sounds, pick a desired sound, generate the sound with a swinging movement, and transfer the sound between multiple devices. Both the quantitative and qualitative results revealed that augmenting everyday sounds led to distinct play types with differences in physical, social, and imaginative behaviors, whereas instrumental sounds were naturally integrated into traditional games. Thus, sound augmentation with specific digital design features (e.g., transparent technology to provide new perspectives, margin for interpretation, and ownership through a sense of achievement) is significant for shaping distinctions in digitally enhanced play and requires considerable design attention.

Many museums have begun to use mobile devices to provide interactive content to visitors. However, such devices can also easily distract visitors, drawing their attention away from the exhibits. In this pictorial, we show the steps in designing Scienscope-a hand-held mediation device that can be used as a stethoscope to explore additional information hidden within exhibits at close range. As Scienscope hovers on authentic objects, it provides digital content and audio-haptic feedback, reinforcing the visitors' exploratory behaviors. After interviewing museum experts and bodystorming the museum space, we followed an iterative design process using low-fi prototyping to explore the users' experience. Three application scenarios were devised to show how Scienscope can be used for different types of exhibits. We discussed how visitors respond to and explore hidden information about exhibits at close range, which can be helpful for museum curators and designers.

We present DexController, which is a hand-held controller leveraging grasp as an additional modality for virtual reality (VR) game. The pressure-sensitive surface of DexController was designed to recognize two different grasp-poses (i.e. precision grip and power grip) and detect grasp-force. Based on the results of two feasibility tests, a VR defense game was designed in which players could attack each enemy using the proper weapon with a proper level of force. A within-subject comparative study is conducted with a button-based controller which has the same physical form of DexController. The results indicated that DexController enhanced the perceived naturalness of the controller and game enjoyment, with having acceptable physical demand. This study clarifies the empirical effect of utilizing grasp-recognition on VR game controller to enhance interactivity. Also, we provide insight for the integration of VR game elements with the grasping modality of a controller.

We developed a hand-held controller named DexController, leveraging grasp as an additional input modality for virtual reality(VR) game. The pressure-sensitive surface of DexController could recognize two different grasp-poses (i.e. precision grip and power grip) and detect grasp-force. For demonstration, we designed a VR defense game in which players should attack different virtual enemies using the proper weapon with a proper level of force. User study confirmed that utilizing meaningful information of grasping facilitates natural mapping with game contents, which led VR game users to experience enhanced presence and enjoyment.

Haptic devices allow a more immersive experience with Virtual and Augmented Reality. However, for a wider range of usage they need to be miniaturized while maintaining the quality of haptic feedback. In this study, we used two kinds of human sensory illusion of vibration. The first illusion involves creating a virtual force (pulling sensation) using asymmetric vibration, and the second involves imparting compliances of complex stress-strain curves (i.e. force-displacement curves of mechanical keyboards) to a rigid object by changing the frequency and amplitude of vibration. Using these two illusions, we developed a wearable tactile device named HapCube, consisting of three orthogonal voicecoil actuators. Four measurement tests and four user tests confirmed that 1) a combination of two orthogonal asymmetric vibrations could provide a 2D virtual force in any tangential directions on a finger pad, and 2) a single voicecoil actuator produced pseudo-force feedback of the complex compliance curves in the normal direction.

Despite the advantages of instant messaging (IM) such as simple and fast messaging, IM cannot be considered as effective as face-to-face actual conversation. The current message-by-message interface shows a gap with actual conversation in terms of temporal structure and information dimension levels. Our study investigated the real-time text-displaying interface as a way of shortening this gap. A comparative user study was conducted with three types of IM interfaces to compare and analyze the duration of silence (short pause when nobody types through the IM) quantitatively. Varying chatting experiences in each IM interface were qualitatively measured through a focus group interview. We report the study results regarding the reduced duration of silence, non-verbal expression channel, and new texting techniques found in the real-time text interface. We also discuss design implications for improving the conversing experience for IM interfaces.

Knowing your own body movement is an essential element of sports. Recently, the popularization of smartphones has enabled people to easily record their performance in most situations. However, these observations have limited applicability in assisting with a clear understanding of body movement. In this paper, we propose the Motion Log Skateboard, which records and shares non-visible information about body movement that is difficult to obtain through current observation methods in skateboarding. A pressure-sensor matrix on a skateboard deck is used to record the pressure distribution data, which are then played using the video function of a smartphone camera. With this logged data, a user can access the feet positions, pressure intensity, and timing of the foot movements. To verify the proposed concept and determine the specific context of its use, an experimental session and interviews were conducted with skateboarders of various skill levels. Based on the results of this research, the shared experiences of non-visible information, which is perceived differently depending on the individual, are expected to become a standard for exploring and training body movement.