The compact explicit finite difference time domain (CE-FDTD) method has been proposed as a non-steady-state analysis method and used for sound field simulation. For sound field simulation, it is necessary to apply the radiation characteristics to reproduce actual sound wave propagation. Radiation characteristics can be applied by setting sound pressure to dense grid arrangement in the CE-FDTD method. However, conventional techniques use a sparse array of microphones and are considered insufficient. Furthermore, the technique of applying captured acoustic signals for dense grid arrangement in the CE-FDTD method has not been considered. In this paper, we propose a hardware and software system that captures the radiation characteristics for a dense grid arrangement and applies them in the CE-FDTD method while controlling the sound wave propagation with non-propagation region. As a result of the proposed method, the average differences of sound pressure, propagation time, nd center frequency from the measured values are 1.8 dB, 0.04 ms, and 200 Hz, respectively, which is more accurate than the conventional techniques. It is shown that this system is useful for improving the accuracy of sound wave propagation reproduction with actual radiation characteristics.

Vehicle navigation systems use both GPS and map data, primarily information derived from map data. Conventional navigation systems assume that the user will look directly at the display to check information. Simultaneously provided text and voice often play only a supplementary role, which can lead to driver distraction and misinterpretation. In contrast, human navigation utilizes visual information, potentially reducing the cognitive load on drivers. Human-like Guidance is aimed at realizing a driving assistance system that supports navigation akin to human guidance. Implementing Human-like Guidance, requires the handling of video footage from in-vehicle cameras during vehicle operation, suggesting the need for an approach combining image recognition and language model. However, images captured during operation often contain superfluous information, making the selection of relevant objects for navigation challenging. Moreover, relying solely on image information makes it difficult to consider the relationship with surrounding objects. Therefore, this study proposes a Spatial-Temporal Scene Graph that can represent spatial and temporal information of objects from driving scene videos. Furthermore, we achieve Human-like Guidance through navigation generation using features extracted from the Spatial-Temporal Scene Graph. Our results show that our proposed method improves the accuracy of navigation generation accuracy compared to traditional image-based navigation methods. In addition, the use of a Spatial-Temporal Scene Graph enables the generation of human-like navigation that focuses on the movements of surrounding vehicle objects.

In this research, we propose a directional sound source representation technique for 3D contents such as volumetric video in metaverse and digital twin. Our proposed technique enables us to have a novel 3D audio-visual experience which is derived from immersive audio presentation expressing the radiation characteristics of sound source. To realize such an experience, we configure the spaced placement of paired microphone array and capture sound source signals completely without obstacles for volumetric video capture. Then, we synthesize the directional sound source signal using our technique which conducts signal processing to capture sound signals based on the positional and directional information of an object relative to a user. We developed and demonstrated a VR application using this technique to evaluate the change of sound with the object or user's movement in accordance with visual rendering. In our user study, we received lots of positive feedback for a novel audio-visual experience.

Lifestyle changes and digitalization have reduced opportunities for face-to-face, intimate communication, which is an indispensable activity for human beings. We have realized a method that allows intimate communication between two persons even though they are faraway from each other by integrating multimodal technologies. Sync Sofa is a new sofa-type communication tool. It senses the partner with a camera, microphones, and accelerometers. The sensed data are cross-modally integrated and then presented to the user through a life-size display, multichannel loudspeakers, and multichannel vibrotactile actuators. We have received positive feedback from many users who have experienced Sync Sofa, such as, "I felt like the other person was really sitting next to me".