Abstract Sound localization is one of the essential tasks in audio-visual learning. Especially, stereo sound localization methods have been proposed to handle multiple sound sources. However, existing stereo-sound localization methods treat sound source localization as a segmentation task and, as a result, require costly annotation of segmentation masks. Another serious problem of the existing stereo-sound localization methods is that they have been trained and evaluated only in a controlled environment, such as a fixed camera and microphone setting with limited variability of scenes. Therefore, their performance on videos recorded in uncontrolled environments, such as in-the-wild videos from the Internet, has not been fully investigated. To address these problems, we propose a weakly supervised method as an extension of a typical stereo-sound localization method by utilizing the spatial relative order of sound sources in recorded videos. The proposed method solves the annotation problem by training the localization model using only sound category labels. Furthermore, our method utilizes the spatial relative order of the sound sources, which is not affected by specific recording settings, and thus can be effectively used for videos recorded in uncontrolled environments. We also collect stereo-recorded videos from YouTube to construct a new dataset to demonstrate the applicability of the proposed method to stereo sounds recorded in various environments. Our method enhances the localization performance by inserting a novel training step that exploits the relative order of sound sources into a typical audio-visual localization method in both existing and newly introduced audio-visual datasets.

In this study, we achieve haptic communication between different structures. In haptic communication, tactile sensation is transmitted to a remote place by cooperating robots. Conventional haptic communication is implemented under the assumption that the masses of the actuators are equal. We have found that haptic communication system is a kind of position/force hybrid control system and that it is not necessary to follow that assumption. In addition, exact modeling of haptic communication system and decoupling of tasks are essential for highly precise haptic communication. First, we describe the kinetic and dynamic behaviors of haptic communication system for cardiac surgery. The deterioration of haptic performance is shown to depend on an interference term, due to off-diagonal parameters in the modal mass matrix. Second, we propose a novel hybrid controller for the decoupling of the responses, and we analyze its performance, stability, and robustness. Simulations and experiments toward cardiac surgery are shown, and the effectiveness of the proposed method is verified.

In this study, we show that tasks can be realized by appropriate coordinate transformation. This approach, oblique coordinate control, decouples tasks. Even though a system is large, they can be designed by a combination of small tasks. Therefore, tasks can be regarded as reusable components. In this study, micro-macro bilateral control is achieved as a complicated task. Small objects can be manipulated as large objects in micro-macro bilateral control. However, ideal micro-macro bilateral controllers are usually derived only for single-degree-of-freedom (DOF) systems. Then, we propose a micro-macro bilateral controller for multi-DOF on the basis of oblique coordinate control. To obtain multi-DOF bilateral control, we first derive control goals as kinematic relations. This study also considers the case where scaling gains of position and force are selected in a different manner. The goals of micro-macro bilateral control are then controlled by oblique coordinate control. The validity of the proposed method is experimentally verified.

In a cel animation workplace, nonphotorealistic rendering techniques are often applied to a 3D scene model, which is constructed by referring to the 2D hand-drawn images of the scene. There exist a lot of scene features that must be modeled from the 2D information. The accentuation effects are amongst the most important features in a hand-drawn scene. In this context, accentuation refers to adding hand-drawn shading of the scene. The specific region that an artist wishes the viewer to observe is drawn brighter than the other regions. A method is proposed for the extraction of the accentuation effects from a single 2D painted image of an animated scene. This method enables the automatic addition of the accentuation effects to each frame of the animation, observed from other viewpoints.