This paper presents an innovative approach to enhancing explainable image retrieval, particularly in situations where a fine-tuning set is unavailable. The widely-used SPatial verification (SP) method, despite its efficacy, relies on a spatial model and the hypothesis-testing strategy for instance recognition, leading to inherent limitations, including the assumption of planar structures and neglect of topological relations among features. To address these shortcomings, we introduce a pioneering technique that replaces the spatial model with a topological one within the RANSAC process. We propose bio-inspired saccade and fovea functions to verify the topological consistency among features, effectively circumventing the issues associated with SP's spatial model. Our experimental results demonstrate that our method significantly outperforms SP, achieving state-of-the-art performance in non-fine-tuning retrieval. Furthermore, our approach can enhance performance when used in conjunction with fine-tuned features. Importantly, our method retains high explainability and is lightweight, offering a practical and adaptable solution for a variety of real-world applications.

Abstract For the realistic and immersive experience in a virtual environment, it is important to estimate and reflect the acoustic characteristic of the real indoor scenes. This article proposes a method of directly measuring the reflection coefficient of a surface, which is an acoustic characteristics in the real environment. Because expensive optimization‐based studies that mainly aim to reproduce recorded sounds indirectly estimate acoustic materials, new estimates are required whenever the actual environment changes. Our approach utilizes the method of the acoustics field to enable anyone to easily and directly measure the reflection coefficient of a real environment and generate sound in a virtual environment. We obtain the impulse response (IR) for the target surface, separate the direct sound and the reflected sound, and calculate the reflection coefficient for each surface. The measurement of the IR and the reflection coefficient calculation takes about 2 s. Our result produces a sound with a similar reverberation time to the sound recorded in the real environment.

In this article, we present a novel localization method for multiple sources in indoor environments. Our approach can estimate different propagation paths, including the reflection and diffraction paths of sound waves based on a backward ray tracing technique. To estimate diffraction propagation paths, we combine a ray tracing algorithm with a uniform theory of diffraction model by exploiting the diffraction properties as propagation paths bend around the wedges of obstacles. We reconstruct the 3-D environments and wedges of obstacles in the precomputation phase and utilize these outcomes to generate primary, reflection, and diffraction acoustic rays in the runtime phase. We localize multiple sources when identifying the convergence regions of these acoustic rays based on Monte Carlo localization (MCL). Our approach supports not only stationary but also moving sources of human speech and clapping sounds. Our approach can also handle nonline-of-sight (NLOS) sources and distinguish between active and inactive source states. We evaluated and analyzed our algorithm in multiple scenarios containing obstacles and NLOS sources. Our approach can localize moving sources with the average of distance errors of 0.65 and 0.74 m in single and multiple source cases, respectively, in rooms, 7 m by 7 m in size with a height of 3 m; errors are measured according to the L2 distance between the estimated and actual source positions. We observed a 130% improvement of the localization accuracy over the prior work (J.-M. Valinet al.).

We present a novel, robust sound source localization algorithm considering back-propagation signals. Sound propagation paths are estimated by generating direct and reflection acoustic rays based on ray tracing in a backward manner. We then compute the back-propagation signals by designing and using the impulse response of the backward sound propagation based on the acoustic ray paths. For identifying the 3D source position, we use a well-established Monte Carlo localization method. Candidates for a source position are determined by identifying convergence regions of acoustic ray paths. Those candidates are validated by measuring similarities between back-propagation signals, under the assumption that the back-propagation signals of different acoustic ray paths should be similar near the ground-truth sound source position. Thanks to considering similarities of back-propagation signals, our approach can localize a source position with an averaged error of 0.55 m in a room of 7 m by 7 m area with 3 m height in tested environments. We also place additional 67 dB and 77 dB white noise at the background, to test the robustness of our approach. Overall, we observe a 7 % to 100 % improvement in accuracy over the state-of-the-art method.

We present a novel sound localization algorithm for a non-line-of-sight (NLOS) sound source in indoor environments. Our approach exploits the diffraction properties of sound waves as they bend around a barrier or an obstacle in the scene. We combine a ray tracing-based sound propagation algorithm with a Uniform Theory of Diffraction (UTD) model, which simulate bending effects by placing a virtual sound source on a wedge in the environment. We precompute the wedges of a reconstructed mesh of an indoor scene and use them to generate diffraction acoustic rays to localize the 3D position of the source. Our method identifies the convergence region of those generated acoustic rays as the estimated source position based on a particle filter. We have evaluated our algorithm in multiple scenarios consisting of static and dynamic NLOS sound sources. In our tested cases, our approach can localize a source position with an average accuracy error of 0.7m, measured by the L2 distance between estimated and actual source locations in a 7m×7m×3m room. Furthermore, we observe 37% to 130% improvement in accuracy over a state-of-the-art localization method that does not model diffraction effects, especially when a sound source is not visible to the robot.

In this paper, we present two novel approaches, super rays and culling region, for efficiently updating grid-based occupancy maps with point clouds. Rays, which traverse from the sensor origin to the sensor data, update the occupancy probabilities of a map representing an environment. Based on the ray model, we define a super ray as a representative ray to multiple rays having the same traversal patterns during the map updates. Our super rays utilize the geometric information of rays and reduce the number of points used for updating the map. For constructing super rays efficiently, we propose mapping lines for handling two- and three-dimensional cases from an observation that edges or grid points branch out the traversal patterns on the map. Furthermore, we introduce a culling region using the occupancy states of the updated map for reducing redundant computations occurred in updates. The super rays perform the update process in a single traversal, and the culling region reduces the number of unnecessary traversals for updating the map. As a result, our combined method improves the update performance without compromising any representation accuracy of a grid-based map. We test the update performance of the proposed method using public indoor and outdoor datasets. Our combined approach shows up to 11.8 times and 2.8 times performance improvement over the state-of-the-art update methods of grid-based maps in the indoor and outdoor scenes, respectively. Also, we compare the update speed and the representation accuracy of our method using the KITTI dataset over the state-of-the-art learning-based occupancy maps. In a navigation scenario that raw point clouds are acquired in 10 Hz, our method shows the best performance on the update speed and thus the highest representation accuracy within a given time.

We present a novel, reflection-aware method for 3D sound localization in indoor environments. Unlike prior approaches, which are mainly based on continuous sound signals from a stationary source, our formulation is designed to localize the position instantaneously from signals within a single frame. We consider direct sound and indirect sound signals that reach the microphones after reflecting off surfaces such as ceilings or walls. We then generate and trace direct and reflected acoustic paths using inverse acoustic ray tracing and utilize these paths with Monte Carlo localization to estimate a 3D sound source position. We have implemented our method on a robot with a cube-shaped microphone array and tested it against different settings with continuous and intermittent sound signals with a stationary or a mobile source. Across different settings, our approach can localize the sound with an average distance error of 0.8 m tested in a room of 7 m by 7 m area with 3 m height, including a mobile and non-line-of-sight sound source. We also reveal that the modeling of indirect rays increases the localization accuracy by 40% compared to only using direct acoustic rays.