Identification and localization of sounds are both integral parts of computational auditory scene analysis. Although each can be solved separately, the goal of forming coherent auditory objects and achieving a comprehensive spatial scene understanding suggests pursuing a joint solution of the two problems. This article presents an approach that robustly binds localization with the detection of sound events in a binaural robotic system. Both tasks are joined through the use of spatial stream segregation which produces probabilistic time-frequency masks for individual sources attributable to separate locations, enabling segregated sound event detection operating on these streams. We use simulations of a comprehensive suite of test scenes with multiple co-occurring sound sources, and propose performance measures for systematic investigation of the impact of scene complexity on this segregated detection of sound types. Analyzing the effect of spatial scene arrangement, we show how a robot could facilitate high performance through optimal head rotation. Furthermore, we investigate the performance of segregated detection given possible localization error as well as error in the estimation of number of active sources. Our analysis demonstrates that the proposed approach is an effective method to obtain joint sound event location and type information under a wide range of conditions.

In realistic acoustic scenes, the detection of particular types of environmental sounds is often impeded by the simultaneous presence of multiple sound sources. In this work, we use simulations to systematically investigate the impact of superimposed distractor sources on sound type classification in a binaural robotic system and suggest techniques for increasing the robustness under such conditions. First, we demonstrate that by superimposing target sounds with strongly varying general environmental sounds during training, sound type classifiers are less affected by the presence of a distractor source. Moreover, we show that generalization performance of such models depends on how similar the angular source configuration and the signal-to-noise ratio are to the conditions under which the models were trained. Based on these results, we demonstrate how robust models can be obtained by including a variety of different conditions in the training data, a procedure called multi-conditional training. We evaluate this technique training both with ambient sources as well as with point sources with varying angular configurations, and show that this is an effective approach to produce models with close-to-optimal performance under a wide range of conditions. Moreover, we investigate the impact of head orientation and find that it has a significant influence on classification performance.