Room reconstruction from sound is the problem of estimating indoor space boundaries given only a set of sound events acquired by an array of microphones. In order to find a solution in realistic scenarios, it is necessary a robust and practical method that can solve the echo labelling problem, i.e. assigning at each signal delay the correct reflector that has generated it. Although being an NP-hard problem, in this paper we demonstrate that it is possible to solve the echo labelling problem in a reasonable computational time without the need of additional hypotheses on the echoes order of arrival.

In this work we present a novel approach to recover objects 3D position and occupancy in a generic scene using only 2D object detections from multiple view images. The method reformulates the problem as the estimation of a quadric (ellipsoid) in 3D given a set of 2D ellipses fitted to the object detection bounding boxes in multiple views. We show that a closed-form solution exists in the dual-space using a minimum of three views while a solution with two views is possible through the use of non-linear optimisation and object constraints on the size of the object shape. In order to make the solution robust toward inaccurate bounding boxes, a likely occurrence in object detection methods, we introduce a data preconditioning technique and a non-linear refinement of the closed form solution based on implicit subspace constraints. Results on synthetic tests and on different real datasets, involving challenging scenarios, demonstrate the applicability and potential of our method in several realistic scenarios.

A novel method is proposed for generic target tracking by audio measurements from a microphone array. To cope with noisy environments characterized by persistent and high energy interfering sources, a classification map (CM) based on spectral signatures is calculated by means of a machine learning algorithm. Next, the CM is combined with the acoustic map, describing the spatial distribution of sound energy, in order to obtain a cleaned joint map in which contributions from the disturbing sources are removed. A likelihood function is derived from this map and fed to a particle filter yielding the target location estimation on the acoustic image. The method is tested on two real environments, addressing both speaker and vehicle tracking. The comparison with a couple of trackers, relying on the acoustic map only, shows a sharp improvement in performance, paving the way to the application of audio tracking in real challenging environments.

This paper shows for the first time that is possible to reconstruct the position of rigid objects and to jointly recover affine camera calibration solely from a set of object detections in a video sequence. In practice, this work can be considered as the extension of Tomasi and Kanade factorization method using objects. Instead of using points to form a rank constrained measurement matrix, we can form a matrix with similar rank properties using 2D object detection proposals. In detail, we first fit an ellipse onto the image plane at each bounding box as given by the object detector. The collection of all the ellipses in the dual space is used to create a measurement matrix that gives a specific rank constraint. This matrix can be factorised and metrically upgraded in order to provide the affine camera matrices and the 3D position of the objects as an ellipsoid. Moreover, we recover the full 3D quadric thus giving additional information about object occupancy and 3D pose. Finally, we also show that 2D points measurements can be seamlessly included in the framework to reduce the number of objects required. This last aspect unifies the classical point-based Tomasi and Kanade approach with objects in a unique framework. Experiments with synthetic and real data show the feasibility of our approach for the affine camera case.

Estimation of Time Difference of Arrivals (TDOAs) corresponding to early room reflections can be formulated as a Blind Channel Identification (BCI) problem, exploiting signals acquired by a set of microphones, given an unknown transmitted source. To cope with the intrinsic noise sensitivity and ill-posedness of the problem, sparsity and non-negativity priors on the Acoustic Impulse Response (AIR) of the room can be exploited. Here we propose a novel iterative method, resulting in a sequence of convex problems relying on a weighted l1constraint. The proposed method allows to outperform current state of the art on speech and non-speech real signals, while lowering the number of parameters to tune and the sensitivity of solution given the few free parameters.

This paper presents a method to deal with the multi-body segmentation problem using a set of 2D points matches between two views. The key feature of our approach is the explicit inclusion of a higher semantic information as given by general purpose object detectors that boost the segmentation of the moving objects. In the classical formulation of the problem, only 2D matched points between views are used to identify independently moving objects based on the principle that a set of points belonging to a moving object would satisfy some given multi-view relations (e.g. multi-body epipolar constraints). We improve and speedup such process by including the information that a set of 2D matches may belong to the same object given the output of a detector. As such, instead of sampling points uniformly with a RANSAC based strategy, the selection of the matches is driven by the position and score confidence of the object detectors. Evaluation on challenging synthetic and real datasets shows a remarkable improvement in respect to previous approaches, regarding both the number of iterations required to segment a scene and the effectiveness of the segmentation itself, often making the difference between satisfying segmentation and almost complete failure.

In computer vision, an object can be modeled in two main ways: by explicitly measuring its characteristics in terms of feature vectors, and by capturing the relations which link an object with some exemplars, that is, in terms of similarities. In this paper, we propose a new similarity-based descriptor, dubbed structural similarity cross-covariance tensor (SS-CCT), where self-similarities come into play: Here the entity to be measured and the exemplar are regions of the same object, and their similarities are encoded in terms of cross-covariance matrices. These matrices are computed from a set of low-level feature vectors extracted from pairs of regions that cover the entire image. SS-CCT shares some similarities with the widely used covariance matrix descriptor, but extends its power focusing on structural similarities across multiple parts of an image, instead of capturing local similarities in a single region. The effectiveness of SS-CCT is tested on many diverse classification scenarios, considering objects and scenes on widely known benchmarks (Caltech-101, Caltech-256, PASCAL VOC 2007 and SenseCam). In all the cases, the results obtained demonstrate the superiority of our new descriptor against diverse competitors. Furthermore, we also reported an analysis on the reduced computational burden achieved by using and efficient implementation that takes advantage from the integral image representation.

Planar arrays are used jointly with filter-and-sum beamforming to achieve 3-D spatial discrimination in processing broadband signals. In these systems, the beams are steered in various directions to investigate a given portion of space. The band can be so wide as to require both superdirective performance (to increase directivity at low frequencies) and sparse aperiodic layouts (to avoid grating lobes at high frequencies). We propose an original method to simultaneously optimize the transducer positions and the coefficients of the Finite Impulse Response FIR filters, providing a solution that maintains its validity for whatever steering direction inside a predefined region of interest. A hybrid strategy, analytical for the coefficients and stochastic for the positions, is devised to minimize the beam pattern (BP) energy while maintaining an unaltered signal from the steering direction and controlling the side lobes. The robustness of the superdirectivity is achieved by taking into account the probability density functions for the characteristics of realistic transducers. A distinctive feature of our method is its ability to maintain the computational tractability of the addressed optimization problem by drastically reducing the burden of evaluating the cost function. The obtained results, addressing tens of transducers and several octaves of band, demonstrate the effectiveness of the proposed method in terms of directivity, contrast, and robustness.

Capturing the essential characteristics of visual objects by considering how their features are inter-related is a recent philosophy of object classification. In this paper, we embed this principle in a novel image descriptor, dubbed Heterogeneous Auto-Similarities of Characteristics (HASC). HASC is applied to heterogeneous dense features maps, encoding linear relations by co variances and nonlinear associations through information-theoretic measures such as mutual information and entropy. In this way, highly complex structural information can be expressed in a compact, scale invariant and robust manner. The effectiveness of HASC is tested on many diverse detection and classification scenarios, considering objects, textures and pedestrians, on widely known benchmarks (Caltech-101, Brodatz, Daimler Multi-Cue). In all the cases, the results obtained with standard classifiers demonstrate the superiority of HASC with respect to the most adopted local feature descriptors nowadays, such as SIFT, HOG, LBP and feature co variances. In addition, HASC sets the state-of-the-art on the Brodatz texture dataset and the Daimler Multi-Cue pedestrian dataset, without exploiting ad-hoc sophisticated classifiers.

We present a novel closed-form solution for the joint self-calibration of video and range sensors. The approach single assumption is the availability of synchronous time of flight (i.e., range distances) measurements and visual position of the target on images acquired by a set of cameras. In such case, we make explicit a rank constraint that is valid for both image and range data. This rank property is used to find an initial and affine solution via bilinear factorization, which is then corrected by enforcing the metric constraints characteristic for both sensor modalities (i.e., camera and anchors constraints). The output of the algorithm is the identification of the target/range sensor position and the calibration of the cameras. The application extent of our approach is broad and versatile. In fact, with the same framework, we can deal with, but not restricted to, two very different applications. The first is aimed at calibrating cameras and microphones deployed in an unknown environment. The second uses a RGB-D device to reconstruct the 3D position of a set of keypoints using the camera and depth map images. Synthetic and real tests show the algorithm performance under different levels of noise and configurations of target locations, number of sensors and cameras.

The estimation of the contrast agent concentration is important for the introduction of effective quantification in ultrasound medical diagnoses. This paper focuses on the comparison of three methods for estimating the number of microbubbles in a small volume. An energy-based linear regression approach and a nonlinear regression approach are applied to estimate the microbubble concentration working from the signals backscattered by a contrast agent population. The nonlinear regression approach, based on the support vector machine, is employed assuming either only one feature, the signal energy, or a set of significant features. Synthetic data are used for the training phase and the test of the methods. The results demonstrate that, although the energy-based methods allows to achieve good results in terms of mean estimated values, the adoption of a vector of features significantly reduces the mean square error. The nonlinear regression approach with multidimensional feature vector allows to improve the measurement precision of the bubble concentration.

This paper presents a novel algorithm for the automatic 3D localization of a set of microphones in an unknown environment. Given the times of arrival at each microphone of a set of sound events, the approach simultaneously estimates the 3D positions of the sensors and the sources that have generated the events. The only assumption made is that the emission time of the sound events must be known in order to measure the time of flight for each event. A closed form solution is also proposed whenever a sound event coincides with a microphone position. Simulated and real experiments show the validity of the approach for different setups of sensors and number of events.

This paper addresses the spatial processing of signals collected by a linear array of sensors that feeds a filter-and-sum, data-independent beamformer. When the frequency band spanned by the signals to be processed is extremely wide, a given array can be shorter than the wavelength (at the lowest frequencies) and, at the same time, too scarcely populated for a correct sampling of the wavefield (at the highest frequencies). Superdirectivity and aperiodic sparse layouts are possible solutions to these two problems. However, these two solutions have never been considered jointly to achieve a broadband beam pattern with a desired profile. Through a mixed stochastic and analytic optimization, a method is proposed herein that synthesizes a sparse array layout and the tap coefficients of the beamformer filters to provide a broadband beam pattern that is superdirective and robust against the fluctuation of the sensors' characteristics, which is free from any grating lobes and possesses a controlled side-lobe level. Different types of beam patterns, from the frequency-invariant pattern to the maximum-directivity pattern, can be obtained and the synthesized solutions retain their validity for any steering direction inside a given interval. The functioning of the method is proven by considering a microphone array with four different design targets and by discussing the performance and the robustness of the synthesized solutions.