Conventional camera calibration techniques rely on discrete reference points extracted from a set of input images. While these approaches have been applied successfully for a long time, omitting all image information apart from reference point positions at the initial stage of the calibration pipeline renders correct treatment of uncertainties difficult and gives rise to complications in timestamping measurements in applications where exposure time cannot be neglected. Drawing inspiration from visual state estimation, we employ a direct formulation of the camera measurement model. To this end, we render a view of the target given all calibration parameters, enabling a maximum likelihood estimator formulated on image intensities as measurements. We demonstrate the advantages of avoiding abstraction from image measurements for determining the line delay of a rolling shutter camera and by estimating camera exposure time from motion blur.

As the applications of Micro Aerial Vehicles (MAVs) get more and more complex, and require highly dynamic motions, it becomes essential to have an accurate dynamic model of the MAV. Such a model can be used for reliable state estimation, control, and for realistic simulation. A good model requires accurate estimates of physical parameters of the system, which we aim to estimate from recorded flight data. In this paper, we present a detailed physical model of the MAV and a maximum likelihood estimation scheme for determining the dominant parameters, such as inertia matrix, center of gravity (CoG) with respect to the IMU, and parameters related to the aerodynamics. To incorporate all information given by the IMU and the physical MAV model, we propose to use two process models in the optimization. We show the effectiveness of the method on simulated data, as well as on a real platform.

An increasing number of robotic systems feature multiple inertial measurement units (IMUs). Due to competing objectives-either desired vicinity to the center of gravity when used in controls, or an unobstructed field of view when integrated in a sensor setup with an exteroceptive sensor for ego-motion estimation-individual IMUs are often mounted at considerable distance. As a result, they sense different accelerations when the platform is subjected to rotational motions. In this work, we derive a method for spatially calibrating multiple IMUs in a single estimator based on the open-source camera/IMU calibration toolbox kalibr. We further extend the toolbox to determine IMU intrinsics, enabling accurate calibration of low-cost IMUs. The results suggest that the extended estimator is capable of precisely determining these intrinsics and even of localizing individual accelerometer axes inside a commercial grade IMU to millimeter precision.

Robust, accurate pose estimation and mapping at real-time in six dimensions is a primary need of mobile robots, in particular flying Micro Aerial Vehicles (MAVs), which still perform their impressive maneuvers mostly in controlled environments. This work presents a visual-inertial sensor unit aimed at effortless deployment on robots in order to equip them with robust real-time Simultaneous Localization and Mapping (SLAM) capabilities, and to facilitate research on this important topic at a low entry barrier. Up to four cameras are interfaced through a modern ARM-FPGA system, along with an Inertial Measurement Unit (IMU) providing high-quality rate gyro and accelerometer measurements, calibrated and hardware-synchronized with the images. This facilitates a tight fusion of visual and inertial cues that leads to a level of robustness and accuracy which is difficult to achieve with purely visual SLAM systems. In addition to raw data, the sensor head provides FPGA-pre-processed data such as visual keypoints, reducing the computational complexity of SLAM algorithms significantly and enabling employment on resource-constrained platforms. Sensor selection, hardware and firmware design, as well as intrinsic and extrinsic calibration are addressed in this work. Results from a tightly coupled reference visual-inertial motion estimation framework demonstrate the capabilities of the presented system.

This video presents experiments conducted within the final review meeting demonstration session of the AIRobots project. AIRobots started at 2010 and the final review meeting took place on 22 of March, 2013. The presented experiments cover a wide area of the challenges related with aerial industrial inspection. In particular, multiple test-cases related with both vision-based and contact-based inspection and in general physical interaction are shown. It is highlighted that these experiments were recorded live during the project demonstration and evaluation process.

In this paper, we present a framework for 6D absolute scale motion and structure estimation of a multi-camera system in challenging indoor environments. It operates in real-time and employs information from two cameras with non-overlapping fields of view. Monocular Visual Odometry supplying up-to-scale 6D motion information is carried out in each of the cameras, and the metric scale is recovered via a linear solution by imposing the known static transformation between both sensors. The redundancy in the motion estimates is finally exploited by a statistical fusion to an optimal 6D metric result. The proposed technique is robust to outliers and able to continuously deliver a reasonable measurement of the scale factor. The quality of the framework is demonstrated by a concise evaluation on indoor datasets, including a comparison to accurate ground truth data provided by an external motion tracking system.

Hearing is amongst the most important senses a modern robot must exhibit. Perceiving the acoustic world enables capabilities such as natural interaction with humans, interpreting spoken commands or the localization of victims during search and rescue tasks. Real-world robotic operations often take place in noisy, reverberant environments while requiring features such as source separation, accurate direction of arrival estimation or high performance noise suppression. This work presents a methodology to design, calibrate and operate large microphone arrays that enable such features. Recent micro electro-mechanical microphones in conjunction with reconfigurable logic tackle the weight, size, power consumption and cost constraints of robotic systems. A novel, automatic array shape calibration algorithm is developed for 2D and 3D arrays to face common experimental problems such as reverberation and poor signal-to-noise ratio when calibrating the array. The special case of a 2D array calibrated using sources moving in 3D is addressed. No prior information on array geometry is required, the process is fully automated and does not require any specific calibration equipment. The example application of an acoustic camera is presented as a proof of concept. High-quality acoustic images are computed in real-time by generalized inverse beamforming. This demonstrates the effectiveness of the proposed design and illustrates the usefulness of such sensing capabilities for various robotic applications.

In this paper, we consider power spectral density estimation of bandlimited, wide-sense stationary signals from sub-Nyquist sampled data. This problem has recently received attention from within the emerging field of cognitive radio for example, and solutions have been proposed that use ideas from compressed sensing and the theory of digital alias-free signal processing. Here we develop a compressed sensing based technique that employs multi-coset sampling and produces multi-resolution power spectral estimates at arbitrarily low average sampling rates. The technique applies to spectrally sparse and nonsparse signals alike, but we show that when the wide-sense stationary signal is spectrally sparse, compressed sensing is able to enhance the estimator. The estimator does not require signal reconstruction and can be directly obtained from a straightforward application of nonnegative least squares.

This work presents a method for estimating the egomotion of an aerial vehicle in challenging industrial environments. It combines binocular visual and inertial cues in a tightly-coupled fashion and operates in real time on an embedded platform. An extended Kalman filter fuses measurements and makes motion estimation rely more on inertial data if visual feature constellation is degenerate. Errors in roll and pitch are bounded implicitly by the gravity vector. Inertial sensors are used for efficient outlier detection and enable operation in poorly and repetitively textured environments. We demonstrate robustness and accuracy in an industrial scenario as well as in general indoor environments. The former is accompanied by a detailed performance evaluation supported with ground truth measurements from an external tracking system.