This paper presents a new flying sensor system to capture 3D data aerially. The hardware system, consisting of a omni-directional laser scanner and a panoramic camera, can be mounted under a mobile platform (e.g., a balloon or a crane) to achieve the aerial scanning with high resolution and accuracy. Since the laser scanner often requires several minutes to complete an omni-directional scan, the raw data is distorted seriously due to the unknown and uncontrollable movement during the scanning period. To overcome this problem, 1) we first synchronize the two sensors and spherically calibrate them together, 2) our approach then recovers the sensor motion by utilizing the spacial and temporal features extracted both from the image sequences and point clouds, and 3) finally the distorted scans can be rectified with the estimated motion and aligned together automatically. In experiments, we demonstrate that the method achieves a substantially good performance for indoor/outdoor aerial scanning in the applications such as Angkor Wat 3D preservation and manufacturing 3D survey with respect to other state-of-the-art methods.

Laser range sensors are often demanded to mount on a moving platform for achieving the good efficiency of 3D reconstruction. However, such moving systems often suffer from the difficulty of matching the distorted range scans. In this paper, we propose novel 3D features which can be robustly extracted and matched even for the distorted 3D surface captured by a moving system. Our feature extraction employs Morse theory to construct Morse functions which capture the critical points approximately invariant to the 3D surface distortion. Then for each critical point, we extract support regions with the maximally stable region defined by extremal region or disconnectivity. Our feature description is designed as two steps: 1) we normalize the detected local regions to canonical shapes for robust matching, 2) we encode each key point with multiple vectors at different Morse function values. In experiments, we demonstrate that the proposed 3D features achieve substantially better performance for distorted surface matching than the state-of-the-art methods.

Detecting potential dangers in the environment is a fundamental ability of living beings. In order to endure such ability to a robot, this paper presents an algorithm for detecting potential falling objects, i.e. physically unsafe objects, given an input of 3D point clouds captured by the range sensors. We formulate the falling risk as a probability or a potential that an object may fall given human action or certain natural disturbances, such as earthquake and wind. Our approach differs from traditional object detection paradigm, it first infers hidden and situated “causes (disturbance) of the scene, and then introduces intuitive physical mechanics to predict possible “effects (falls) as consequences of the causes. In particular, we infer a disturbance field by making use of motion capture data as a rich source of common human pose movement. We show that, by applying various disturbance fields, our model achieves a human level recognition rate of potential falling objects on a dataset of challenging and realistic indoor scenes.

In this paper, we present an approach for scene understanding by reasoning physical stability of objects from point cloud. We utilize a simple observation that, by human design, objects in static scenes should be stable with respect to gravity. This assumption is applicable to all scene categories and poses useful constraints for the plausible interpretations (parses) in scene understanding. Our method consists of two major steps: 1) geometric reasoning: recovering solid 3D volumetric primitives from defective point cloud, and 2) physical reasoning: grouping the unstable primitives to physically stable objects by optimizing the stability and the scene prior. We propose to use a novel disconnectivity graph (DG) to represent the energy landscape and use a Swendsen-Wang Cut (MCMC) method for optimization. In experiments, we demonstrate that the algorithm achieves substantially better performance for i) object segmentation, ii) 3D volumetric recovery of the scene, and iii) better parsing result for scene understanding in comparison to state-of-the-art methods in both public dataset and our own new dataset.

We present a novel non-rigid surface registration method that achieves high accuracy and matches characteristic features without manual intervention. The key insight is to consider the entire shape as a collection of local structures that individually undergo rigid transformations to collectively deform the global structure. We realize this locally rigid but globally non-rigid surface registration with a newly derived dual-grid Free-form Deformation (FFD) framework. We first represent the source and target shapes with their signed distance fields (SDF). We then superimpose a sampling grid onto a conventional FFD grid that is dual to the control points. Each control point is then iteratively translated by a rigid transformation that minimizes the difference between two SDFs within the corresponding sampling region. The translated control points then interpolate the embedding space within the FFD grid and determine the overall deformation. The experimental results clearly demonstrate that our method is capable of overcoming the difficulty of preserving and matching local features.

Representing 2D and 3D data sets with implicit polynomials (IPs) has been attractive because of its applicability to various computer vision issues. Therefore, many IP fitting methods have already been proposed. However, the existing fitting methods can be and need to be improved with respect to computational cost for deciding on the appropriate degree of the IP representation and to fitting accuracy, while still maintaining the stability of the fit. We propose a stable method for accurate fitting that automatically determines the moderate degree required. Our method increases the degree of IP until a satisfactory fitting result is obtained. The incrementability of QR decomposition with Gram-Schmidt orthogonalization gives our method computational efficiency. Furthermore, since the decomposition detects the instability element precisely, our method can selectively apply ridge regression-based constraints to that element only. As a result, our method achieves computational stability while maintaining fitting accuracy. Experimental results demonstrate the effectiveness of our method compared with prior methods.