Autonomous systems need to understand the semantics and geometry of their surroundings in order to comprehend and safely execute object-level task specifications. This paper proposes an expressive yet compact model for joint object pose and shape optimization, and an associated optimization algorithm to infer an object-level map from multi-view RGB-D camera observations. The model is expressive because it captures the identities, positions, orientations, and shapes of objects in the environment. It is compact because it relies on a low-dimensional latent representation of implicit object shape, allowing onboard storage of large multi-category object maps. Different from other works that rely on a single object representation format, our approach has a bi-level object model that captures both the coarse level scale as well as the fine level shape details. Our approach is evaluated on the large-scale real-world ScanNet dataset and compared against state-of-the-art methods.

Introducing object-level semantic information into simultaneous localization and mapping (SLAM) system is critical. It not only improves the performance but also enables tasks specified in terms of meaningful objects. This work presents OrcVIO, for visual-inertial odometry tightly coupled with tracking and optimization over structured object models. OrcVIO differentiates through semantic feature and bounding-box reprojection errors to perform batch optimization over the pose and shape of objects. The estimated object states aid in real-time incremental optimization over the IMU-camera states. The ability of OrcVIO for accurate trajectory estimation and large-scale object-level mapping is evaluated using real data.

This paper focuses on building semantic maps, containing object poses and shapes, using a monocular camera. This is an important problem because robots need rich understanding of geometry and context if they are to shape the future of transportation, construction, and agriculture. Our contribution is an instance-specific mesh model of object shape that can be optimized online based on semantic information extracted from camera images. Multi-view constraints on the object shape are obtained by detecting objects and extracting category-specific keypoints and segmentation masks. We show that the errors between projections of the mesh model and the observed keypoints and masks can be differentiated in order to obtain accurate instance-specific object shapes. We evaluate the performance of the proposed approach in simulation and on the KITTI dataset by building maps of car poses and shapes.

We present an approach for two-view motion segmentation for freely moving cameras, by formulating the epipolar constraint and spacial consistency into a discrete energy minimization problem, which can be efficiently solved using graph cut algorithms. With dense optical flow and proper sampling, a set of matched points is acquired for computing the fundamental matrix and the corresponding epipolar lines. The points distance to the epipolar lines, and their position on the image plane are used to construct a Markov Random Field (MRF) with discrete label-space. The 2-dimension label-space, i.e. motion area or static area, is computed using graph cut algorithms. We demonstrate the effectiveness of our method with the Johns Hopkins 155 motion dataset.

Recent developments in robotics sensing using either the active sensor-laser range finder (LRF) or passive sensors-camera systems have shown that the existing approaches are able to estimate the motion and reconstruct the environment in a typical GPS-denied environment such as indoor environments. However, for a 2D LRF, it can only provide a planar measurement due to the hardware limitations and the cost for 3D LRF is still too high for robotic systems. For camera systems, the 3D perception capability and lightweight are promising while it is not effective in long range, low illumination and low textured environment com- pared to the LRF. In the proposed framework, our first contribution is a senor integration system that combines a stereo camera with a rotating LRF. The stereo camera can achieve a fast and smooth motion estimation and the rotating LRF could construct a dense 3D environment. Our second contribution is an integration of a fast feature-based motion estimation with an accurate shape matching refinement. The proposed approaches have been built in our customized unmanned aerial vehicle (UAV) platform. We have verified our proposed approach through a series of extensive evaluations in clustered indoor environments and open outdoor environments.

Recently, visual odometry (VO) has experienced a rapid growth, which makes it viable for a range of applications. This survey paper attempts to provide a timely and comprehensive review of this field, focusing specifically on micro aerial vehicles (MAVs), with monocular, stereo or RGB-D cameras onboard. In this survey, the milestones in the development of VO will be reviewed, followed by an illustration of its general workflow, the commonly used datasets. The survey is concluded by an overall discussion.

Sparse coding (SC) is making a significant impact in computer vision and signal processing communities, which achieves the state-of-the-art performance in a variety of applications for images, e.g., denoising, restoration, and synthesis. We propose an adaptive and robust SC algorithm exploiting the characteristics of typical laser range data and the availability of both range and reflectance data to realize range data denoising and inpainting. Specifically, our method estimates the informative level of each patch according to the variation in both range and reflectance modalities, followed by adaptive dictionary training that assigns dynamic sparsity weights to the patches with different informative levels. Furthermore, the l1-norm-based representation fidelity measure is applied to make our method robust to outliers which are common in laser range measurements. Extensive experiments on synthetic and real data demonstrate that our method works effectively, resulting in superior performance both visually and quantitatively, compared with competitive methods including the available sparse-representation-based algorithm, wavelets, partial differential equation, and non-local means.