Abstract Motion cue is pivotal in moving object analysis, which is the root for motion segmentation and detection. These preprocessing tasks are building blocks for several applications such as recognition, matching and estimation. To devise a robust algorithm for motion analysis, it is imperative to have a comprehensive dataset to evaluate an algorithm’s performance. The main limitation in making these kind of datasets is the creation of ground-truth annotation of motion, as each moving object might span over multiple frames with changes in size, illumination and angle of view. Besides the optical changes, the object can undergo occlusion by static or moving occluders. The challenge increases when the video is captured by a moving camera. In this paper, we tackle the task of providing ground-truth annotation on motion regions in videos captured from a moving camera. With minimal manual annotation of an object mask, we are able to propagate the label mask in all the frames. Object label correction based on static and moving occluder is also performed by applying occluder mask tracking for a given depth ordering. A motion annotation dataset is also proposed to evaluate algorithm performance. The results show that our cascaded-naive approach provides successful results. All the resources of the annotation tool are publicly available at http://dixie.udg.edu/anntool/ .

Forward-looking sonar can be used for underwater mapping when water visibility is poor. The generation of an acoustic mosaic of the environment is of high interest when underwater vehicles are used for surveys or search tasks. Moreover, if the mosaic is generated in real-time it can be used to provide instantaneous location feedback (e.g. to a ROV pilot or to an AUV), to ensure complete coverage of an area or facilitate the immediate location of targets. In this paper, we present an approach for achieving such a real-time mosaicing capability. Our system considers a simplified imaging model and estimates 2D sonar motions from the pairwise registration of sonar frames. The registration is performed by using a Fourier-based technique, avoiding the extraction of features and ensuring a fast implementation. The mosaicing problem is formulated using a pose graph, with the vertices being the sonar image positions and the edges being constraints from the vehicle odometry and the registration of consecutive and non-consecutive frames. The graph is incrementally optimized using the g2o framework and the optimized poses are then used to build the mosaic online. We apply the method on data gathered on real conditions and show that the resulting sonar mosaic closely matches both the offline generated mosaic as well as ground truth measurements while operating under real-time constraints.

Laser range finder and omnidirectional cameras are becoming a promising combination of sensors to extract rich environmental information. This information includes textured plane extraction, vanishing points, catadioptric projection of vertical and horizontal lines, or invariant image features. However, many indoor scenes do not have enough texture information to describe the environment. In these situations, vertical edges could be used instead. This study presents a sensor model that is able to extract three‐dimensional position of vertical edges from a range‐augmented omnidirectional vision sensor. Using the unified spherical model for central catadioptric sensors and the proposed sensor model, the vertical edges are locally projected, improving the data association for mapping and localisation. The proposed sensor model was tested using the FastSLAM algorithm to solve the simultaneous localisation and mapping problem in indoor environments. Real‐world qualitative and quantitative experiments are presented to validate the proposed approach using a Pioneer‐3DX mobile robot equipped with a URG‐04LX laser range finder and an omnidirectional camera with parabolic mirror.

Dense acquisition for moving scenarios represents an active field of research in Structured Light (SL). A common solution is to project a single one-shot fringe pattern, extracting depth from the phase deviation of the imaged pattern. This implies the use of a phase unwrapping algorithm, which can fail in the presence of depth discontinuities and occlusions. Our work presents a new one-shot dense pattern where De-Bruijn and Windowed Fourier Transform are combined obtaining a dense, absolute, accurate and computationally fast 3D reconstruction regarding the other existing techniques.

Recent work by Mullane, Vo, and Adams has re-examined the probabilistic foundations of feature-based Simultaneous Localization and Mapping (SLAM), casting the problem in terms of filtering with random finite sets. Algorithms were developed based on Probability Hypothesis Density (PHD) filtering techniques that provided superior performance to leading feature-based SLAM algorithms in challenging measurement scenarios with high false alarm rates, high missed detection rates, and high levels of measurement noise. We investigate this approach further by considering a hierarchical point process, or single-cluster multi-object, model, where we consider the state to consist of a map of landmarks conditioned on a vehicle state. Using Finite Set Statistics, we are able to find tractable formulae to approximate the joint vehicle-landmark state based on a single Poisson multi-object assumption on the predicted density. We describe the single-cluster PHD filter and the practical implementation developed based on a particle-system representation of the vehicle state and a Gaussian mixture approximation of the map for each particle. Synthetic simulation results are presented to compare the novel algorithm against the previous PHD filter SLAM algorithm. Results presented indicate a superior performance in vehicle and map landmark localization, and comparable performance in landmark cardinality estimation.

This paper presents a method to build large-scale mosaics adapted to underwater sonar imagery. By assuming a simplified imaging model, we propose to address the registrations between images using Fourier-based methods which, unlike feature-based methods, prove well suited to handle the characteristics of forward-looking sonar images, such as low resolution, noise, occlusions and moving shadows. The registration between spatially and temporally distant images resulting from loop-closing situations or registrations in featureless areas are feasible, overcoming the main difficulties of feature-based methods. The problem is cast as a pose-based graph optimization, taking into account the uncertainties of the pairwise registrations and being able to incorporate navigation information. After the optimization, a consistent mosaic from different tracklines is generated with increased resolution and higher signal-to-noise ratio than the original images, while the vehicle motion in x,y and heading is also estimated.

This paper presents a novel approach to simultaneously compute the motion segmentation and the 3D reconstruction of a set of 2D points extracted from an image sequence. Starting from an initial segmentation, our method proposes an iterative procedure that corrects the misclassified points while reconstructing the 3D scene, which is composed of objects that move independently. This optimization procedure is made by considering two well-known principles: firstly, in multi-body Structure from Motion the matrix describing the 3D shape is sparse, secondly, the segmented 2D points must give a valid 3D reconstruction given the rotational metric constraints. Our formulation results in a bilinear optimization where sparsity and metric constraints are enforced at each iteration of the algorithm. The final result is the corrected segmentation, the 3D structure of the moving objects and an orthographic camera matrix for each motion and each frame. Results are shown on synthetic sequences and a preliminary application on real sequences of the Hopkins 155 database is presented.

Autonomous Underwater Vehicles (AUVs) need positioning systems different than the Global Positioning System (GPS), which does not work in underwater scenarios. A possible solution to this lack of GPS signal are the Simultaneous Localization and Mapping (SLAM) algorithms. SLAM algorithms aim to build a map while simultaneously localize the vehicle within it. These algorithms suffer from several limitations in front of large scale scenarios. For instance, they do not perform consistent maps for large areas, mainly because uncertainties increase with the scenario. In addition, the computational cost increases with the map size. It has been demonstrated that the use of local maps reduces computational cost and improves map consistency. Following this idea, in this paper we propose a new SLAM technique based on using independent local maps, combined with a global level stochastic map. The global level contains the relative transformations between local maps. These local maps are updated once a new loop is detected and the amount of overlapping between local maps is high. Thus, maps sharing a high number of features are updated through fusion, maintaining the correlation between landmarks and vehicle. Experimental results on real data obtained from the REMUS-100 AUV show that our approach is able to obtain large map areas consistently.

In this paper a novel rank estimation technique for trajectories motion segmentation within the Local Subspace Affinity (LSA) framework is presented. This technique, called Enhanced Model Selection (EMS), is based on the relationship between the estimated rank of the trajectory matrix and the affinity matrix built by LSA. The results on synthetic and real data show that without any a priori knowledge, EMS automatically provides an accurate and robust rank estimation, improving the accuracy of the final motion segmentation.

This paper presents a novel technique to align partial 3D reconstructions of the seabed acquired by a stereo camera mounted on an autonomous underwater vehicle. Vehicle localization and seabed mapping is performed simultaneously by means of an Extended Kalman Filter. Passive landmarks are detected on the images and characterized considering 2D and 3D features. Landmarks are re-observed while the robot is navigating and data association becomes easier but robust. Once the survey is completed, vehicle trajectory is smoothed by a Rauch-Tung-Striebel filter obtaining an even better alignment of the 3D views and yet a large-scale acquisition of the seabed.

We propose a full automatic technique to project virtual texture on a real textureless 3D object. Our system is composed of cameras and projector and are used to determine the pose of the object in the real world with the projector as reference and then estimate the image seen by the projector if it would be a camera.

Positioning a robot with respect to objects by using data provided by a camera is a well known technique called visual servoing. In order to perform a task, the object must exhibit visual features which can be extracted from different points of view. Then, visual servoing is object-dependent as it depends on the object appearance. Therefore, performing the positioning task is not possible in presence of non-textured objects or objects for which extracting visual features is too complex or too costly. This paper proposes a solution to tackle this limitation inherent to the current visual servoing techniques. Our proposal is based on the coded structured light approach as a reliable and fast way to solve the correspondence problem. In this case, a coded light pattern is projected providing robust visual features independently of the object appearance

This paper considers the problem of positioning an eye-in-hand system so that it becomes parallel to a planar object. Our approach to this problem is based on linking to the camera a structured light emitter designed to produce a suitable set of visual features. The aim of using structured light is not only for simplifying the image processing and allowing low-textured objects to be considered, but also for producing a control scheme with nice properties like decoupling, convergence, and adequate camera trajectory. This paper focuses on an image-based approach that achieves decoupling in all the workspace, and for which the global convergence is ensured in perfect conditions. The behavior of the image-based approach is shown to be partially equivalent to a 3-D visual servoing scheme, but with a better robustness with respect to image noise. Concerning the robustness of the approach against calibration errors, it is demonstrated both analytically and experimentally

International audience

Omnidirectional cameras offer a much wider field of view than the perspective ones and alleviate the problems due to occlusions. However, both types of cameras suffer from the lack of depth perception. A practical method for obtaining depth in computer vision is to project a known structured light pattern on the scene avoiding the problems and costs involved by stereo vision. This paper is focused on the idea of combining omnidirectional vision and structured light with the aim to provide 3D information about the scene. The resulting sensor is formed by a single catadioptric camera and an omnidirectional light projector. It is also discussed how this sensor can be used in robot navigation applications.

This paper focuses on the problem of realizing a plane-to-plane virtual link between a camera attached to the end-effector of a robot and a planar object. In order to do the system independent to the object surface appearance, a structured light emitter is linked to the camera so that 4 laser pointers are projected onto the object. In a previous paper we showed that such a system has good performance and nice characteristics like partial decoupling near the desired state and robustness against misalignment of the emitter and the camera (J. Pages et al., 2004). However, no analytical results concerning the global asymptotic stability of the system were obtained due to the high complexity of the visual features utilized. In this work we present a better set of visual features which improves the properties of the features in (J. Pages et al., 2004) and for which it is possible to prove the global asymptotic stability.

The registration of full 3-D models is an important task in computer vision. Range finders only reconstruct a partial view of the object. Many authors have proposed several techniques to register 3D surfaces from multiple views in which there are basically two aspects to consider. First, poor registration in which some sort of correspondences are established. Second, accurate registration in order to obtain a better solution. A survey of the most common techniques is presented and includes experimental results of some of them.

In this paper we face the problem of positioning a camera attached to the end-effector of a robotic manipulator so that it gets parallel to a planar object. Such problem has been treated for a long time in visual servoing. Our approach is based on linking to the camera several laser pointers so that its configuration is aimed to produce a suitable set of visual features. The aim of using structured light is not only for easing the image processing and to allow low-textured objects to be treated, but also for producing a control scheme with nice properties like decoupling, stability, well conditioning and good camera trajectory.

This paper presents the implementation details of a coded structured light system for rapid shape acquisition of unknown surfaces. Such techniques are based on the projection of patterns onto a measuring surface and grabbing images of every projection with a camera. Analyzing the pattern deformations that appear in the images, 3D information of the surface can be calculated. The implemented technique projects a unique pattern so that it can be used to measure moving surfaces. The structure of the pattern is a grid where the color of the slits are selected using a De Bruijn sequence. Moreover, since both axis of the pattern are coded, the cross points of the grid have two codewords (which permits to reconstruct them very precisely), while pixels belonging to horizontal and vertical slits have also a codeword. Different sets of colors are used for horizontal and vertical slits, so the resulting pattern is invariant to rotation. Therefore, the alignment constraint between camera and projector considered by a lot of authors is not necessary.

Obtaining automatic 3D profile of objects is one of the most important issues in computer vision. With this information, a large number of applications become feasible: from visual inspection of industrial parts to 3D reconstruction of the environment for mobile robots. In order to achieve 3D data, range finders can be used. Coded structured light approach is one of the most widely used techniques to retrieve 3D information of an unknown surface. An overview of the existing techniques as well as a new classification of patterns for structured light sensors is presented. This kind of systems belong to the group of active triangulation method, which are based on projecting a light pattern and imaging the illuminated scene from one or more points of view. Since the patterns are coded, correspondences between points of the image(s) and points of the projected pattern can be easily found. Once correspondences are found, a classical triangulation strategy between camera(s) and projector device leads to the reconstruction of the surface. Advantages and constraints of the different patterns are discussed.

A large quantity of products are sold by their shape features, hence, process engineers are including more and more accurate 3D digitisers in their manufacturing assembly lines. Data acquisition and data processing speed are becoming important issues to be considered when choosing the right device to be incorporated in the inspection equipment without the loss of accuracy. This paper presents an exhaustive review of the state-of-the-art 3D laser scanning digitisers, and proposes a classification technique of such systems.

Epipolar geometry is a key point in computer vision and the fundamental matrix estimation is the only way to compute it. This article surveys several methods of fundamental matrix estimation which have been classified into linear methods, iterative methods and robust methods. All of these methods have been programmed and their accuracy analysed using real images. A summary, accompanied with experimental results, is given.

The recovery of the three-dimensional structure of the environment is a pre-requisite for many tasks in mobile robotics. Unfortunately, calibration acts as a brake upon visual adaptation and robotic autonomy. We provide tools and constraints based on structured light for a complete and efficient uncalibrated Euclidean reconstruction of the environment. Experimental results achieved both on simulated and real data validate the method.

We present a summary of the most significant techniques, used in the last few years, concerning the coded structured light methods employed to get 3D information. In fact, depth perception is one of the most important subjects in computer vision. Stereovision is an attractive and widely used method, but, rather limited to make 3D surface maps, due to the correspondence problem. The correspondence problem can be improved using a method based on a structured light concept, projecting a given pattern on the measuring surfaces, although some relations between the projected pattern and the reflected one must be solved. This relationship can be directly found codifying the projected light, so that, each imaged region of the projected pattern carries the necessary information to solve the correspondence problem. We do not need to mention the numerous advantages in accurate obtention of 3D information for many research subjects, such as: robotics, autonomous navigation, shape analysis, and so on.

The absolute necessity of obtaining 3D information of structured and unknown environments in autonomous navigation reduce considerably the set of sensors that can be used. The necessity to know, at each time, the position of the mobile robot with respect to the scene is indispensable. Furthermore, this information must be obtained in the least computing time. Stereo vision is an attractive and widely used method, but, it is rather limited to make fast 3D surface maps, due to the correspondence problem. The spatial and temporal correspondence among images can be alleviated using a method based on structured light. This relationship can be directly found codifying the projected light; then each imaged region of the projected pattern carries the needed information to solve the correspondence problem. We present the most significant techniques, used in recent years, concerning the coded structured light method.