We present a new lightweight CNN-based algorithm for multi-frame optical flow estimation. Our solution introduces a double recurrence over spatial scale and time through repeated use of a generic “STaR” (SpatioTemporal Recurrent) cell. It includes (i) a temporal recurrence based on conveying learned features rather than optical flow estimates; (ii) an occlusion detection process which is coupled with optical flow estimation and therefore uses a very limited number of extra parameters. The resulting STaRFlow algorithm gives state-of-the-art performances on MPI Sintel and Kitti2015 and involves significantly less parameters than all other methods with comparable results.

We explore a new sensor suite to provide a precise and robust navigation information, primarily intended for pedestrian localisation. We use an IMU sensor augmented with an array of magnetometers, called MIMU (for Magneto-Inertial measurement Unit) hereafter, and a single central camera as the vision sensor. The MIMU sensor has been shown in previous work to significantly improve the inertial dead-reckoning for pedestrian, provided we can assume a stationary and non-uniform magnetic environment. Such conditions are encountered in particular in indoor environment, where visual methods can be challenged by textureless or low-light areas. Alternatively, visual methods are useful in outdoor trajectories, where MIMU often fails because of a uniform magnetic field. We build a fusion estimator based on an inverse square-root filter fed with the raw measurement from magnetometers, accelerometers, gyrometers and visual features extracted and tracked in the video. Real-data experiments show the benefits of the fusion : on the one hand, the added information from the MIMU allows to complement the vision/inertial system in case where the vision does not provide useful information during an extended period of time; on the other hand, vision does extend the operational domain of the navigation system compared to a pure MIMU solution, in particular for outdoor sections of the trajectory.

This paper aims to leverage magnetic information from a Magneto-Inertial Measurement Unit - an IMU sensor augmented with an array of magnetometers, called MIMU hereafter - in a vision/inertial navigation system (VINS). This ego-motion estimation problem is formulated as an optimization over a sliding window fusing data from the MIMU with features tracked in a monocular camera image stream. The novelty of our approach lies in the formulation of preintegrated magnetic measurements that are computed from successive measurements of the local variations of the magnetic field, in the line of the preintegration of IMU data introduced in [1]. The resulting magnetic error terms participate to the minimized cost function along with the classical reprojection and IMU error terms. Our experiments show the benefits of this fusion. On the one hand, the added magnetic information from the MIMU allows to complement the VINS in cases where vision does not provide useful information during an extended period of time; on the other hand, vision does extend the operational domain of the navigation system compared to a pure MIMU solution, in particular for the outdoor portions of the trajectory.

We present a novel visual-magneto-inertial system for pedestrian indoor navigation. It includes magnetic, inertial and depth sensors integrated into a device that can be hand held by a pedestrian. Our method builds upon a magneto-inertial tachymeter that is able to accurately reconstruct body speed in presence of magnetic gradient and a depth registration algorithm for computing inter-image movement. We propose a simple fusion strategy using the magneto-inertial tachymeter in the predition step and the depth image registration in the correction step. We demonstrate accurate reconstruction of indoor trajectories in infrastructureless environments, with a drift often below one percent of the length of the trajectory. We emphasize the benefit of the proposed hybrid system in situations where the 3D environment is too simple, making depth registration useless, or situations where the magnetic environment lead to failure of the magneto-inertial tachymetry.

This paper considers passive vision for robotics and focuses on devising a real-time process for moving object detection using a stereo rig. As several previous works, our method relies on the use of dense stereo and of optical flow. Observing that the main computational load of existing methods is related to the estimation of the optical flow, we propose to use a fast algorithm based on Lucas-Kanade's paradigm. We derive a new uncertainty model which explicitly takes into account all errors originating from each estimation step of the process. In contrast with most previous works, we describe a rigorous expansion of the error related to vision based ego-motion estimation. Finally, we present a comparative study of performance on the KITTI dataset, which demonstrates the effectiveness of the proposed approach.

The navigation of a miniature aerial vehicle (MAV) in GPS-denied environments requires a robust embedded visual localization method. In this paper, we describe a simple but efficient stereo visual odometry algorithm, called eVO, running onboard our quadricopter MAV at video-rate. The proposed eVO algorithm relies on a keyframe scheme which allows to decrease the estimation drift and to reduce the computational cost. We study quantitatively the influence of the main parameters of the algorithm and tune them for optimal performance on various datasets. The eVO algorithm has been submitted to the KITTI odometry benchmark [1] where it ranks first at the date of submission, with an average translational drift of 1.93% and an average angular drift of less than 0.076 degres/m. Besides, we have made several experiments with our MAV with egolocalization given by eVO, for instance for autonomous 3D environment modeling.

A new algorithm is proposed to detect small objects by background suppression. It relies on a modified version of the non-local means filter introduced by Buades et al. for background prediction. Background pixels are estimated by a weighted average depending on the similarity between neighborhoods pixels. For background suppression, the similarity criterion is modified to be less sensitive to point object presence. The resulting D-NLM filter better detects dim point objects than current detection filters. A straightforward spatio-temporal extension is also proposed, it has the advantage of not requiring prior accurate background motion compensation. Our method is well adapted to slow object velocity and large background motion contexts, where it outperforms significantly conventional spatio-temporal filters.

We present a new approach for spatially varying blur identification using a single image. Within each local patch in the image, the local blur is selected between a finite set of candidate PSFs by a maximum likelihood approach. We propose to work with a Generalized Likelihood to reduce the number of parameters and we use the Generalized Singular Value De- composition to limit the computing cost, while making proper image boundary hypotheses. The resulting method is fast and demonstrates good performance on simulated and real examples originating from applications such as motion blur identification and depth from defocus.

We consider the recovery of 1-D image features. Such features can be described by a noisy, blurred and undersampled image of a unique 1-D profile. The profile’s recovery is modeled as a 1-D continuous super-resolution (SR) problem. We adopt a functional estimation within a Tikhonov regularization framework. A linear closed-form solution is derived and applied to real data for bar code recovery from low-resolution video frames. Performance modeling in then considered. Thanks to a continuous stochastic model of the input profile, we define a quantitative performance measure which is a mean-square error averaged over a class of profiles with tunable regularity. As a result, an expected SR resolution enhancement ratio is computed, which depends on experimental parameters: SNR, number of input images, sampling rate. A good agreement is found between this theoretical study and empirical performance in experimental SR recovery of bar code profiles.

Robust estimation of the optical flow is addressed through a multiresolution energy minimization. It involves repeated evaluation of spatial and temporal gradients of image intensity which rely usually on bilinear interpolation and image filtering. We propose to base both computations on a single pyramidal cubic B-spline model of image intensity. We show empirically improvements in convergence speed and estimation error and validate the resulting algorithm on real test sequences.

Super-resolution (SR) techniques make use of subpixel shifts between frames in an image sequence to yield higher resolution images. We propose an original observation model devoted to the case of nonisometric inter-frame motion as required, for instance, in the context of airborne imaging sensors. First, we describe how the main observation models used in the SR literature deal with motion, and we explain why they are not suited for nonisometric motion. Then, we propose an extension of the observation model by Elad and Feuer adapted to affine motion. This model is based on a decomposition of affine transforms into successive shear transforms, each one efficiently implemented by row-by-row or column-by-column one-dimensional affine transforms. We demonstrate on synthetic and real sequences that our observation model incorporated in a SR reconstruction technique leads to better results in the case of variable scale motions and it provides equivalent results in the case of isometric motions.

We study dense optical flow estimation using iterative registration of local window, also known as iterative Lucas-Kanade (LK) [B. Lucas et al, 1981]. We show that the usual iterative-warping scheme encounters divergence problems and propose a modified scheme with better behavior. It yields good results with a much lower cost than the exact dense LK algorithm, on simulated and real sequences.

We are concerned with analysis of linear reconstruction-based super-resolution (SR) techniques. We propose a novel Fourier interpretation of an "archetype" linear SR algorithm and derive practical results on the expected performances of such techniques.

The on-orbit calibration of the transfer function (TF) of a spaceborne optical telescope is of paramount importance for the restoration of its recorded images. The authors propose a novel method based on a physical modeling of the TF and on the automatic extraction from the image of sub-images containing appropriate patterns such as step functions. A least-square criterion is computed with all the extracted subimages, and is minimized as a function of the unknowns, which are the TF parameters and the step parameters. The simulation results validate the method and show a precise estimation of the full 2D TF.

Entropy-based methods are widely used for solving inverse problems, particularly when the solution is known to be positive. Here, we address linear ill-posed and noisy inverse problems of the form z=Ax+n with a general convex constraint x/spl isin/X, where X is a convex set. Although projective methods are well adapted to this context, we study alternative methods which rely highly on some "information-based" criteria. Our goal is to clarify the role played by entropy in this field, and to present a new point of view on entropy, using general tools and results coming from convex analysis. We present then a new and broad scheme for entropic-based inversion of linear-noisy inverse problems. This scheme was introduced by Navaza in 1985 in connection with a physical modeling for crystallographic applications, and further studied by Dacunha-Castelle and Gamboa (1990). Important features of this paper are: (i) a unified presentation of many well-known reconstruction criteria, (ii) proposal of new criteria for reconstruction under various prior knowledge and with various noise statistics, (iii) a description of practical inversion of data using the aforementioned criteria, and (iv) a presentation of some reconstruction results.

High resolution digital elevation model (DEM) are used in many applications, including topographic description of the ground (urban, telecommunications...). We consider the derivation of DEMs from two rectified stereoscopic images. This problem can be formulated as a large scale non-convex optimization problem under constraints, where the unknown is a disparity image and the cost function includes sum of correlation coefficients and a regularization term. We propose two optimization schemes: one deterministic and mainly inspired by the graduated non-convexity (GNC) technique proposed by Blake and Zisserman (1987); the other relies on simulated annealing (SA) and can be related to the method proposed by Barnard (1989). We compare the resulting algorithms and show some results on urban area aerial images.

The subject of this communication is the restoration of spiky sequences distorted by a linear system and corrupted by an additive noise. A (now) classical way of coping with this problem is to use a Bayesian approach with a BernoulliGaussian (bg) prior model of the sequence. We will refine this method using a Bernoulli-Gaussian plus Gaussian (bgg) prior model. This estimation method requires maximization of a posterior probability distribution, which cannot be performed optimally. Thus we propose here a new non-Bayesian estimation scheme, derived from the Kullback-Leibler information or cross-entropy. This quite general method, called the Maximum Entropy on the Mean Method (memm) in [1] and [2], is firmly based on convex analysis and yields a unique solution which can be efficiently calculated in practice, and which is, in this sense, truly optimal. As a conclusion, we present results obtained with both methods on a synthetic case. 1. PROBLEM STATEMENT In this communication, we add...

A technique for improving the numerical stability of vector lattice algorithms is presented. It consists of replacing the J-orthogonal transformations used in standard SRF (square-root form) algorithms by orthogonal ones, which provides a better robustness to round-off errors. To illustrate the generality of the approach, explicit modified SRFs of the generalized Levinson algorithm and the Chandrasekhar equations are derived. Simulations confirm the gain in numerical stability, which is obtained at a relatively small extra cost.>