We present a new method for the application of image-based river velocimetry. Our method is dedicated to high velocity river flows and it is based on an optical flow scheme. Unlike traditional Particle Image Velocimetry (PIV) methods, optical flow provides dense estimations and it is highly adaptable to different applications. We therefore propose a new physics-based optical flow derived from scalar transport equation and augmented with a weighted diffusion term. The results are evaluated on a Lagrangian basis by means of trajectories reconstruction. The trajectories of few passive particles of interest are reconstructed, based on the successive estimated vector fields. Our method outperformed traditional PIV-based and original optical flow methods.

Based on physical laws describing the multiscale structure of turbulent flows, this paper proposes a regularizer for fluid motion estimation from an image sequence. Regularization is achieved by imposing some scale invariance property between histograms of motion increments computed at different scales. By reformulating this problem from a Bayesian perspective, an algorithm is proposed to jointly estimate motion, regularization hyperparameters, and to select the most likely physical prior among a set of models. Hyperparameter and model inference are conducted by posterior maximization, obtained by marginalizing out non--Gaussian motion variables. The Bayesian estimator is assessed on several image sequences depicting synthetic and real turbulent fluid flows. Results obtained with the proposed approach exceed the state-of-the-art results in fluid flow estimation.

In this paper, a stochastic formulation of the brightness consistency used in many computer vision problems involving dynamic scenes (for instance, motion estimation or point tracking) is proposed. Usually, this model, which assumes that the luminance of a point is constant along its trajectory, is expressed in a differential form through the total derivative of the luminance function. This differential equation linearly links the point velocity to the spatial and temporal gradients of the luminance function. However, when dealing with images, the available information only holds at discrete time and on a discrete grid. In this paper, we formalize the image luminance as a continuous function transported by a flow known only up to some uncertainties related to such a discretization process. Relying on stochastic calculus, we define a formulation of the luminance function preservation in which these uncertainties are taken into account. From such a framework, it can be shown that the usual deterministic optical flow constraint equation corresponds to our stochastic evolution under some strong constraints. These constraints can be relaxed by imposing a weaker temporal assumption on the luminance function and also in introducing anisotropic intensity-based uncertainties. We also show that these uncertainties can be computed at each point of the image grid from the image data and hence provide meaningful information on the reliability of the motion estimates. To demonstrate the benefit of such a stochastic formulation of the brightness consistency assumption, we have considered a local least-squares motion estimator relying on this new constraint. This new motion estimator significantly improves the quality of the results.

Selecting optimal models and hyperparameters is crucial for accurate optical-flow estimation. This paper provides a solution to the problem in a generic Bayesian framework. The method is based on a conditional model linking the image intensity function, the unknown velocity field, hyperparameters, and the prior and likelihood motion models. Inference is performed on each of the three levels of this so-defined hierarchical model by maximization of marginalized a posteriori probability distribution functions. In particular, the first level is used to achieve motion estimation in a classical a posteriori scheme. By marginalizing out the motion variable, the second level enables to infer regularization coefficients and hyperparameters of non-Gaussian M-estimators commonly used in robust statistics. The last level of the hierarchy is used for selection of the likelihood and prior motion models conditioned to the image data. The method is evaluated on image sequences of fluid flows and from the "Middlebury" database. Experiments prove that applying the proposed inference strategy yields better results than manually tuning smoothing parameters or discontinuity preserving cost functions of the state-of-the-art methods.

This paper presents a novel, efficient scheme for the analysis of Sea Surface Temperature (SST) ocean images. We consider the estimation of the velocity fields and vorticity values from a sequence of oceanic images. The contribution of this paper lies in proposing a novel, robust and simple approach based on Weighted Ensemble Transform Kalman filter (WETKF) data assimilation technique for the analysis of real SST images, that may contain coast regions or large areas of missing data due to the cloud cover.

This paper focuses on the tracking of free curves using non-linear stochastic filtering techniques. It relies on a particle filter which includes color measurements. The curve and its velocity are defined through two coupled implicit level set representations. The stochastic dynamics of the curve is expressed directly on the level set function associated to the curve representation and combines a velocity field captured from the additional second level set attached to the past curve's points location. The curve's dynamics combines a low-dimensional noise model and a data-driven local force. We demonstrate how this approach allows the tracking of highly and rapidly deforming objects, such as convective cells in infra-red satellite images, while providing a location-dependent assessment of the estimation confidence.

This paper focuses on the tracking and analysis of convective cloud systems from Meteosat Second Generation images. The highly deformable nature of convective clouds, the complexity of the physical processes involved, and also the partially hidden measurements available from image data make difficult the direct use of conventional image-analysis techniques for tasks of detection, tracking, and characterization. In this paper, we face these issues using variational-data-assimilation tools. Such techniques enable us to perform the estimation of an unknown state function according to a given dynamical model and to noisy and incomplete measurements. The system state we are setting in this study for the cloud representation is composed of two nested curves corresponding to the exterior frontiers of the clouds and to the interior coldest parts (core) of the convective clouds. Since no reliable simple dynamical model exists for such phenomena at the image grid scale, the dynamics on which we are relying has been directly defined from image-based motion measurements and takes into account an uncertainty modeling of the curve dynamics along time. In addition to this assimilation technique, we show in the Appendix how each cell of the recovered cloud system can be labeled and associated to characteristic parameters (birth or death time, mean temperature, velocity, growth, etc.) of great interest for meteorologists.

Based on scaling laws describing the statistical structure of turbulent motion across scales, we propose a multiscale and non-parametric regularizer for optic-flow estimation. Regularization is achieved by constraining motion increments to behave through scales as the most likely self-similar process given some image data. In a first level of inference, the hard constrained minimization problem is optimally solved by taking advantage of lagrangian duality. It results in a collection of first-order regularizers acting at different scales. This estimation is non-parametric since the optimal regularization parameters at the different scales are obtained by solving the dual problem. In a second level of inference, the most likely self-similar model given the data is optimally selected by maximization of Bayesian evidence. The motion estimator accuracy is first evaluated on a synthetic image sequence of simulated bi-dimensional turbulence and then on a real meteorological image sequence. Results obtained with the proposed physical based approach exceeds the best state of the art results. Furthermore, selecting from images the most evident multiscale motion model enables the recovery of physical quantities, which are of major interest for turbulence characterization.

In this paper, Bayesian inference is used to select the most evident Gibbs prior model for motion estimation given some image sequence. The proposed method supplements the maximuma posteriorimotion estimation scheme proposed in He¿as et al. (2008). Indeed, in this recent work, the authors have introduced a family of multiscale spatial priors in order to cure the ill-posed inverse motion estimation problem. We propose here a second level of inference where the most likely prior model is optimally chosen given the data by maximization of Bayesian evidence. Model selection and motion estimation are assessed on Meteorological Second Generation (MSG) image sequences. Selecting from images the most evident multiscale model enables the recovery of physical quantities which are of major interest for atmospheric turbulence characterization.

This paper is interested with the tracking and the analysis of convective cells on Meteosat Second Generation images. Due to the highly deformable nature of convective clouds, the noisy measurements obtained with image data and the complexity of the physics that governs such phenomena, the conventional tools issued from computer vision to detect and track usual (rigid) objects are not well adapt. In this paper, we face this problem using variational data assimilation tools. Such techniques enable to perform the estimation of an unknown state function according to a given dynamical model and to noisy and incomplete measurements. The system state is composed of two curves (represented with implicit surfaces) corresponding to the the whole cloud and the coldest part (heart) of the convective system. Since no precise and manageable dynamical model exists concerning such phenomena, the dynamics is directly measured from images using some motion estimation techniques devoted to fluid motions.

In this paper, we present a method for the temporal tracking of fluid flow velocity fields. The technique we propose is formalized within a sequential Bayesian filtering framework. The filtering model combines an Itô diffusion process coming from a stochastic formulation of the vorticity-velocity form of the Navier-Stokes equation and discrete measurements extracted from the image sequence. In order to handle a state space of reasonable dimension, the motion field is represented as a combination of adapted basis functions, derived from a discretization of the vorticity map of the fluid flow velocity field. The resulting nonlinear filtering problem is solved with the particle filter algorithm in continuous time. An adaptive dimensional reduction method is applied to the filtering technique, relying on dynamical systems theory. The efficiency of the tracking method is demonstrated on synthetic and real-world sequences.

This paper proposes an hybrid approach to estimate the 3D pose of an object. The integration of texture information based on image intensities in a more classical non-linear edge-based pose estimation computation has proven to highly increase the reliability of the tracker. We propose in this work to exploit the data provided by an optical flow algorithm for a similar purpose. The advantage of using the optical flow is that it does not require any a priori knowledge on the object appearance. The registration of 2D and 3D cues for monocular tracking is performed by a non linear minimization. Results obtained show that using optical flow enables to perform robust 3D hybrid tracking even without any texture model.

The complexity of dynamical laws governing 3D atmospheric flows associated with incomplete and noisy observations make the recovery of atmospheric dynamics from satellite images sequences very difficult. In this paper, we face the challenging problem of estimating physical sound and time-consistent horizontal motion fields at various atmospheric depths for a whole image sequence. Based on a vertical decomposition of the atmosphere, we propose a dynamically consistent atmospheric motion estimator relying on a multi-layer dynamical model. This estimator is based on a weak constraint variational data assimilation scheme and is applied on noisy and incomplete pressure difference observations derived from satellite images. The dynamical model consists in a simplified vorticity-divergence form of a multi-layer shallow-water model. Average horizontal motion fields are estimated for each layer. The performance of the proposed technique is assessed on real world meteorological satellite image sequences.

In this paper, a variational technique derived from optimal control theory is used in order to realize a dynamically consistent motion estimation of a whole fluid image sequence. The estimation is conducted through an iterative process involving a forward integration of a given dynamical model followed by a backward integration of an adjoint evolution law. By combining physical conservation laws and image observations, a physically grounded temporal consistency is imposed, and the quality of the motion estimation is significantly improved. The method is validated on two synthetic image sequences provided by numerical simulation of fluid flows and on real world meteorological examples.

In this paper, we address the problem of estimating 3-D motions of a stratified atmosphere from satellite image sequences. The analysis of 3-D atmospheric fluid flows associated with incomplete observation of atmospheric layers due to the sparsity of cloud systems is very difficult. This makes the estimation of dense atmospheric motion field from satellite image sequences very difficult. The recovery of the vertical component of fluid motion from a monocular sequence of image observations is a very challenging problem for which no solution exists in the literature. Based on a physically sound vertical decomposition of the atmosphere into cloud layers of different altitudes, we propose here a dense motion estimator dedicated to the extraction of 3-D wind fields characterizing the dynamics of a layered atmosphere. Wind estimation is performed over the complete 3-D space, using a multilayer model describing a stack of dynamic horizontal layers of evolving thickness, interacting at their boundaries via vertical winds. The efficiency of our approach is demonstrated on synthetic and real sequences.

In this paper, we present a framework for dynamic consistent estimation of dense motion fields over a sequence of images. The originality of the approach is to exploit recipes related to optimal control theory. This setup allows performing the estimation of an unknown state function according to a given dynamical model and to noisy and incomplete measurements. The overall process is formalized through the minimization of a global spatio-temporal cost functional w.r.t the complete sequence of motion fields. The minimization is handled considering an adjoint formulation. The resulting scheme consists in iterating a forward integration of the evolution model and a backward integration of the adjoint evolution model guided by a discrepancy measurement between the state variable and the available noisy observations. Such an approach allows us to cope with several delicate situations (such as the absence of data) which are not well managed with usual estimators.

In this paper, a new framework for the tracking of closed curves is described. The proposed approach, formalized through an optimal control technique, enables a continuous tracking along an image sequence of a deformable curve. The associated minimization process consists in a forward integration of a dynamical model followed by a backward integration of an adjoint dynamics. This latter pde includes a term related to the discrepancy between the state variables evolution law and discrete noisy measurements of the system. The closed curves are represented through an implicit surface.

In this paper, we present a framework for dynamic consistent estimation of dense motion fields over a sequence of Meteosat Second Generation (MSG) images. The originality of the approach is to exploit recipes related to optimal control theory developed in geophysical sciences. This framework, known as variational data assimilation, enables to perform the estimation of an unknown state function according to a given dynamical model and to noisy and incomplete measurements. In our work, the measurements are defined according to a smoothed brightness consistency model whereas the dynamical model on which we rely is derived from a velocity conservation law. The overall assimilation process is formalized through the minimization of a global spatio- temporal cost functional w.r.t to the complete sequence of motion fields. The minimization is handled considering an adjoint formulation. The resulting scheme consists in iterating a forward integration of the evolution model and a backward integration of the adjoint evolution model guided by a discrepancy measurement between the state variable and the available noisy observations. Such an approach allows us to cope with several delicate situations (such as the absence of data) which are not well managed with usual estimators. The efficiency of our approach is demonstrated on real data. It enables to estimate a sequence of dense motion fields even in situations where data are strongly corrupted.

In this paper, we address the problem of estimating mesoscale dynamics of atmospheric layers from satellite image sequences. Due to the great deal of spatial and temporal distortions of cloud patterns and because of the sparse 3-D nature of cloud observations, standard dense-motion field-estimation techniques used in computer vision are not well adapted to satellite images. Relying on a physically sound vertical decomposition of the atmosphere into layers, we propose a dense-motion estimator dedicated to the extraction of multilayer horizontal wind fields. This estimator is expressed as the minimization of a global function including data and spatio-temporal smoothness terms. A robust data term relying on the integrated-continuity equation mass-conservation model is proposed to fit sparse-transmittance observations related to each layer. A novel spatio-temporal smoother derived from large eddy prediction of a shallow-water momentum-conservation model is used to build constraints for large-scale temporal coherence. These constraints are combined in a global smoothing framework with a robust second-order smoother, preserving divergent and vorticity structures of the flow. For optimization, a two-stage motion estimation scheme is proposed to overcome multiresolution limitations when capturing the dynamics of mesoscale structures. This alternative approach relies on the combination of correlation and optical-flow observations in a variational context. An exhaustive evaluation of the novel method is first performed on a scalar image sequence generated by direct numerical simulation of a turbulent 2-D flow. By qualitative comparisons, the method is then assessed on a METEOSAT image sequence.

In this paper, we present a method for the tracking of fluid flows velocity fields. The technique we propose is formalized within sequential Bayesian filter framework. The filter we propose here combines an Ito diffusion process coming from a stochastic formulation of the vorticity-velocity form of Navier-Stokes equation and discrete measurements extracted from an image sequence. The resulting tracker provides robust and consistent estimations of instantaneous motion fields along the whole image sequence. In order to handle a state space of reasonable dimension for the stochastic filtering problem, we represent the motion field as a combination of adapted basis functions. The used basis functions ensue from a mollification of Biot-Savart integral and a discretization of the vorticity and divergence maps of the fluid vector field. The efficiency of the method is demonstrated on a long real world sequence showing a vortex launch at tip of airplane wing.

Because of their lack of rules, general broadcast videos are more difficult to analyze than news or sport videos. To retrieve human interventions in this context, a robust face tracker is needed. The approach we investigate for face tracking combines three main modules that are a face detector, a region-based tracker and an eye tracker. The region-based tracker relies on a robust parametric motion estimation technique. The eye tracker is based on a Kalman filter. The analysis of the coherence of the trackers output provides an efficient way to detect profile positions and tracking errors. We have thus defined an entirely automatic tracker, able to manage several appearing/disappearing faces, without any a priori knowledge on the image sequence. Experimental results on broadcast videos demonstrate its efficiency to deal with large and rapid motions, occlusions and faces in profile position.

In this paper we present a technique for the tracking of textured almost planar object. The target is modeled as a noisy planar cloud of points. The tracking is led with an appropriate non linear stochastic filter. The particular system that we devised is conditionally Gaussian and can be efficiently implemented through variance reduction principle known as Rao-Blackwellisation. Our model allows also to melt a correlation measurements with dynamic model estimated from the images. Such a cooperation within a stochastic filtering framework allows the tracker to be robust to occlusions and target's unpredictable changes of speed and direction. We demonstrate the efficiency of the tracker on different types of real world sequences.

In this paper, a new conditional formulation of classical filtering methods is proposed. This formulation is dedicated to image sequence-based tracking. These conditional filters allow solving systems whose measurements and state equation are estimated from the image data. In particular, the model that is considered for point tracking combines a state equation relying on the optical flow constraint and measurements provided by a matching technique. Based on this, two point trackers are derived. The first one is a linear tracker well suited to image sequences exhibiting global-dominant motion. This filter is determined through the use of a new estimator, called the conditional linear minimum variance estimator. The second one is a nonlinear tracker, implemented from a conditional particle filter. It allows tracking of points whose motion may be only locally described. These conditional trackers significantly improve results in some general situations. In particular, they allow for dealing with noisy sequences, abrupt changes of trajectories, occlusions, and cluttered background.

Tensor-based approaches for optical flow have been often criticized because of their limitations in handling large motions. This paper shows how to adapt them in a multiscale coarse-to-fine strategy. We show how the same ideas used in the variational framework can be adapted by working with both a multiscale image sequence as well as a multiscale, motion compensated tensor field. Several experiments are presented in order to compare it to some recent well known multiscale techniques. We demonstrate how this approach offers a good compromise between precision and computational efficiency.

We present a method for fully automatic three-dimensional (3D) reconstruction from a pair of weakly calibrated images in order to deal with the modeling of complex rigid scenes. A two-dimensional (2D) triangular mesh model of the scene is calculated using a two-step algorithm mixing sparse matching and dense motion estimation approaches. The 2D mesh is iteratively refined to fit any arbitrary 3D surface. At convergence, each triangular patch corresponds to the projection of a 3D plane. The proposed algorithm relies first on a dense disparity field. The dense field estimation modelized within a robust framework is constrained by the epipolar geometry. The resulting field is then segmented according to homographic models using iterative Delaunay triangulation. In association with a weak calibration and camera motion estimation algorithm, this 2D planar model is used to obtain a VRML-compatible 3D model of the scene.

In this paper, we address the problem of estimating and analyzing the motion of fluids in image sequences. Due to the great deal of spatial and temporal distortions that intensity patterns exhibit in images of fluids, the standard techniques from computer vision, originally designed for quasi-rigid motions with stable salient features, are not well adapted in this context. We thus investigate a dedicated minimization-based motion estimator. The cost function to be minimized includes a novel data term relying on an integrated version of the continuity equation of fluid mechanics, which is compatible with large displacements. This term is associated with an original second-order div-curl regularization which prevents the washing out of the salient vorticity and divergence structures. The performance of the resulting fluid flow estimator is demonstrated on meteorological satellite images. In addition, we show how the sequences of dense motion fields we estimate can be reliably used to reconstruct trajectories and to extract the regions of high vorticity and divergence.

A new method for medical image registration is formulated as a minimization problem involving robust estimators. We propose an efficient hierarchical optimization framework which is both multiresolution and multigrid. An anatomical segmentation of the cortex is introduced in the adaptive partitioning of the volume on which the multigrid minimization is based. This allows to limit the estimation to the areas of interest, to accelerate the algorithm, and to refine the estimation in specified areas. At each stage of the hierarchical estimation, we refine current estimate by seeking a piecewise affine model for the incremental deformation field. The performance of this method is numerically evaluated on simulated data and its benefits and robustness are shown on a database of 18 magnetic resonance imaging scans of the head.

In this paper we describe a new method for medical image registration. The registration is formulated as a minimization problem involving robust estimators. We propose an efficient hierarchical optimization framework which is both multiresolution and multigrid. An anatomical segmentation of the cortex is introduced in the adaptive partitioning of the volume on which the multigrid minimization is based. This allows to limit the estimation to the areas of interest, to accelerate the algorithm, and to refine the estimation in specified areas. Furthermore we introduce a methodology to constrain the registration with landmarks such as anatomical structures. The performances of this method are objectively evaluated on simulated data and its benefits are demonstrated on a large database of real acquisitions.

We address the problem of fluid motion estimation in image sequences. For such motions, standard optical flow methods, based on intensity conservation and spatial coherence of motion field, are not suitable. This is due to the highly deformable nature of a fluid medium. For all applications where fluid motions are to be recovered from images, it is then important to have specific techniques. We investigate such dedicated models which include an original observation constraint, based on the continuity equation from fluid mechanics, and a new div-curl-type smoothness term. Our method is validated on synthetic and real meteorological images.

In this paper we propose studying several ways to implement a realistic and efficient VLSI design for a gradient-based dense motion estimator. The kind of estimator we focus on belongs to the class of differential methods. It is classically based on the optical flow constraint equation in association with a smoothness regularization term and also incorporates robust cost functions to alleviate the influence of large residuals. This estimator is expressed as the minimization of a global energy function defined within the framework of an incremental formulation associated with a multiresolution setup. In order to make possible the conception of efficient hardware, we consider a modified minimization strategy. This new minimization strategy is not only well suited to VLSI derivation, it is also very efficient in terms of quality of the result. The complete VLSI derivation is realized using high-level specifications.

Addresses the problem of estimating and analyzing the motion in image sequences that involve fluid phenomena. In this context, standard motion estimation techniques are not well-adapted, and more dedicated approaches have to be designed. We thus propose to estimate, in a joint and cooperative way, a dense motion field and a peculiar parametric representation of the flow. The parametric model is derived from an extension of the Rankine vortex model and includes a laminar flow field. Dense and parametric fields are estimated by minimizing a robust global objective function, thanks to a specific alternate scheme. The method has been validated on different kinds of meteorological image sequences.

In this work, we propose to cope in a joint and cooperative way with two types of problems that were addressed within distinct approaches so far: (a) the estimation of dense velocity (or displacement) fields of fluid flows, using a robust extension of the minimization-based approach proposed by Horn and Schunck; (b) the extraction of kinematic entities of particular interest from the fluid mechanics point of view: vortices, sinks, and sources. As concerns the second item, we extend Rankine vortex model, and thus design an original non-linear parametric model of the motion stemming from vortices, sinks and sources. We make interact this parametric model with the dense flow field within a global objective function, and we design a specific alternate scheme to perform the minimization of this function.

This paper focuses on the estimation of the apparent motion field between two consecutive frames in an image sequence. The approach developed here is a tradeoff between methods based on global parameterized flow models and local dense optic flow estimators. The method relies on an adaptive multigrid minimization approach. In addition to accelerated convergence toward good estimates, it allows to mix different parameterizations of the estimate relative to adaptive partitions of the image. The performances of the resulting algorithms are demonstrated in the difficult context of a non-convex energy. Experimental results on real world Meteosat sequences are presented.

In the present paper we propose a new algorithm for planar facets segmentation of sequences of uncalibrated images in order to recover 3D models of complex scenes. This is performed using a two-step algorithm. First a robust and regularized dense disparity field is computed under the epipolar geometry constraint. The resulting field is segmented according to homographic models using an iterative Delaunay triangulation.

In this paper, we address the issue of recovering and segmenting the apparent velocity field in sequences of images. As for motion estimation, we minimize an objective function involving two robust terms. The first one cautiously captures the optical flow constraint, while the second (a priori) term incorporates a discontinuity-preserving smoothness constraint. To cope with the nonconvex minimization problem thus defined, we design an efficient deterministic multigrid procedure. It converges fast toward estimates of good quality, while revealing the large discontinuity structures of flow fields. We then propose an extension of the model by attaching to it a flexible object-based segmentation device based on deformable closed curves (different families of curve equipped with different kinds of prior can be easily supported). Experimental results on synthetic and natural sequences are presented, including an analysis of sensitivity to parameter tuning.

Motion estimation is crucial in domains such as robot vision or image sequence coding. Fast or real time computing is often required but is problematic due to computational complexity. In this paper, an algorithm using multiresolution multigrid Markov random fields is parallelized. Such realizations have already been studied for fine parallelizations on massively parallel machines. The goal of this paper is to achieve a coarse parallelization on a network of workstations, using the Parallel Virtual Machine (PVM) software package.

We address the problem of incremental optical flow estimation. Following Black et al., we design a cost function whose data and prior terms both involve robust M-estimators. The non-convex minimization is lead with an efficient multigrid algorithm which converges fast toward estimates of good quality while providing, at very low cost, crude estimates revealing the large discontinuity structures of the flow field.

We present a new algorithmic framework which enables making a full use of the large potential of data parallelism available on 2D processor arrays for the implementation of nonlinear multigrid relaxation methods. This framework leads to fast convergence towards quasi-optimal solutions. It is demonstrated on two different low-level vision applications.>