Flow visualization through motion estimation using time-sequenced images plays a significant role in analyzing and understanding complex flow phenomena, and it is widely used in meteorology, oceanography, medicine, astronomy, experimental fluid mechanics, etc. However, it is difficult for current motion estimators to adapt to illumination changes, remove instable perturbation, and capture diverse motion patterns. In this paper, a novel flow visualization tool is developed to address these issues by employing a structure-enhanced motion estimator composed of a data term and a regularization term. Specifically, a statistical correlation descriptor is designed for the data term to improve the accuracy of motion estimation by enhancing both illumination robustness and matching discrimination. Inspired by the strong distinguishability of a structure-texture distribution in a local window, a structure-enhanced regularizer that considers the physical mechanism of fluid diffusion is introduced to capture different motion patterns, enhance prominent flow structures, and remove unnecessary ripples or textures caused by instable perturbation or noise. The experimental results demonstrate that our approach significantly outperforms current motion estimators in handling illumination changes and predicting complex fluid flows, and it also achieves state-of-the-art evaluation results on the public fluid flow datasets. Furthermore, the designed flow visualization tool successfully captures diverse motion patterns in Jupiter’s White Ovals, which is crucial for understanding the physical mechanisms behind their formation and sustenance.

In this paper, we proposed a novel motion model with Helmholtz decomposition for complex fluid flows in a filtering-based optical flow framework, where the optimization of the regularization term is treated as a filtering process, and different motion patterns can be captured by finding appropriate filter kernels based on the designed filters. In this framework, we introduce a novel optical flow method with a joint spatial filter, which is based on the Helmholtz decomposition theorem that assumes a local motion field is composed of a curl field and a divergence field. By adjusting the scale of the weights in the filter kernels and combining a curl filter with a divergence filter in a certain ratio, it can simulate different motion patterns. In addition, if the correlation between the horizontal and vertical components of the optical flow field in the filter kernel is eliminated, it will be transformed into a linear motion model. Based on this linear motion model, we also develop a novel optical flow method with an adaptive guided filter. By finding an adaptive filter kernel driven by both the input image and the guided motion field for the designed filter, it can successfully capture different motion patterns, and yield an edge-preserving smoothing optical flow field. Most importantly, the proposed optical flow model provides a new way to design the regularizers for capturing different motion patterns in complex fluid flow. In particular, the designed optical flow method with an adaptive guided filter significantly outperforms the current state-of-the-art optical flow methods in predicting complex fluid flows.

Optical flow computation for video under the dynamic illumination is a challenging issue in video multimedia applications. In this paper, we solve this issue by introducing an illumination-invariant framework for variational optical flow estimation. It consists of an illumination-invariance model that handles complex illumination changes and a data enhancement model that guarantees highly accurate optical flow estimation. In this framework, we design a log-correlation descriptor for the data term, which handles complex illumination changes by eliminating the common parameters shared by the neighboring pixels in the corresponding illumination change model while improving the accuracy of optical flow estimation by enhancing the discriminability of the data term matching. We also introduce a novel optical flow model with$L_{0}$norm regularization, which reconstructs optical flow field by a sparse flow gradient counting scheme. Different from other edge-preserving regularizers, it does not depend on local motion features, but locates important flow edges globally. Therefore, it will not cause edge blurriness due to avoiding local filtering or average operation. It is particularly effective for enhancing major flow edges while eliminating a manageable degree of low-amplitude motion structures to control smoothing and reduce oversegmentation artifacts. Even small-scale motion structures with high contrast can be preserved remarkably well. The experimental results show our method significantly outperforms previous illumination-robust optical flow methods in handling complex illumination changes, and achieves competitive evaluation results on the challenging MPI-Sintel and Kitti datasets.

Accurate optical flow estimation with the frequency-domain regularization is a challenging problem in computer vision. In this paper, we solve this issue by introducing a novel optical flow method related to the frequency domain that uses TV-wavelet regularization. Specifically, we regard TV-wavelet regularization as a filtering process. After wavelet transform for optical flow field, we firstly remove outliers by performing a threshold operation. Then, we make up for lost motion information (such as flow edges and important motion details) determined by these missing or damaged wavelet coefficients by adding TV-wavelet coefficients that are obtained from transform spectrum of the prior flow geometrical features, which are controlled by the image structures. By combining the advantages of total variation to recover geometric structures with the strengths of wavelet representation to remove outliers, the proposed method significantly outperforms the current frequency-domain optical flow methods in removing outliers, preserving sharp flow edges, and restoring important motion details. It also shows competitive optical flow evaluation results on the challenging MPI-Sintel, Kitti, and Middlebury datasets.

Many recent variational optical flow methods are not robust for illumination variance, and they only consider local image relation in terms of illumination. In this paper, we propose a new efficient illumination-invariance total variation optical flow method called the weighted regularization transform, which uses and optimizes the Weber's Law. Our method exploits unequal probability as the weight that has non-local information to estimate stable optical flow despite illumination changes. The proposed method uses a coarse-to-fine pyramid model to reduce the influence on the data term from illumination. Then, an energy optimization procedure is introduced to constrain the minimization of the data term with the non-local regularization. Experimentation with the proposed method has been performed on three optical flow datasets and a face liveness detection database, which have challenging illumination variations, and the results demonstrate that the proposed method is quite robust with respect to variations in illumination.

We present a novel optical flow estimation framework that reinterprets most popular algorithms (e.g., the Horn-Schunck model, PDE-based models, TV-based models, and nonlocal-based models) from an iterative filtering perspective. Different regularizers in the related optical flow estimation algorithms can be achieved by adopting different filtering operations. The key merit of the proposed framework is that a variety of filtering models and corresponding optimization strategies, which are used in image processing, can be directly utilized in the regularization term, leading to a convenient way of designing appropriate optical flow estimation algorithms. Under the proposed filtering framework, we demonstrate easily designing optical flow estimation algorithms with respect to translation consistency, rotation consistency, and divergence consistency constraints by adopting different spatial filters, respectively. We also derive a novel optical flow estimation algorithm with 3D filtering-based model for the regularization term, which makes use of nonlocal self-similarity and sparse characteristic of optical flow field. Benefited from the advantages of patch-based nonlocal sparse regularizer, the derived algorithm can remove outliers while preserving sharp flow edges and important motion details. At present, the proposed algorithm achieves a high ranking on the challenging MPI-Sintel dataset and shows good performance on the Kitti flow 2015 and Middlebury datasets.

Efficient optical flow estimation with high accuracy is a challenging problem in computer vision. In this paper, we present a simple but efficient segmentation-based PatchMatch framework to address this issue. Specifically, it firstly generates sparse seeds without losing important motion information by over-segmentation, and then yields sparse matches by adopting a coarse-to-fine PatchMatch with sparse seeds. Such a scheme enhances the robustness of global regularization and yields better matching results compared with the existing NNF techniques while leading to a significant speed-up due to the sparsity of these seeds. Simultaneously, we introduce an extended nonlocal propagation and adaptive random search to address the basic limitation of the traditional coarse-to-fine framework in handing motion details that often vanish at coarser levels. Finally, we obtain dense matches at the finest level through an efficient sparse-to-dense matching according to the cues of over-segmentation. While performing an efficient approximation for over-segmentation, the proposed algorithm runs significantly fast and is robust to large displacements while preserving important motion details. It also achieves good performance on the challenging MPI-Sintel and Kitti flow 2015 datasets.

Fast and accurate optical flow estimation is a challenging problem in computer vision. In this paper, we present a novel model to solve the optical flow problem by combining the strengths of the Split Bregman method with the advantages of an efficient variational framework. It allows us to employ different regularization tensors for the Split Bregman regularizer to preserve motion discontinuities. Simultaneously, a novel nonlocal Split Bregman method with adaptive support weights is also developed for the smoothness regularization to preserve motion details, repel outliers, enhance contrast, and reduce motion blurring and staircase effect. The proposed algorithm shows significant improvements in preserving sharp flow edges and important motion details. Most of all, it requires only a few iterations to achieve fast convergence and runs faster than the classic TV-L1 flow method. Our method significantly outperforms the current state-of-the-art methods on the challenging MPI-Sintel dataset and shows good performance on the Kitti flow 2015 and Middlebury datasets.