Event cameras deliver visual information characterized by a high dynamic range and high temporal resolution, offering significant advantages in estimating optical flow for complex lighting conditions and fast-moving objects. Current advanced optical flow methods for event cameras largely adopt established image-based frameworks. However, the spatial sparsity of event data limits their performance. In this paper, we present BAT, an innovative framework that estimates event-based optical flow using bidirectional adaptive temporal correlation. BAT includes three novel designs: 1) a bidirectional temporal correlation that transforms bidirectional temporally dense motion cues into spatially dense ones, enabling accurate and spatially dense optical flow estimation; 2) an adaptive temporal sampling strategy for maintaining temporal consistency in correlation; 3) spatially adaptive temporal motion aggregation to efficiently and adaptively aggregate consistent target motion features into adjacent motion features while suppressing inconsistent ones. Our BAT achieves state-of-the-art performance on the DSEC-Flow benchmark, outperforming existing methods by a large margin while also exhibiting sharp edges and high-quality details. Our BAT can accurately predict future optical flow using only past events, significantly outperforming E-RAFT’s warm-start approach.

Recent approaches to VO have significantly improved performance by using deep networks to predict optical flow between video frames. However, existing methods still suffer from noisy and inconsistent flow matching, making it difficult to handle challenging scenarios and long-sequence estimation.To overcome these challenges, we introduce Spatio-Temporal Visual Odometry (STVO), a novel deep network architecture that effectively leverages inherent spatio-temporal cues to enhance the accuracy and consistency of multi-frame flow matching. With more accurate and consistent flow matching, STVO can achieve better pose estimation through the bundle adjustment (BA).Specifically, STVO introduces two innovative components: 1) the Temporal Propagation Module that utilizes multi-frame information to extract and propagate temporal cues across adjacent frames, maintaining temporal consistency; 2) the Spatial Activation Module that utilizes geometric priors from the depth maps to enhance spatial consistency while filtering out excessive noise and incorrect matches.Our STVO achieves state-of-the-art performance on TUM-RGBD, EuRoc MAV, ETH3D and KITTI Odometry benchmarks. Notably, it improves accuracy by 77.8% on ETH3D benchmark and 38.9% on KITTI Odometry benchmark over the previous best methods.

Stereo matching recovers depth from image correspondences. Existing methods struggle to handle ill-posed regions with limited matching cues, such as occlusions and textureless areas. To address this, we propose MonSter, a novel method that leverages the complementary strengths of monocular depth estimation and stereo matching. MonSter integrates monocular depth and stereo matching into a dual-branch architecture to iteratively improve each other. Confidence-based guidance adaptively selects reliable stereo cues for monodepth scale-shift recovery. The refined monodepth is in turn guides stereo effectively at ill-posed regions. Such iterative mutual enhancement enables MonSter to evolve monodepth priors from coarse object-level structures to pixel-level geometry, fully unlocking the potential of stereo matching. As shown in Fig. 2, MonSter ranks 1stacross five most commonly used leaderboards — SceneFlow, KITTI 2012, KITTI 2015, Middlebury, and ETH3D. Achieving up to 49.5% improvements (Bad 1.0 on ETH3D) over the previous best method. Comprehensive analysis verifies the effectiveness of MonSter in ill-posed regions. In terms of zero-shot generalization, MonSter significantly and consistently outperforms state-of-the-art across the board. The code is publicly available at: https://github.com/Junda24/MonSter.

Stereo matching is a core component in many computer vision and robotics systems. Despite significant advances over the last decade, handling matching ambiguities in ill-posed regions and large disparities remains an open challenge. In this paper, we propose a new deep network architecture, called IGEV++, for stereo matching. The proposed IGEV++ constructs Multi-range Geometry Encoding Volumes (MGEV), which encode coarse-grained geometry information for ill-posed regions and large disparities, while preserving fine-grained geometry information for details and small disparities. To construct MGEV, we introduce an adaptive patch matching module that efficiently and effectively computes matching costs for large disparity ranges and/or ill-posed regions. We further propose a selective geometry feature fusion module to adaptively fuse multi-range and multi-granularity geometry features in MGEV. Then, we input the fused geometry features into ConvGRUs to iteratively update the disparity map. MGEV allows to efficiently handle large disparities and ill-posed regions, such as occlusions and textureless regions, and enjoys rapid convergence during iterations. Our IGEV++ achieves the best performance on the Scene Flow test set across all disparity ranges, up to 768px. Our IGEV++ also achieves state-of-the-art accuracy on the Middlebury, ETH3D, KITTI 2012, and 2015 benchmarks. Specifically, IGEV++ achieves a 3.23% 2-pixel outlier rate (Bad 2.0) on the large disparity benchmark, Middlebury, representing error reductions of 31.9% and 54.8% compared to RAFT-Stereo and GMStereo, respectively. We also present a real-time version of IGEV++ that achieves the best performance among all published real-time methods on the KITTI benchmarks.

The leaf area index (LAI) is a key structural parameter of vegetation canopies. Accordingly, several moderate-resolution global LAI products have been produced and widely used in the field of remote sensing. However, the accuracy of the current moderate-resolution global LAI products cannot satisfy the requirements recommended by the LAI application communities, especially in heterogeneous areas composed of mixed land cover types. In this study, we propose a mixed-pixel correction (MPC) method to improve the accuracy of LAI retrievals over heterogeneous areas by considering the influence of heterogeneity caused by the mixture of different biome types with the help of high-resolution land cover maps. The DART-simulated LAI, the aggregated Landsat LAI, and the site-based high-resolution LAI reference maps are used to evaluate the performance of the MPC method. The results indicate that the MPC method can reduce the influences of spatial heterogeneity and biome misclassification to obtain the LAI with much better accuracy than the Moderate Resolution Imaging Spectroradiometer (MODIS) main algorithm, given that the high-resolution land cover map is accurate. The root mean square error (RMSE) (bias) decreases from 0.749 (0.486) to 0.414 (0.087), while the R2 increases from 0.084 to 0.524, and the proportion of pixels that fulfill the uncertainty requirement of the GCOS increases from 38.2% to 84.6% for the results of site-based high-resolution LAI reference maps. Spatially explicit information about vegetation fractional cover can further reduce uncertainties induced by variations in canopy density for the results of DART simulated data. The proposed method shows potential for improving global moderate-resolution LAI products.

High-accuracy estimates of fractional vegetation cover (FVC) are vital for regional-scale vegetation growth monitoring. In this context, a question worth exploring is whether FVC estimation methods developed at 300-m to 1-km spatial resolution are suitable for finer spatial resolution satellite images. This study compared the performances of three types of algorithms [i.e., the pixel dichotomy model (PDM) based on either the normalized difference vegetation index (NDVI) or an index of near-infrared reflectance of vegetation (NIRv), the gap probability theory (GPT), and linear spectral mixture analysis (LSMA)] based on FVC ground measurements and fine resolution reference maps from the Validation of Land European Remote sensing Instrument (VALERI) project and the ImagineS field campaigns. For all vegetation types, the FVC estimates from the GPT method showed the best consistency with ground measurements of FVC [root mean square error (RMSE) = 0.17 and bias (BIAS) = 0.05]. For forest types, the PDM method based on NDVI also showed satisfactory results with ground measurements (RMSE = 0.17 and BIAS = 0.11). For sparse grasses, the PDM method based on NIRv showed better agreement with ground measurements (RMSE = 0.15 and BIAS = 0.01). This study provides a reference for selecting the method of FVC estimation with fine spatial resolution images.

Spatiotemporally continuous long-term leaf area index (LAI) products are urgently needed to monitor environmental changes. The current filter- or curve-fitting-based time series reconstructive algorithms fail to reconstruct the LAI time series with many continuous missing values or missing values in key phenological periods, which are common issues in high-spatial-resolution LAI time series. In this article, a meteorological data-driven backpropagation neural network (MBPNN) was proposed to reconstruct discontinuous LAI profiles with a two-step process using vegetation phenological information. As the basis of the strong dependence of vegetation growth on meteorological conditions, a reasonable growth trajectory of reconstructed LAI can be guaranteed by the algorithm even though if many observed values are missing. Validations for reconstructed LAI were conducted both spatially and temporally based on reference maps and field-measured long-term observations. The results showed that the LAI predicted by the MBPNN had a similar accuracy (RMSE = 0.4076) as the Landsat LAI inversions (RMSE = 0.4083) and a similar reconstructed trajectory as the field-measured LAI series even though over 100 days of continuous data were missing (RMSE = 0.1620). A comparison with the Harmonic ANalysis of Time Series (HANTS) algorithm showed that the accuracy of MBPNN was more stable regardless of the size/position of the missing data, and the proposed method performed much better when the data were continuously missing for 50 days or more.

Disparity upsampling methods are derived for restoring high-quality disparity maps from active three-dimensional imaging techniques such as time-of-flight method. Although effective, traditional upsampling methods exhibit certain drawbacks. For example, noisy disparity values in the low-scaled level are taken indiscriminately to carry out the upsampling assignment, which could lead to dramatically weakened results. To solve this problem, we herein present a Markov random field (MRF) based disparity upsampling method using confidence evaluations. We utilize confidence measures under the stereo configuration to evaluate the reliability of disparity value. The confidence maps are then properly integrated into the state-of-the-art MRF-based depth upsampling (MBU) method to constrain the negative effect caused by the noisy disparity. More specifically, the confidence is used to remove unqualified pixels from participating in the disparity upsampling computation. Extensive experiments were performed to validate the proposed new method. In these experiments, initial low-scaled disparity maps were from either ground truth or stereo matching methods. Results verified that our method can obtain significantly better upsampled disparity maps than that from the original MBU or other nonconfidence-based methods. In addition, we implemented our algorithms on general public utilities to improve the execution speed.