Feature matching between image pairs is a fundamental problem in computer vision that drives many applications, such as SLAM. Recently, semi-dense matching approaches have achieved substantial performance enhancements and established a widely-accepted coarse-to-fine paradigm. However, the majority of existing methods focus on improving coarse feature representation rather than the fine-matching module. Prior fine-matching techniques, which rely on point-to-patch matching probability expectation or direct regression, often lack precision and do not guarantee the continuity of feature points across sequential images. To address this limitation, this paper concentrates on enhancing the fine-matching module in the semi-dense matching framework. We employ a lightweight and efficient homography estimation network to generate the perspective mapping between patches obtained from coarse matching. This patch-to-patch approach achieves the overall alignment of two patches, resulting in a higher sub-pixel accuracy by incorporating additional constraints. By leveraging the homography estimation between patches, we can achieve a dense matching result with low computational cost. Extensive experiments demonstrate that our method achieves higher accuracy compared to previous semi-dense matchers. Meanwhile, our dense matching results exhibit similar end-point-error accuracy compared to previous dense matchers while maintaining semi-dense efficiency.

Semi-dense feature matching methods have shown strong performance in challenging scenarios. However, the existing pipeline relies on a global search across the entire feature map to establish coarse matches, limiting further improvements in accuracy and efficiency. Motivated by this limitation, we propose a novel pipeline, CasP, which leverages cascaded correspondence priors for guidance. Specifically, the matching stage is decomposed into two progressive phases, bridged by a region-based selective cross-attention mechanism designed to enhance feature discriminability. In the second phase, one-to-one matches are determined by restricting the search range to the one-to-many prior areas identified in the first phase. Additionally, this pipeline benefits from incorporating high-level features, which helps reduce the computational costs of low-level feature extraction. The acceleration gains of CasP increase with higher resolution, and our lite model achieves a speedup of $\sim2.2\times$ at a resolution of 1152 compared to the most efficient method, ELoFTR. Furthermore, extensive experiments demonstrate its superiority in geometric estimation, particularly with impressive cross-domain generalization. These advantages highlight its potential for latency-sensitive and high-robustness applications, such as SLAM and UAV systems. Code is available at https://github.com/pq-chen/CasP.

Carbon neutrality has become a global priority, and high spatio-temporal resolution data on the column-average dry-air mole fraction carbon dioxide (XCO2) is essential for tracking progress and guiding policy adjustments. However, satellite-derived XCO2exhibits significant temporal and spatial gaps due to influences such as orbital dynamics and cloud cover. Additionally, the low spatial resolution of CarbonTracker (CT) is insufficient to meet the current demands for fine-scale monitoring. Current XCO2assessment methods often rely solely on single-pixel data, overlooking the spatio-temporal correlations. In this article, we introduce a deep learning-based spatio-temporal model (DSTM) that extracts features from multiple data sources related to atmospheric transport, carbon emissions, and carbon sinks, enabling fine-scale XCO2assessments. Additionally, XCO2data from the Orbiting Carbon Observatory-2 (OCO-2) and CT were fused at a 0.1° spatial resolution to generate training labels with broader coverage and more samples, serving as fitting labels. Our approach produced daily, full-coverage 0.1° resolution XCO2maps for China from 2015 to 2020, and analyzed changes in XCO2growth trends over this period. Numerical results show that our model outperforms traditional deep learning methods. Model validation using data from four ground-based observation sites of the Total Carbon Column Observing Network (TCCON) achieved an average R² of 0.86 and an RMSE of 2.67 ppm. The extraction and fusion of spatio-temporal features from multiple data sources provide a novel approach for reconstructing missing XCO2data.

Prior studies on Remote Sensing Foundation Model (RSFM) reveal immense potential towards a generic model for Earth Observation. Nevertheless, these works primar-ily focus on a single modality without temporal and geo-context modeling, hampering their capabilities for diverse tasks. In this study, we present SkySense, a generic billion-scale model, pretrained on a curated multimodal Remote Sensing Imagery (RSI) dataset with 21.5 million temporal sequences. SkySense incorporates a factorized multimodal spatiotemporal encoder taking temporal sequences of opti-cal and Synthetic Aperture Radar (SAR) data as input. This encoder is pretrained by our proposed Multi-Granularity Contrastive Learning to learn representations across different modal and spatial granularities. To further enhance the RSI representations by the geo-context clue, we introduce Geo-Context Prototype Learning to learn region-aware prototypes upon RSI's multimodal spatiotemporal features. To our best knowledge, SkySense is the largest Multi-Modal RSFM to date, whose modules can be flexibly combined or used individually to accommodate various tasks. It demonstrates remarkable generalization capabilities on a thor-ough evaluation encompassing 16 datasets over 7 tasks, from single- to multimodal, static to temporal, and classification to localization. SkySense surpasses 18 recent RSFMs in all test scenarios. Specifically, it outperforms the latest models such as GFM, SatLas and Scale-MAE by a large margin, i.e., 2.76%, 3.67% and 3.61% on average respectively. We will release the pretrained weights to facilitate future research and Earth Observation applications.

Abstract Modeling in computer vision has long been dominated by convolutional neural networks (CNNs). Recently, in light of the excellent performance of self-attention mechanism in the language field, transformers tailored for visual data have drawn significant attention and triumphed over CNNs in various vision tasks. These vision transformers heavily rely on large-scale pre-training to achieve competitive accuracy, which not only hinders the freedom of architectural design in downstream tasks like object detection, but also causes learning bias and domain mismatch in the fine-tuning stages. To this end, we aim to get rid of the “pre-train and fine-tune” paradigm of vision transformer and train transformer based object detector from scratch. Some earlier works in the CNNs era have successfully trained CNNs based detectors without pre-training, unfortunately, their findings do not generalize well when the backbone is switched from CNNs to a vision transformer. Instead of proposing a specific vision transformer based detector, in this work, our goal is to reveal the insights of training vision transformer based detectors from scratch. In particular, we expect those insights to help other researchers and practitioners, and inspire more interesting research in other fields, such as remote sensing, visual-linguistic pre-training, etc. One of the key findings is that both architectural changes and more epochs play critical roles in training vision transformer based detectors from scratch. Experiments on the MS COCO dataset demonstrate that vision transformer based detectors trained from scratch can also achieve similar performance to their counterparts with ImageNet pre-training.

As one of the most important application for remote sensing monitoring, land cover mapping has witnessed notable advancements in data acquisition, algorithmic diversity, and classification accuracy. Despite the instrumental role data-driven algorithms have played in the development of global land cover products, their inherent limitations as “black box” methods often fall short of meeting end-users’ specific requirements. In this study, built upon the foundation of the earlier land cover monitoring platform [FROM-GLC plus(FGP)], a data and knowledge dual-driven framework (FGP 2.0) was developed as a user-adaptive framework for intelligent remote sensing land cover mapping. By incorporating ontology-based semantic descriptions with advanced data-driven algorithms, FGP 2.0 provides the capacity for both traditional annual mapping and emerging dynamic mapping. Our results illustrate that FGP 2.0 significantly improves the overall accuracy of annual maps by ~5%, and dynamic maps by ~20% compared to FGP. Moreover, an operational dynamic mapping tool has been developed on the Google Earth engine (GEE), enabling the generation of near-real-time land cover maps for any given place. With an extensible and flexible mapping framework, FGP 2.0 demonstrates the potential of customized land cover monitoring results to suit different application scenarios. This innovative approach not only meets the current demand for reliable annual and dynamic land cover maps but also sets a new benchmark for the integration of geoscientific expertise with machine learning techniques in remote sensing monitoring.

Under the impact of climate change, monitoring forest carbon stock becomes an important task to evaluate the changes in carbon sequestrated from the atmosphere. Forest carbon stock estimation is still a challenging task, due to limited data sources that have a high correlation with above-ground biomass. With the help of the NASA Global Ecosystem Dynamics Investigation (GEDI) mission, above-ground biomass (AGB) can be measured by using the LiDAR data provided. However, GEDI data is sparse since it only samples about 4% of the Earth’s land surface between 51.6° N&S. Previous studies demonstrated L-Band SAR’s promising ability in retrieving forest stem volumes and estimating above-ground biomass. In this work, we propose a Deep Learning based workflow which utilizes PALSAR-2 L-Band images and GEDI to generate wall-to-wall above-ground biomass maps of North America. The workflow uses Convolutional Neural Network as the DL model and leverages both PALSAR-2 L-Band images and GEDI Relative Heights data to estimate the dense above-ground biomass maps. The results show that, by fusing GEDI Level 2 Relative Heights data with PALSAR-2 L-Band SAR data, it is possible to achieve a significantly high correlation with GEDI level 4 AGB data, as the final R-squared score of our model is as high as 0.83.

Given a text query, the text-to-video retrieval task aims to find the relevant videos in the database. Recently, model-based (MDB) methods have demonstrated superior accuracy than embedding-based (EDB) methods due to their excellent capacity of modeling local video/text correspondences, especially when equipped with large-scale pre-training schemes like ClipBERT. Generally speaking, MDB methods take a text-video pair as input and harness deep models to predict the mutual similarity, while EDB methods first utilize modality-specific encoders to extract embeddings for text and video, then evaluate the distance based on the extracted embeddings. Notably, MDB methods cannot produce explicit representations for text and video, instead, they have to exhaustively pair the query with every database item to predict their mutual similarities in the inference stage, which results in significant inefficiency in practical applications.

Mapping highly dynamic cropping systems using satellite image time series is still challenging even when robust approaches are used. We assessed the potential of using high spatial and temporal resolution PlanetScope time series and deep neural networks (Convolutional Neural Networks (CNN) in one dimension - Conv1D, Long Short-Term Memory (LSTM), and Multi-Layer Perceptron (MLP)) for mapping integrated crop-livestock systems (ICLS) and different land covers in the western region of São Paulo State, Brazil. We used 10-day and 15-day composite EVI and NDVI time series (both individually and combined) as input data in the neural network classifiers. Conv1D using both EVI and NDVI 10 day-composite time series outperformed the other classifiers evaluated in this study (LSTM and MLP), allowing improved discrimination of land parcels with ICLS in our study area.