Real-time 3D object detection from point clouds is essential for dynamic scene understanding in applications such as augmented reality, robotics, and navigation. We introduce a novel Spatial-prioritized and Rank-aware 3D object detection (SR3D) framework for indoor point clouds, to bridge the gap between how detectors are trained and how they are evaluated. This gap stems from the lack of spatial reliability and ranking awareness during training, which conflicts with the ranking-based prediction selection used at inference. Such a training-inference gap hampers the model’s ability to learn representations aligned with inference-time behavior. To address the limitation, SR3D consists of two components tailored to the spatial nature of point clouds during training: a novel spatial-prioritized optimal transport assignment that dynamically emphasizes well-located and spatially reliable samples, and a rank-aware adaptive self-distillation scheme that adaptively injects ranking perception via a self-distillation paradigm. Extensive experiments on ScanNet V2 and SUN RGB-D show that SR3D effectively bridges the training-inference gap and significantly outperforms prior methods in accuracy while maintaining real-time speed.

This paper proposes a multi-view collaborative matching strategy for reliable track construction in complex scenarios. We observe that the pairwise matching paradigms applied to image set matching often result in ambiguous estimation when the selected independent pairs exhibit significant occlusions or extreme viewpoint changes. This challenge primarily stems from the inherent uncertainty in interpreting intricate 3D structures based on limited two-view observations, as the 3D-to-2D projection leads to significant information loss. To address this, we introduce CoMatcher, a deep multi-view matcher to (i) leverage complementary context cues from different views to form a holistic 3D scene understanding and (ii) utilize cross-view projection consistency to infer a reliable global solution. Building on CoMatcher, we develop a groupwise framework that fully exploits cross-view relationships for large-scale matching tasks. Extensive experiments on various complex scenarios demonstrate the superiority of our method over the mainstream two-view matching paradigm.

Deep learning has achieved promising progress for digital elevation model (DEM) super-resolution (SR). However, the existing methods rarely consider the integration of multi-modal data with auxiliary high-frequency information. A primary challenge stems from the heterogeneous feature representations among these data sources, which complicates the effective learning of the terrain feature mapping relationships. In this paper, we propose a novel framework for DEM SR by integrating optical remote sensing imagery as the auxiliary data. A terrain-guided texture-edge feature fusion network is constructed to transfer the feature representation of high-resolution image textures with the guidance of informative terrain features, for adapting DEM SR learning. By exploiting the multi-dimensional attention mechanism, the meaningful components from the image conforming to the terrain features provide high-frequency information for DEM SR, while noisy features related to spectral variations are excluded from modelling. The terrain-oriented textural and edge features are then fused to generate the SR result with the constraint of a terrain feature-aware loss function. Extensive experiments on both simulated and real datasets indicate that the proposed method can reconstruct a DEM with high-accuracy elevation and sharper terrain details, and outperforms the state-of-art methods.

Class activation map (CAM) based weakly-supervised semantic segmentation (WSSS) of remote sensing (RS) images has attracted extensive research interests for its potential in reducing annotation cost. However, challenged by unconstrained activation issue, existing methods struggle to delineate object boundaries clearly, making them particularly difficult to separate multiple densely packed objects, which are common in RS images. By conducting an in-depth analysis of RS image characteristics, we observed a strong correlation between object shapes and their semantics. Inspired by this finding, we propose an Intrinsic Shape Activation Network (ISANet) to learn the category-relevant shape priors as geometry constraints for target-focused region activation in WSSS of RS images. The key idea is to distill the intrinsic shape priors from the hybrid features that are deterministic in classification. Specifically, we adopt a dual-branch architecture to decouple the learning of shape and texture features and leverage a shape awareness alignment module to generate boundary-clear CAMs for computing pseudo labels. In this way, CAMs are generated with perception of target shapes, which increases the completeness of activation regions and alleviates the ultrarange responses. Extensive experiments demonstrates the superiority of our method in delineating densely-packed objects with clear contours, which is especially beneficial for separating multiple targets in RS images. Our method improves the mIoU of the state-of-the-art method by 7.9% and 3.3% on the NWPU VHR-10 and iSAID dataset respectively.

Snow is a core part of the cryosphere that plays an important role in climate regulation, hydrological cycles, and ecosystems. Therefore, deriving long-term and large-scale snow depth (SD) time series and monitoring their spatial and temporal variations are crucial. The high-resolution, all-weather SAR imagery provided by Sentinel-1 offers an important data source for monitoring SD. This study proposes a deep learning network called STCTNet, which combines the multi-channel characteristics of convolutional neural networks with the attention mechanism of Transformer to enhance the model ’ s ability to comprehensively process local and global features. The spatiotemporal distribution characteristics of SD stations are also considered to construct spatiotemporally weighted SD variable. This model was trained using data obtained from Colorado between 2017 and 2019, and results show that this model achieves an RMSE of 4.828 cm, MAE of 2.398 cm, and R2of 0.909. The trained model was then used to predict the average SD for February from 2017 to 2022 at a spatial resolution of 20 m. STCTNet obtained overall RMSE and MAE values of 8.7 cm and 4.6 cm, respectively, which show lower RMSE and MAE compared with those of AMSR2 (17.9 cm and 10 cm) and ERA5-Land datasets (18.5 cm and 13.1 cm). An analysis of SD changes from 2017 to 2022 shows that SD may be affected by a variety of climatic anomalies. The application of active microwave data and deep learning techniques in this study provides an approach that improves both the spatial resolution and estimation accuracy of SD.

Image-level weakly-supervised land-cover classification (WSLC) relies on image labels to free land-cover classification networks from expensive pixel-level annotations. Existing weakly-supervised methods commonly generate class activation maps (CAMs) from a trained classification network to produce pixel-level pseudo labels for training land-cover classification networks. However, CAMs struggle to completely and accurately capture land-covers in remote sensing images due to the lack of precise localization information in weak supervision. To alleviate semantic uncertainty in CAMs, this paper proposes an Expanding-Separating (ES) scheme to generate high-quality CAMs of land-covers. The ES scheme expands activation areas in CAMs by lifting activation values in inactivated regions and suppresses CAM noises by decreasing values in confusing regions. Experimental results on the DeepGlobe dataset demonstrate that the proposed ES scheme significantly improves the quality of CAMs and benefits the training of land-cover classification, achieving state-of-the-art performance.

Image super-resolution (SR) enhances the spatial resolution and details of remote sensing images to obtain a better visual quality. However, the existing SR methods rarely consider the imaging degradation factors caused by unfavorable atmospheric conditions, which hinders them from accurately modeling the scale dependencies for the real texture features and results in artifacts in the reconstructed images. In this letter, we propose the degradation-oriented progressive learning SR (DoPSR) framework, which handles the degradation issues posed by the low resolution, together with haze, in remote sensing images. Specifically, DoPSR progressively optimizes the SR estimation for the haze-corrupted input by decomposing the challenging restoration task into feasible sub-problems. It begins with two parallel sub-networks for haze component prediction and detail enhancement, respectively, and then generates the preliminary haze removal result. Furthermore, a novel collaborative refinement mechanism is introduced to encourage the refinement of the coarse result by leveraging the extracted degradation component of the former stage as prior information. In particular, a degradation-aware fusion module is designed to integrate the degradation prior through cross-stage feature aggregation. Extensive experimental results demonstrate the superiority of the proposed method, which outperforms competitive methods by 0.3-1.6 dB in the peak signal-to-noise ratio (PSNR) on the synthetic dataset.

The quality of digital elevation models (DEMs) is easily affected by data voids in regions with complex terrain conditions. Numerous methods have been proposed to fill DEM voids by effectively exploiting the topographic information from neighboring areas or auxiliary DEMs. However, few studies have considered the integration of multi-modal data, which can provide valuable supplementary information in the areas with no high-quality reference DEM data. In this letter, we propose a generative DEM void filling method by exploring the integration of optical remote sensing images. The core idea is to utilize the image textures to infer the elevation values in the void regions with terrain texture-guided transfer learning. Specifically, the image context attention module (ICAM) is used to preliminarily estimate the missing topographic features by searching the similar patches with the guidance of image context. The terrain feature-guided residual pixel attention block (TFG-RPAB) is then employed to refine the void-filled features by transferring the image textures to topographic features. Finally, the void-filled DEM can be obtained by decoding the reconstructed topographic features. The results shows that the RMSE of RSAGAN is improved by 14.5% to 71.5% when DEM void filling. Both quantitative and qualitative evaluations demonstrate the superiority of the proposed method over the competitive methods in terms of DEM void filling. The source code is available at https://github.com/gaobingcug/RSAGAN.

Hyperspectral images (HSIs) with rich spectral information have been widely used in surface classification, object detection, and other real application problems. However, due to the hardware limitations, the low spatial resolution HSIs hinder the exploration of their application potential. Deep learning-based methods are currently the most common solutions for single HSI super-resolution (HSI SR) tasks. However, such methods often overlook the degradation principle from high-resolution HSI to low-resolution HSI. In this article, we propose a CNN-transformer embedded unfolding network (CTUNet), in which an unfolding framework with an effective spatial-spectral prior network is designed for HSI SR by incorporating the degradation principle of HSIs. Specifically, a maximum posterior-based energy model is employed, enabling alternate optimization to seek the optimal solution in an iterative mechanism. To effectively utilize the structure prior of HSI, multiscale self-calibrated convolution (MSSC) and edge-guided transformer module are combined to learn latent spatial-spectral priors. Additionally, hidden feature connections between adjacent iterations enhance the representation of the image features. Extensive experiments conducted on three available HSI datasets demonstrate that our method outperforms several state-of-the-art HSI SR methods. The code will be available athttps://github.com/YoeTon/CTUNet.

Diatom blooms frequently observed in the river tributaries pose a threat to the regional water environment. Satellite remote sensing provides us with an effective tool to delineate the extent of large-scale harmful algal blooms (HABs) in both oceans and inland waters. However, the diatom bloom detection in river systems remains challenging, due to the high demand for a robust model to characterize the spatial and spectral features from the satellite images with high resolution and limited spectral bands. In this study, we developed a novel deep learning-based framework for the detection of the status of diatom blooms in river tributaries using Sentinel-2 MultiSpectral Imager (MSI) images. Distinct from the previous works for detection of bloom extent, the water pixels are categorized as ‘non-bloom’, ‘mild bloom’, and ‘severe bloom’, corresponding to the different bloom intensities. To achieve this, a simple convolutional neural network (CNN) is trained using the collected spectral samples from MSI images characterizing the different bloom statuses. The input features include the spectral variables highly correlated to the pivotal absorption and backscattering properties of diatoms, the chlorophyll-a (Chla), and water temperature. The classification results can then be obtained based on the spectral and environmental characteristics learned by the constructed model. The trained model was extensively tested in tributaries of Yangtze River, where both visual and quantitative validation demonstrated that the proposed model can achieve reliable detection results.

Passive microwave remote sensing can effectively capture the near-surface soil freeze/thaw onsets. Accurately understanding the transition of permafrost freeze/thaw state is helpful for us to respond to climate change in time. In order to improve the retrieval accuracy of freeze/thaw onsets, we propose an XGBoost modeling method that combines SMAP and AMSR2 for freeze/thaw onsets detection. We conducted experiments using data covering Alaska from 2015 to 2020 to demonstrate the effectiveness of our method. The proposed model was applied to the whole study area to obtain the spatial and temporal distribution of freezing periods. During the study period, the shortening of the freezing period has been most evident in 2018–2019. The variation of the freezing period is related to climate anomalies.

Image super-resolution (SR) is an effective solution to the limitation of the spatial resolution of video satellite images, which is caused by the degradation and compression in the imaging phase. For the processing of satellite videos, the commonly employed deep-learning-based single-frame SR (SFSR) framework has limited performance without using complementary information between the video frames. On the other side, the multiframe SR (MFSR) can utilize temporal subpixel information to super-resolve the high-resolution (HR) imagery. However, although deeper and wider deep learning network provides powerful feature representations for SR methods, it has always been a challenge to accurately reconstruct the boundaries of ground objects in video satellite images. In this article, to address these issues, we propose an edge-guided video SR (EGVSR) framework for video satellite image SR, which couples the MFSR model and the edge-SFSR (E-SFSR) model in a unified network. The EGVSR framework is composed of an MFSR branch and an edge branch. The MFSR branch is used to extract the complementary features from the consecutive video frames. Concurrently, the edge branch acts as an SFSR model to translate the edge maps from the low-resolution modality to the HR one. At the final SR stage, the DBFM is built to focus on the promising inner representations of the features of the two branches and fuse them. Extensive experiments on video satellite imagery show that the proposed EGVSR method can achieve superior performance compared to the representative deep-learning-based SR methods.

Vegetation water content (VWC) is recognized as an important parameter in vegetation growth study. Recently, the ground-based GNSS-R method is emerging in monitoring VWC owing to its high accuracy. However, the small footprint and sparse distribution hinder its application. Therefore, we propose a method to improve the spatial resolution of GNSS-R VWC products by downscaling with other products highly correlated with VWC, using random forest (RF). Satisfactory downscaling results with cross-validation R values of 0.83 and RMSE of 0.025 were obtained. VWC images with a 500-m spatial resolution were then acquired, which is consistent with the distribution of NDVI and GPP, further indicating the accuracy of the downscaling results.

In the commonly employed regularization models of image restoration and super-resolution (SR), the norm determination is often challenging. This paper proposes a method to adaptively determine the optimal norms for both fidelity term and regularization term in the (SR) restoration model. Inspired by a generalized likelihood ratio test, a piecewise function is proposed to solve the norm of the fidelity term. This function can find the stable norm value in a certain number of iterations, regardless of whether the noise type is Gaussian, impulse, or mixed. For the regularization norm, the main advantage of the proposed method is that it is locally adaptive. Specifically, it assigns different norms for different pixel locations, according to the local activity measured by a structure tensor metric. The proposed method was tested using different types of images. The experimental results and error analyses verify the efficacy of the method.

As a significant digital representation of terrain surface, varieties of DEM products have been available to the public. The most widely used global DEM products are SRTM and ASTER GDEM. Given the comparable horizontal resolution and vertical error, accuracy validation and comparison have been of interest since the release, however, usually on a wide range. In this paper, we presented the results of accuracy assessment for ASTER GDEM v2 and SRTM v4.1 in Yulong Mountain, Yunnan province, China. Topographic map was chosen as the benchmark. The results and discussions were centered on the relationship between error distribution in elevation and mountainous hypsography based on data causes. The results revealed their levels of reliability for applied glaciology and hydrology in the typical snow mountain area.

The digital elevation model (DEM) is a significant digital representation of a terrain surface. Although a variety of DEM products are available, they often suffer from problems varying in spatial coverage, data resolution, and accuracy. However, the multi-source DEMs often contain supplementary information, which makes it possible to produce a higher-quality DEM through blending the multi-scale data. Inspired by super-resolution (SR) methods, we propose a regularized framework for the production of high-resolution (HR) DEM data with extended coverage. To deal with the registration error and the horizontal displacement among multi-scale measurements, robust data fidelity with weighted norm is employed to measure the conformance of the reconstructed HR data to the observed data. Furthermore, a slope-based Markov random field (MRF) regularization is used as the spatial regularization. The proposed method can simultaneously handle complex terrain features, noises, and data voids. Using the proposed method, we can reconstruct a seamless DEM data with the highest resolution among the input data, and an extensive spatial coverage. The experiments confirmed the effectiveness of the proposed method under different cases.