Unsupervised Domain Adaptation (UDA) has emerged as a critical research direction in remote sensing (RS) interpretation, aiming to bridge the gap between the labeled source and unlabeled target domains. However, most methods are designed for either a single target domain or multi-domain setting with clear boundaries, which necessitates retraining UDA models for each target domain, making it even harder to directly generalize to unseen domains. This paper proposes a novel two-stage open compound domain adaptation framework for semantic segmentation in RS images, which models the target domain as a composite of multiple unknown but homogeneous domains and leverages image translation and meta-learning techniques. In the first stage, we meticulously design a cross-domain image translation model (CDIT) based on contrastive learning to rapidly align the appearance of target domain images with the source domain style. In the second stage, the translated target images are first processed by a pre-trained model to generate pseudo-labels. Subsequently, a dynamic class-wise memory model (DCWM) is designed to progressively update abstract categorical features, serving as external class guidance for semantic segmentation within a meta-learning framework. Specifically, meta-training is employed to iteratively learn and update domain-agnostic categorical memory of semantic classes, while meta-testing simulates memory retrieval and gradually refines the categorical memory using pseudo-labels to adapt to new domains. Additionally, intra-class cohesion and inter-class divergence losses are incorporated to enhance the abstraction and retention of categorical features, aligning more closely with human cognitive patterns. Extensive experiments on RS benchmarks and unseen real images demonstrate the superior generalization of our method compared to state-of-the-art approaches.

Land surface temperature (LST) serves as a critical biophysical parameter characterizing global-scale surface energy partitioning and water exchange processes. The Split-Window (SW) and Temperature-Emissivity Separation (TES) methods are the most widely used LST retrieval methods, but they rely on accurate land surface emissivity and atmospheric correction as prior knowledge, respectively. While the SW-TES hybrid method mitigates these limitations, atmospheric correction errors in the SW stage can accumulate into the TES stage, thereby affecting the accuracy of LST retrieval. To address this limitation, this study develops an improved SW-TES algorithm that incorporates total column water vapor as a correction factor for surface-leaving radiance calculations. Based on GOES-16 ABI data, we utilized the improved algorithm to retrieve hourly LST estimates for the contiguous United States in 2020 and validated the results against the GOES-16 LST product and ground-based measurements from the Surface Radiation Budget Network (SURFRAD). The retrieved LST exhibits high consistency with the GOES-16 LST product, demonstrating a root mean square error (RMSE) of 1.48 K and a bias of 0.90 K. Based on SURFRAD ground observations, the improved SW-TES algorithm achieves an average RMSE of 2.27 K and a bias of 0.26 K across seven validation sites. In comparison, the GOES-16 LST product shows an RMSE of 2.86 K and a bias of –1.21 K at the same sites. Furthermore, the algorithm maintains superior stability under high temperature and high water vapor content conditions.

Understanding human mobility behavior is crucial for numerous applications, including crowd management, location-based recommendations, and the estimation of pandemic spread. Machine learning models can predict the Points of Interest (POIs) that individuals are likely to visit in the future by analyzing their historical visit patterns. Previous studies address this problem by learning a POI classifier, where each class corresponds to a POI. However, this limits their applicability to predict a new POI that was not in the training data, such as the opening of new restaurants. To address this challenge, we propose a model designed to predict a new POI outside the training data as long as its context is aligned with the user’s interests. Unlike existing approaches that directly predict specific POIs, our model first forecasts the semantic context of potential future POIs, then combines this with a proximity-based prior probability distribution to determine the exact POI. Experimental results on real-world visit data demonstrate that our model outperforms baseline methods that do not account for semantic contexts, achieving a 17% improvement in accuracy. Notably, as new POIs are introduced over time, our model remains robust, exhibiting a lower decline rate in prediction accuracy compared to existing methods.

Although Deep Learning based methods for remote sensing (RS) image interpretation have reported promising results, the domain gap between RS images and the absence of sensor-specific labeled datasets result in the significant deterioration of well-trained models to adapt to new images. In practical applications, we propose an RS image translation method based on contrastive learning (CDTCL) to quickly achieve the data simulation conversion for different sensors and domains. Specifically, for unpaired images, we design a content contrastive loss for content consistency constraints and a style contrastive loss for swift alignment of appearance style. Additionally, we integrate the semantic segmentation model, a flexible model that can be retrained at the discretion of users, into the image translation framework to establish a complementary closed loop. Extensive experiments on aerial images, including visible and infrared images, verify that our method works effectively in cross-domain semantic segmentation and achieves the best performance.

Sudden-onset Earth’s surface anomalies, such as natural disasters and man-made incidents, pose severe threats to human life and property security, emphasizing the crucial role of accurate detection and rapid response in Humanitarian Assistance and Disaster Response (HADR). In this work, we propose a hierarchical graph neural network (GNN) based framework for Earth’s surface anomaly detection, called L2S-Net, to integrate from local to semantic (L2S) information for rapid and accurate detection of multi-class anomalies. Specifically, L2S-Net only utilizes a single very high-resolution (VHR) image as input to expedite processing speed, while employing a hierarchical graph representation for better image understanding. Meanwhile, drawing from brain-inspired research and graph theory, we design a local-to-semantic fusion network, called L2S-GNN, to explicitly learn relationships between nodes at different levels facilitating accurate detection of Earth’s surface anomaly. L2S-Net significantly reduces data requirements while capturing valuable higher-order information from images, achieving a superior balance between accuracy and efficiency. Furthermore, due to the lack of a public dataset for Earth’s surface anomaly detection, we create a novel and large-scale benchmark dataset ESADv2. Extensive experiments on the ESADv2 dataset and two real-world cases demonstrate that the proposed L2S-Net outperforms many state-of-the-art methods in both model size and performance while exhibiting exceptional generalizability and robustness.

Few-shot learning enables rapid generalization from extremely limited training examples. While previous efforts have utilized meta-learning or data augmentation methods to mitigate the problem of data scarcity, such approaches may struggle to maintain robustness and generalize effectively due to overfitting and noise sensitivity. In this paper, we propose a novel approach, the Semi-Supervised Label Correction method for Few-Shot Learning (SSLC-FSL), which leverages the data distribution of readily available and easily obtainable unlabeled data. SSLC-FSL iteratively corrects the labels of testing samples with alternating steps of pseudo-labeling and sample selection. The objective of pseudo-labeling is to repurpose graph-based semi-supervised learning for joint prediction of the entire testing set. We then introduce a Modulation Selection Network (MSN) to rank testing samples by learning with noisy labels. The training set is expanded by selecting confident pseudo-labeled samples. In the MSN, a Modulation Aggregation Layer is designed to encode support class information into each testing sample, thereby highlighting target category features and mitigating the negative impact of incorrect labels. The iterative label correction process is repeated until all testing samples are recalled to the expanded support set. To boost the SSLC-FSL algorithm, we pre-train a feature extractor to produce general-purpose representations. Particularly, we investigate two types of auxiliary tasks and their collaborative learning to acquire transferable visual information via an end-to-end multi-task learning model. Our SSLC-FSL outperforms current state-of-the-art methods in any shot and all data settings, with up to +27.74% on standard remote sensing benchmarks and +5.70% on standard natural scene benchmarks.

Video satellites observe the Earth with high spatial-temporal resolution, empowering remote sensing with better capability for emergency needs. The tracking of moving targets naturally has received a great deal of attention. Many methods are proposed and achieve satisfying results after years of research, where the low-rank representation (LRR) shows great potential due to its weak assumptions on data. However, the original LRR lacks the consideration of frame-wise geometric inconsistency led by imperfect video stabilization, significant view change, and elevation difference, resulting in pseudo-motion and declined performance on some video sequences. Therefore, we propose a geometry-aware LRR (Geo-LRR) that simultaneously considers the alignment and the target tracking in videos. Moreover, to relieve the dilemma of high-quality samples, we generate pixel-wise annotations with straightforward post-processing on sparse images. Extensive experiments demonstrate the effectiveness of our method with 2.051 pixel-wise RMSE compared with groundtruth.

Deep learning (DL) based methods for motion deblurring, taking advantage of large-scale datasets and sophisticated network structures, have reported promising results. However, two challenges still remain: existing methods usually perform well on synthetic datasets but cannot deal with complex real-world blur, and in addition, over- and under-estimation of the blur will result in restored images that remain blurred and even introduce unwanted distortion. We propose a motion deblurring framework that includes a Blur Space Disentangled Network (BSDNet) and a Hierarchical Scale-recurrent Deblurring Network (HSDNet) to address these issues. Specifically, we train an image blurring model to facilitate learning a better image deblurring model. Firstly, BSDNet learns how to separate the blur features from blurry images, which is adaptable for blur transferring, dataset augmentation, and ultimately directing the deblurring model. Secondly, to gradually recover sharp information in a coarse-to-fine manner, HSDNet makes full use of the blur features acquired by BSDNet as a priori and breaks down the non-uniform deblurring task into various subtasks. Moreover, the motion blur dataset created by BSDNet also bridges the gap between training images and actual blur. Extensive experiments on real-world blur datasets demonstrate that our method works effectively on complex scenarios, resulting in the best performance that significantly outperforms many state-of-the-art approaches.

Due to the critical role of pavement crack detection for road maintenance and eventually ensuring safety, remarkable efforts have been devoted to this research area, and such a trend is further intensified for the coming unmanned vehicle era. However, such crack detection task still remains unexpectedly challenging in practice since the appearance of both cracks and the background are diverse and complex in real scenarios. In this work, we propose an automatic pavement crack detection method via synergizing low rank representation (LRR) and deep learning techniques. First, leveraging LRR which facilitates anomaly detection without making any specific assumption, we can easily discriminate most of the frames with cracks from the long sequence with a consistent pavement base, followed by a straightforward algorithm to localize the cracks. In order to achieve the intelligence of detecting cracks with different pavement basis under unconstrained imaging conditions, we resort to deep learning techniques and propose a deep convolutional neural network for crack detection leveraging on multi-level features and atrous spatial pyramid pooling (ASPP). We train this network based on the training data obtained in the previous stage in an end-to-end manner. Extensive experiments on a wide range of pavements demonstrate the high performance in terms of both accuracy and automaticity. Moreover, the dataset generated by us is much more extensive and challenging than public ones. We put it online athttps://gaozhinuswhu.comto benefit the community.

Many successful methods have been proposed for learning low-dimensional representations on large-scale networks, while almost all existing methods are designed in inseparable processes, learning embeddings for entire networks even when only a small proportion of nodes are of interest. This leads to great inconvenience, especially on large-scale or dynamic networks, where these methods become almost impossible to implement. In this paper, we formalize the problem of separated matrix factorization, based on which we elaborate a novel objective function that preserves both local and global information. We compare our SMF framework with approximate SVD algorithms and demonstrate SMF can capture more information when factorizing a given matrix. We further propose SepNE, a simple and flexible network embedding algorithm which independently learns representations for different subsets of nodes in separated processes. By implementing separability, our algorithm reduces the redundant efforts to embed irrelevant nodes, yielding scalability to large networks. To further incorporate complex information into SepNE, we discuss several methods that can be used to leverage high-order proximities in large networks. We demonstrate the effectiveness of SepNE on several real-world networks with different scales and subjects. With comparable accuracy, our approach significantly outperforms state-of-the-art baselines in running times on large networks.

In this work, an inverse domain adaptation (IDA) method is proposed to cope with the distributional mismatch between the training images in the source domain and the test images in the target domain in remote sensing. More specifically, a cycleGAN structure using the Wasserstein distance is developed to learn the distribution of the remote sensing images in the source domain before the images in the target domain are transformed into similar distribution while preserving the image details and semantic consistency of the target images via style transfer. Extensive experiments using the GF1 data are performed to confirm the effectiveness of the proposed IDA method.

Molecular representation learning is the first yet vital step in combining deep learning and molecular science. To push the boundaries of molecular representation learning, we present PhysChem, a novel neural architecture that learns molecular representations via fusing physical and chemical information of molecules. PhysChem is composed of a physicist network (PhysNet) and a chemist network (ChemNet). PhysNet is a neural physical engine that learns molecular conformations through simulating molecular dynamics with parameterized forces; ChemNet implements geometry-aware deep message-passing to learn chemical / biomedical properties of molecules. Two networks specialize in their own tasks and cooperate by providing expertise to each other. By fusing physical and chemical information, PhysChem achieved state-of-the-art performances on MoleculeNet, a standard molecular machine learning benchmark. The effectiveness of PhysChem was further corroborated on cutting-edge datasets of SARS-CoV-2.

Learning low-dimensional representations on graphs has proved to be effective in various downstream tasks. However, noises prevail in real-world networks, which compromise networks to a large extent in that edges in networks propagate noises through the whole network instead of only the node itself. Whereas existing methods tend to focus on preserving structural properties, the robustness of the learned representations against noises is generally ignored. In this paper, we propose a novel framework to learn noise-free node representations and eliminate noises simultaneously. Since noises are often unknown on real graphs, we design two generators, namely a graph generator and a noise generator, to identify normal structures and noises in an unsupervised setting. On the one hand, the graph generator serves as a unified scheme to incorporate any useful graph prior knowledge to generate normal structures. We illustrate the generative process with community structures and power-law degree distributions as examples. On the other hand, the noise generator generates graph noises not only satisfying some fundamental properties but also in an adaptive way. Thus, real noises with arbitrary distributions can be handled successfully. Finally, in order to eliminate noises and obtain noise-free node representations, two generators need to be optimized jointly, and through maximum likelihood estimation, we equivalently convert the model into imposing different regularization constraints on the true graph and noises respectively. Our model is evaluated on both real-world and synthetic data. It outperforms other strong baselines for node classification and graph reconstruction tasks, demonstrating its ability to eliminate graph noises.

Many successful methods have been proposed for learning low dimensional representations on large-scale networks, while almost all existing methods are designed in inseparable processes, learning embeddings for entire networks even when only a small proportion of nodes are of interest. This leads to great inconvenience, especially on super-large or dynamic networks, where these methods become almost impossible to implement. In this paper, we formalize the problem of separated matrix factorization, based on which we elaborate a novel objective function that preserves both local and global information. We further propose SepNE, a simple and flexible network embedding algorithm which independently learns representations for different subsets of nodes in separated processes. By implementing separability, our algorithm reduces the redundant efforts to embed irrelevant nodes, yielding scalability to super-large networks, automatic implementation in distributed learning and further adaptations. We demonstrate the effectiveness of this approach on several real-world networks with different scales and subjects. With comparable accuracy, our approach significantly outperforms state-of-the-art baselines in running times on large networks.

Graph Convolutional Networks (GCNs) have proved to be a most powerful architecture in aggregating local neighborhood information for individual graph nodes. Low-rank proximities and node features are successfully leveraged in existing GCNs, however, attributes that graph links may carry are commonly ignored, as almost all of these models simplify graph links into binary or scalar values describing node connectedness. In our paper instead, links are reverted to hypostatic relationships between entities with descriptional attributes. We propose GCN-LASE (GCN with Link Attributes and Sampling Estimation), a novel GCN model taking both node and link attributes as inputs. To adequately captures the interactions between link and node attributes, their tensor product is used as neighbor features, based on which we define several graph kernels and further develop according architectures for LASE. Besides, to accelerate the training process, the sum of features in entire neighborhoods are estimated through Monte Carlo method, with novel sampling strategies designed for LASE to minimize the estimation variance. Our experiments show that LASE outperforms strong baselines over various graph datasets, and further experiments corroborate the informativeness of link attributes and our model's ability of adequately leveraging them.