Large Language Models (LLMs), such as the GPT series, have achieved remarkable performance in question answering through large-scale pretraining. However, LLMs often lack transparency in their reasoning processes and struggle with hallucination. To overcome these challenges, we propose Dual-NeSy, a Dual-system framework that integrates Neural networks with Symbolic logic for interpretable question answering. Specifically, Dual-NeSy leverages fast, heuristic reasoning for knowledge composition (System 1) and utilizes structured, multi-step symbolic reasoning (System 2) for more deliberate, logical verification. This combined approach enhances both the interpretability and accuracy of reasoning. Our approach outperforms previous methods in commonsense reasoning and reading comprehension tasks, achieving state-of-the-art results on three benchmark datasets: QASC, WorldTree, and WikiHop.

Many question-answering problems can be approached as textual entailment tasks, where the hypotheses are formed by the question and candidate answers, and the premises are derived from an external knowledge base. However, current neural methods often lack transparency in their decision-making processes. Moreover, first-order logic methods, while systematic, struggle to integrate unstructured external knowledge. To address these limitations, we propose a neuro-symbolic reasoning framework calledFinal, which combinesFIrst-order logic withNAturalLogic for question answering. Our framework utilizesfirst-order logicto systematically decompose hypotheses andnatural logicto construct reasoning paths from premises to hypotheses, employing bidirectional reasoning to establish links along the reasoning path. This approach not only enhances interpretability but also effectively integrates unstructured knowledge. Our experiments on three benchmark datasets, namely QASC, WorldTree, and WikiHop, demonstrate thatFinaloutperforms existing methods in commonsense reasoning and reading comprehension tasks, achieving state-of-the-art results. Additionally, our framework also provides transparent reasoning paths that elucidate the rationale behind the correct decisions.

Hyperspectral anomaly detection (HAD) aims to find small targets deviating from surroundings in an unsupervised manner. Recently, various deep models have been applied to HAD, such as autoencoder series and generative adversarial networks (GAN) series, which mainly use a proxy task, i.e., iteratively reconstructing low-frequency components (backgrounds) to separate anomalies (two-step paradigm). However, in such an unsupervised manner, most deep HAD model is trained and tested on the same image. Since the learned low-frequency background varies from image to image and the trained model cannot be directly transferred to unseen images. In this paper, the one-step detection paradigm is first proposed, where the model is optimized directly for the HAD task and can be transferred to unseen datasets. The one-step paradigm is optimized to identify the spectral deviation relationship according to the anomaly definition. Compared to learning the specific background distribution in the two-step paradigm, the spectral deviation relationship is universal for different images and guarantees transferability. Further, we instantiated the one-step paradigm as an unsupervised transferred direct detection (TDD) model. To train the TDD model in an unsupervised manner, an anomaly sample simulation strategy is proposed to generate numerous pairs of anomaly samples. A global self-attention module and a local self-attention module are designed to help the model focus on the “spectrally deviating” relationship. The TDD model was validated on six public datasets. The results show that TDD is superior to the recent two-step methods in detection and transferability aspects.

Earth anomalies can locate valuable targets in an unsupervised manner for many defense and surveillance applications. Most models assign a continuous score at the pixel-level, resulting in object-agnostic results with higher false alarms than the instance level results. However, since the anomaly objects contain a variety of categories and have a large intraclass variance, the current state-of-the-art (SOTA) query-based models designed for certain categories perform unsatisfactorily when applied to the anomaly instances. The larger intraclass variance of anomalies makes the learning of general representation more difficult. To bridge this gap, we propose general adaptations guided by the pixel-level anomaly map for any query-based model, which adapts the model from learning certain category representation to learning anomaly-aware representation in different categories. The proposed adaptation first builds a separate branch to output the pixel-level anomaly map, where anomaly information is then extracted to guide the pixel embeddings and queries to focus on a variety of anomaly categories. Especially, the anomaly rank embeddings are devised to make the pixel embeddings aware of the anomaly rank order. The queries are dynamically selected from the anomaly candidates after aligning the anomaly map and pixel embeddings for better locating different anomalies. Finally, the selected queries dot-product the anomaly-aware pixel embeddings to output the anomaly instances. The proposed adaptations are simple, general, and additive, which bring the average improvements of +4.9 box AP and +5.1 mask AP in infrared, synthetic aperture radar (SAR), and hyperspectral modalities.

Positive and unlabeled (PU) learning is aimed at building a binary classifier to distinguish the target from the background using only the known positive samples, which is an advanced solution for the hyperspectral target detection (HTD) task. However, when PU learning (PUL) meets complex hyperspectral scenarios, there are two main challenges: 1) How to estimate the class prior accurately? The class prior, i.e., the target proportion, is an important prior for PUL to learn the discriminant boundary, but it is difficult to estimate in hyperspectral imagery, due to the interclass spectral similarity and 2) How to remove redundancy and improve the discriminative features of the target? The diagnostic spectral feature extraction is important for the weakly supervised PUL models as it can help with separating the target from the background. In this article, to tackle these challenges, a robust PUL framework with key band selection (PU-KBS) is proposed, which is modeled as an end-to-end and class prior free PUL framework, where the accurate class prior and the most discriminative key band subset are jointly initialized and iteratively updated until reaching the optimal result by evolutionary search. Meanwhile, a deep PUL detector is introduced for guiding the subsequent search direction and discriminative deep feature extraction. The proposed PU-KBS framework was verified using different hyperspectral datasets, where accurate class prior estimation, diagnostic spectral characteristics, and robust detection results could be obtained simultaneously by the PU-KBS framework. Furthermore, the improvement in band selection interpretability and detection performance was proven experimentally.

An inevitable trend of hyperspectral remote sensing has been toward hyperspectral with high spatial resolution (H2) images. However, the higher resolution also brings higher spatial/spectral heterogeneity of surface features, which increases the difficulty of fine classification. Fully using global spatial–spectral features and contextual information is an effective method to alleviate spatial/spectral heterogeneity. Recently, to extract global spatial–spectral features with long-range dependencies, the self-attention mechanism has been widely used in H2 image classification and has achieved excellent results. As is well known, the simultaneous use of spatial and spectral information has always been a key aspect of hyperspectral image (HSI) processing; however, the current spatial and spectral attention modules only focus on the spatial and spectral features separately. This prevents further improvement in network performance, especially when the sample size is small. Therefore, a spatial–spectral attention-in-attention network (S2AiANet) is proposed, which solves the problem of the current spatial–spectral attention maps only focusing on single features through the spatial–spectral attention-in-attention (S2AiA) module. In addition, a multiscale attention (MSA) module is proposed to enhance the network’s adaptability to various complex scenarios. The experiments on two H2 datasets and one classic HSI dataset demonstrate that S2AiANet can achieve a significant performance improvement compared with the state-of-the-art hyperspectral classifiers.

Remote sensing anomaly detector can find the objects deviating from the background as potential targets for Earth monitoring. Given the diversity in earth anomaly types, designing a transferring model with cross-modality detection ability should be cost-effective and flexible to new earth observation sources and anomaly types. However, the current anomaly detectors aim to learn the certain background distribution, the trained model cannot be transferred to unseen images. Inspired by the fact that the deviation metric for score ranking is consistent and independent from the image distribution, this study exploits the learning target conversion from the varying background distribution to the consistent deviation metric. We theoretically prove that the large-margin condition in labeled samples ensures the transferring ability of learned deviation metric. To satisfy this condition, two large margin losses for pixel-level and feature-level deviation ranking are proposed respectively. Since the real anomalies are difficult to acquire, anomaly simulation strategies are designed to compute the model loss. With the large-margin learning for deviation metric, the trained model achieves cross-modality detection ability in five modalities-hyperspectral, visible light, synthetic aperture radar (SAR), infrared and low-light-in zero-shot manner.

Anomaly segmentation in high spatial resolution (HSR) remote sensing imagery is aimed at segmenting anomaly patterns of the earth deviating from normal patterns, which plays an important role in various Earth vision applications. However, it is a challenging task due to the complex distribution and the irregular shapes of objects, and the lack of abnormal samples. To tackle these problems, an anomaly segmentation model based on pixel descriptors (ASD) is proposed for anomaly segmentation in HSR imagery. Specifically, deep one-class classification is introduced for anomaly segmentation in the feature space with discriminative pixel descriptors. The ASD model incorporates the data argument for generating virtual abnormal samples, which can force the pixel descriptors to be compact for normal data and meanwhile to be diverse to avoid the model collapse problems when only positive samples participated in the training. In addition, the ASD introduced a multi-level and multi-scale feature extraction strategy for learning the low-level and semantic information to make the pixel descriptors feature-rich. The proposed ASD model was validated using four HSR datasets and compared with the recent state-of-the-art models, showing its potential value in Earth vision applications.

Positive-unlabeled learning (PU learning) in hyperspectral remote sensing imagery (HSI) is aimed at learning a binary classifier from positive and unlabeled data, which has broad prospects in various earth vision applications. However, when PU learning meets limited labeled HSI, the unlabeled data may dominate the optimization process, which makes the neural networks overfit the unlabeled data. In this paper, a Taylor variational loss is proposed for HSI PU learning, which reduces the weight of the gradient of the unlabeled data by Taylor series expansion to enable the network to find a balance between overfitting and underfitting. In addition, the self-calibrated optimization strategy is designed to stabilize the training process. Experiments on 7 benchmark datasets (21 tasks in total) validate the effectiveness of the proposed method. Code is at: https://github.com/Hengwei-Zhao96/T-HOneCls.

Hyperspectral imagery (HSI) one-class classification is aimed at identifying a single target class from the HSI by using only knowing positive data, which can significantly reduce the requirements for annotation. However, when one-class classification meets HSI, it is difficult for classifiers to find a balance between the overfitting and underfitting of positive data due to the problems of distribution overlap and distribution imbalance. Although deep learning-based methods are currently the mainstream to overcome distribution overlap in HSI multi-classificaiton, few researches focus on deep learning-based HSI one-class classification. In this paper, a weakly supervised deep HSI one-class classifier, namelyHOneClsis proposed, where a risk estimator—theOne-Class Risk Estimator—is particularly introduced to make the full convolutional neural network (FCN) with the ability of one class classification in the case of distribution imbalance. Extensive experiments (20 tasks in total) were conducted to demonstrate the superiority of the proposed classifier.

Large scale crop mapping is an important task in agricultural resource monitoring. To obtain a distribution map of crops, traditional methods usually require the well-designed manufacture features and the ground-truth labels of all land-cover types for training a multi-class classifier. However, the redundant labeling for each land-cover type is time-consuming and labor-intensive, and the feature design for different remote sensing data is complex and limited to human prior knowledge. In this paper, a deep one-class crop extraction framework is proposed to solve the problems mentioned above. Specifically, it uses the deep one-class crop extraction module to extract the feature automatically for any remote sensing imagery and the one-class crop extraction loss to address the lack of negative class in the deep one-class classification model. In addition, the proposed framework can be applied to multi-modal remote sensing data, i.e. hyperspectral, multispectral, and SAR images, which is verified in the experiments and the proposed framework can achieve the highest accuracy on each multi-modal data.