Hyperspectral imaging (HSI) has been widely used in agricultural applications for non-destructive estimation of plant nutrient composition and precise quantification of sample nutritional elements. Recently, 3D reconstruction methods, such as Neural Radiance Field (NeRF), have been used to create implicit neural representations of HSI scenes. This capability enables the rendering of hyperspectral channel compositions at every spatial location, thereby helping localize the target object's nutrient composition both spatially and spectrally. However, it faces limitations in training time and rendering speed. In this paper, we propose Diffusion-Denoised Hyperspectral Gaussian Splatting (DD-HGS), which enhances the state-of-the-art 3D Gaussian Splatting (3DGS) method with wavelength-aware spherical harmonics, a Kullback-Leibler divergence-based spectral loss, and a diffusion-based denoiser to enable 3D explicit reconstruction of the hyperspectral scenes for the entire spectral range. We present extensive evaluations on diverse real-world hyperspectral scenes from the Hyper-NeRF dataset to show the effectiveness of our DD-HGS. The results demonstrate that DD-HGS achieves the new state-of-the-art performance compared to all the previously published methods. Project page: https://dragonpg2000.github.io/DDHGS-website/

Faithful free-text explanations are important to ensure transparency in high-stakes AI decisionmaking contexts, but they are challenging to generate by language models and assess by humans.In this paper, we present a measure for Prediction-EXplanation (PEX) consistency, by extending the concept of weight of evidence.This measure quantifies how much a free-text explanation supports or opposes a prediction, serving as an important aspect of explanation faithfulness.Our analysis reveals that more than 62% explanations generated by large language models lack this consistency.We show that applying direct preference optimization improves the consistency of generated explanations across three model families, with improvement ranging from 43.1% to 292.3%.Furthermore, we demonstrate that optimizing this consistency measure can improve explanation faithfulness by up to 9.7%. 1

The accurate imputation of missing load data in building energy consumption is essential for optimizing energy management and scheduling in Internet of Things (IoT)-based smart energy management systems. However, in real-world applications, building load data often suffers from the issue of missing critical samples due to IoT device failures and maintenance. To address this problem, we propose an effective scheme by designing a load data augmentation model named the DAM based on deep neural networks. In the DAM, the partial missing data are generated in each round, followed by stacking with the semi-dataset to perform a new generation round. After several rounds, the missing critical load data are recovered with high precision. A building load dataset collected from a real IoT-based energy-efficiency management system is used for evaluation in this work. Experimental results demonstrate that the proposed scheme can effectively replenish the missing critical data and exhibit excellent stability. Additionally, we compare the prediction performance of the DAM approach with other comparison methods. The results show that our proposed approach outperforms the comparison methods, achieving the highest R2 score of 0.963. Hence, the DAM approach presents an effective solution for addressing the problem of missing critical data in IoT-based smart energy management systems, which is vital for optimizing energy dispatch.

Pixel-level annotation tasks are important in the intelligent processing of remote sensing images. For these tasks, Interactive Image Segmentation (IIS) models using click prompts are developing fast in the field of natural images. However, most interactive segmentation models using click prompts are unsuitable for remote sensing images with their current design of click prompts and their interaction schemes with image information. Based on the situation, we used a DETR-like model as the basic framework and redesigned the pixel decoder and the transformer decoder to better suit the task of IIS for remote sensing images. In the pixel decoder, we designed a click prompt with feature encoding to learn click information and a composite attention structure to facilitate interaction between click and image information, allowing the image feature at the click locations to more easily dominate annotation masks. In the transformer decoder, we utilized deformable attention, using only a single initialized query to obtain annotation masks and IoU prediction. In this paper, we trained our model on a composite remote sensing dataset and evaluated its performance on external datasets. The results showcased the model’s adaptability, achieving superior performance compared to existing methods. The code will be available at https://github.com/songbingze/ClickPromptRSIIS.

In the open-source community, selecting models that meet user requirements and data distributions is essential due to numerous models with unique characteristics. However, existing model search methods often fail to meet diverse user requirements, varied data distributions, and have slow search speeds. To address these issues, we introduce ModelCS, a two-stage framework for model search based on recall-ranking. Its key idea is to preliminary screening of numerous models using representation learning and then precise ranking of selected ones. Specifically, we study model feature extraction and representation methods. We construct a dataset for this study and propose a rule-based data augmentation method to enhance its diversity. Based on the augmented dataset, we conduct an empirical study and propose the multidimensional feature representation, which influences the design of ModelCS. The recall stage of ModelCS involves a preliminary screening method based on the multidimensional feature representation, while the ranking stage of ModelCS involves a ranking method based on the extension to an existing method. We evaluate ModelCS on the multi-task model zoo in the PaddlePaddle framework. Experimental results indicate that ModelCS can reduce search time by up to 500 times and improve search effectiveness by up to 13.27 % compared to existing methods.

In the field of 3D object detection for autonomous driving, LiDAR-Camera (LC) fusion is the top-performing sensor configuration. Still, LiDAR is relatively high cost, which hinders adoption of this technology for consumer automobiles. Alternatively, camera and radar are commonly deployed on vehicles already on the road today, but performance of Camera-Radar (CR) fusion falls behind LC fusion. In this work, we propose Camera-Radar Knowledge Distillation (CRKD) to bridge the performance gap between LC and CR detectors with a novel cross-modality KD framework. We use the Bird'View (BEV) representation as the shared feature space to enable effective knowledge distillation. To accommodate the unique cross-modality KD path, we propose four distillation losses to help the student learn crucial features from the teacher model. We present extensive evaluations on the nuScenes dataset to demonstrate the effectiveness of the proposed CRKD framework. The project page for CRKD is https://song-jingyu.github.io/CRKD.

Language models will inevitably err in situations with which they are unfamiliar.However, by effectively communicating uncertainties, they can still guide humans toward making sound decisions in those contexts.We demonstrate this idea by developing HEAR, a system that can successfully guide humans in simulated residential environments despite generating potentially inaccurate instructions.Diverging from systems that provide users with only the instructions they generate, HEAR warns users of potential errors in its instructions and suggests corrections.This rich uncertainty information effectively prevents misguidance and reduces the search space for users.Evaluation with 80 users shows that HEAR achieves a 13% increase in success rate and a 29% reduction in final location error distance compared to only presenting instructions to users.Interestingly, we find that offering users possibilities to explore, HEAR motivates them to make more attempts at the task, ultimately leading to a higher success rate.To our best knowledge, this work is the first to show the practical benefits of uncertainty communication in a long-horizon sequential decision-making problem. 1

We propose LiRaFusion to tackle LiDAR-radar fusion for 3D object detection to fill the performance gap of existing LiDAR-radar detectors. To improve the feature extraction capabilities from these two modalities, we design an early fusion module for joint voxel feature encoding, and a middle fusion module to adaptively fuse feature maps via a gated network. We perform extensive evaluation on nuScenes to demonstrate that LiRaFusion leverages the complementary information of LiDAR and radar effectively and achieves notable improvement over existing methods.

Deep learning (DL) techniques have recently ignited remarkable prosperity in the Synthetic Aperture Radar Automatic Target Recognition (SAR ATR) field. Nevertheless, as targets of new categories are observed continually with few-shot examples in openly dynamic scenarios, endowing the DL-based SAR ATR systems with Few-Shot Class-Incremental Learning (FSCIL) ability is urgently demanded. In response, a Decoupled Scattering Augmentation Classifier (DSAC) is proposed to mitigate both intrinsic and domain-specific challenges of the FSCIL of SAR ATR. Specifically, as the significant partability of target structures in SAR imagery, virtual targets with potential scattering patterns are synthesized and pre-allocated by a Scattering Augmentation Module (SAM) to unleash the model’s forward compatibility for future categories. Once deployed, the DSAC is decoupled with dynamic worlds for prompt knowledge representation. Also, a prototypical Nearest-Class-Mean (NCM) classifier with cosine criterion is leveraged for stable and general identification. Extensive experiments conducted on an FSCIL of SAR ATR dataset verify the superiority of our method compared to various latest benchmarks.

Metaverse seamlessly integrates the real world with the virtual world and allows avatars to carry out rich activities including creation, display, entertainment, social, and trading. It integrates the most fundamental technologies, such as Blockchain , Interaction, Games, Artificial Intelligence, Networks, and the Internet of Things , named BIGANT. Interaction technologies are significant to allow users to interact with virtual entities in physical environments via sensors, such as AR, MR, and VR. However, there are still great challenges regarding how to access the metaverse in a more intelligent, faster, and effective way, especially in capturing human positions and activities. Intelligent wireless sensing technology, integrating AI , can serve as an intelligent, flexible, non-contact way to access the metaverse and expedite the establishment of a bridge between the real physical world and the metaverse. Hence, this paper elaborates on the existing work and discusses potential important trends and hotspots in wireless sensing, especially localization, activity recognition, and pattern analysis. After that, we discussed how intelligent wireless sensing will evolve in the metaverse, together with current challenges and open issues in this topic. Through this overview, we wish readers can better understand how intelligent wireless sensing accelerates the accessing to metaverse and the insights behind the wireless sensing in the metaverse.

Numerous studies have been conducted on missing data recovery in remote sensing images, such as cloud removal and dead pixels restoration. Nevertheless, reconstructing continuous, extensive, and complete missing areas still poses a significant challenge. In this letter, we propose a new architecture named Extend-generative adversarial network (GAN), which leverages only a low-resolution image with relaxed requirements on spatial resolution and acquisition time as a condition to reconstruct a high-resolution image with large-scale missing areas. We equip Extend-GAN with learnable adaptive region normalization (LARN) to adjust the intensity distribution of pixels to reduce color distortion. We also introduce a new loss function into the training process of Extend-GAN, namely the structural similarity (SSIM)-based triplet loss, which helps to preserve the between missing parts and known regions. Gaofen-2 and Landsat-9 image pairs are used to validate the proposed method. Extend-GAN performs better when comprehensively evaluated on visual effect, quantitative metrics, processing speed, etc. Code is available athttps://github.com/yc-cui/Extend-GAN.

Fine-grained aircraft detection from synthetic aperture radar (SAR) imagery is of significance in transportation and military domains. Based on the prior knowledge that aircraft are frequently found in airports, current research adopts airport-to-aircraft detection pipelines for aircraft detection and classification in panoramic SAR images. Geospatial information, including the approximate airport location and the spatial relationships between the airport and aircraft, represents valuable supplementary information that can enhance fine-grained aircraft detection performance. However, due to unreliable geographical information and the limited perceptual field, it is challenging to detect and classify aircraft in panoramic SAR imagery. To address this, a novel geospatial contextual prior-enabled knowledge reasoning framework is proposed. First, an unsupervised and lightweight geospatial-driven airport detection (AD) method is presented by combining geographic information matching and optical-to-SAR image registration, which can quickly locate airports and narrow the scope for fine detection. Then, an improved real-time model for object detection with a recursive-gated spatial interaction module (RTMDet-RSIM) is proposed for fine-grained aircraft detection. RTMDet-RSIM uses frequency-domain analysis to improve global information modeling ability without excessive computational cost. Finally, a relational prior-based reasoning strategy is proposed by modeling the geospatial category relationships from historical images to strengthen classification performance. Experiments on 61 panoramic SAR images covering 13 aircraft categories show that the proposed method achieves high detection accuracy with a mean average precision (mAP) of 81.2%, while the average test time for an image size of$8738\times 7636$is about 8.58 s. The source code and dataset will be released.

With the rapid acquisition of high-resolution Synthetic Aperture Radar(SAR) images, new categories are continually observed with few-shot instances in openly non-cooperative scenarios. Powering a SAR Automatic Target Recognition (SAR ATR) system with an ability of few-shot class-incremental learning (FSCIL) is nontrivial. Observing the pronounced azimuth-dependence and part-sparsity of targets in SAR images, an Azimuth-aware Subspace Classifier (AASC) on the Grassmannian manifold is proposed to tackle the FSCIL of SAR ATR stably and accurately. In the AASC, losses covering both semantic and manifold facets, which include Semantic Margin Separation (SMS), Deep Subspace Separation (DSS), and Structure Less Forgetting (SLF), are designed to strike both the intrinsic model’s stability and plasticity dilemma and domain-specific challenges. For plasticity, the novel-to-old semantic margins are enlarged by the SMS loss for knowledge transferring while avoiding inappropriate adaptions. The DSS loss derived from the Grassmannian geometry aims to regularize class subspaces orthogonality. For stability, semantic drifts of target spatial and global structures are punished by the SLF loss. As the periodicity and volatility of target azimuth-aware patterns, an Azimuth-aware Exemplar Selection (AES) strategy is designed to select representative and complementary exemplars. In experiments, the advantages of the subspace classifier and the designed losses and strategies are deeply verified. Comprehensive experiments on three FSCIL scenarios derived from both airborne and spaceborne datasets, including the MSTAR, the SAR-AIRcraft-1.0, and self-collected data sets, show that our method significantly outperforms various task-specific benchmarks, verifying its effectiveness for the FSCIL in real SAR ATR scenarios.

Detecting ships in synthetic aperture radar (SAR) images poses a formidable challenge, primarily attributed to limited observation samples and complex environments. To address this problem, driven by latent similarity between optical and SAR images, we propose a domain-adaptive few-shot detection algorithm for SAR ship detection [single shot multibox detector (SSD)]. The algorithm requires only a few training samples of SAR images and effectively combines them with rich optical images to utilize domain information. First, we develop an efficient plug-and-play distance metric function. This function accurately measures the distances between features from the optical domain and the SAR domain. Second, we design a lossy branching mechanism to effectively utilize SAR domain knowledge. This branching mechanism is driven by the observed latent similarity in domain knowledge distribution between optical and SAR images. In addition, we introduce a dual-stream branching feature alignment extraction network with weight sharing. This network architecture enables better knowledge extraction and sharing between optical and SAR domains. To evaluate our method, we conducted experiments on a newly created dataset, DIOR2SSDD, which is designed for few-shot SAR image ship detections across optical and SAR domains. The experimental results show that under three-, five-, and ten-shot settings, the mean average precision (mAP) of our method can reach 59.2%, 61.2%, and 64.6%, and with only 10% SAR training data, the mAP can reach 89.3%. It indicates that our method can effectively transfer domain knowledge and achieve excellent ship detection performance in SAR images.

Medical image processing plays an important role in the interaction of real world and metaverse for healthcare. Self-supervised denoising based on sparse coding methods, without any prerequisite on large-scale training samples, has been attracting extensive attention for medical image processing. Whereas, existing self-supervised methods suffer from poor performance and low efficiency. In this paper, to achieve state-of-the-art denoising performance on the one hand, we present a self-supervised sparse coding method, named the weighted iterative shrinkage thresholding algorithm (WISTA). It does not rely on noisy-clean ground-truth image pairs to learn from only a single noisy image. On the other hand, to further improve denoising efficiency, we unfold the WISTA to construct a deep neural network (DNN) structured WISTA, named WISTA-Net. Specifically, in WISTA, motivated by the merit of the$l_{p}$-norm, WISTA-Net has better denoising performance than the classical orthogonal matching pursuit (OMP) algorithm and the ISTA. Moreover, leveraging the high-efficiency of DNN structure in parameter updating, WISTA-Net outperforms the compared methods in denoising efficiency. In detail, for a 256 by 256 noisy image, the running time of WISTA-Net is 4.72 s on the CPU, which is much faster than WISTA, OMP, and ISTA by 32.88 s, 13.06 s, and 6.17 s, respectively.

Recent years have witnessed a remarkable breakthrough in Synthetic Aperture Radar Automatic Target Recognition (SAR ATR) with the development of deep learning (DL). Nonetheless, once deployed, the DL-based methods’ ability to incrementally learn new knowledge from few-shot samples without forgetting the old is fragile, hindering them from discriminating unseen targets in real-world situations. In this paper, we propose a Cosine Prototype Learning (CPL) framework to first unlock few-shot class-incremental learning (FSCIL) in the SAR ATR field inspired by the intrinsic relationships between target azimuth-aware knowledge and semantic features under the cosine criterion. By condensing class-specific characteristics into individual prototypes, stable profiles of targets are depicted without losing generalization. For the model’s plasticity, a pairwise structure separation (PSS) loss is introduced to separate old and new classes and compact intra-class features. Meanwhile, the model’s transferability on new classes is guaranteed by a prototype consistency (PC) loss. For the model’s stability, we propose a prototype-exemplar distillation (PED) loss and a prototype re-calibration (PR) strategy to penalize semantic drifts of old-class feature spaces and alleviate the misalignment of the learned prototypes successively. At inference, a nearest-class-mean (NCM) classifier is adopted for evaluation by comparing cosine similarity scores between testing samples and class-specific prototypes. In experiments, the proposed components of our method are explored by ablation studies. Strong baselines are established, and extensive experiments conducted on the MSTAR dataset show that our method outperforms state-of-the-art methods under various FSCIL conditions, verifying its effectiveness for the FSCIL of SAR ATR.

As computer programs run on top of blockchain, smart contracts have proliferated a myriad of decentralized applications while bringing security vulnerabilities, which may cause huge financial losses. Thus, it is crucial and urgent to detect the vulnerabilities of smart contracts. However, existing fuzzers for smart contracts are still inefficient to detect sophisticated vulnerabilities that require specific vulnerable transaction sequences to trigger. To address this challenge, we propose a novel vulnerability-guided fuzzer based on reinforcement learning, namely RLF, for generating vulnerable transaction sequences to detect such sophisticated vulnerabilities in smart contracts. In particular, we firstly model the process of fuzzing smart contracts as a Markov decision process to construct our reinforcement learning framework. We then creatively design an appropriate reward with consideration of both vulnerability and code coverage so that it can effectively guide our fuzzer to generate specific transaction sequences to reveal vulnerabilities, especially for the vulnerabilities related to multiple functions. We conduct extensive experiments to evaluate RLF’s performance. The experimental results demonstrate that our RLF outperforms state-of-the-art vulnerability-detection tools (e.g., detecting 8%-69% more vulnerabilities within 30 minutes).

The performance of synthetic aperture radar (SAR) target recognition has been substantially enhanced by the deep learning technology. It is still difficult to get strong recognition performance due to the distribution differences across heterogeneous SAR images. To enhance target recognition performance in heterogeneous SAR situations, a pixel-level and feature-level domain adaptation (PFDA) approach is proposed in this letter to deal with this problem. The pixel-level translation module in the first step generates images with high visual similarity with another distributed images from one distributed images. The second step involves introducing feature alignment into the network to lower the probability distribution divergence of heterologous SAR images in the feature space and enhance recognition performance. We evaluated our method on the Synthetic and Measured Paired Labeled Experiment (SAMPLE) dataset and a self-built airplane dataset to ensure its effectiveness. Compared to existing domain adaption approaches, experimental results reveal that our strategy greatly improves recognition performance and model stability for heterogeneous SAR target recognition.

Synthetic aperture radar (SAR) images are highly susceptible to imaging conditions. However, the majority of deep learning (DL) models in SAR automatic target recognition (ATR) adopt enhanced network structures similar to those in dealing with optical image classification tasks, which is obviously unreasonable since the huge gap of the imaging conditions between training and testing data severely deteriorates the recognition performance. The main idea of the framework is to introduce SAR imaging condition information into the DL training stage to eliminate domain discrepancies between training and testing data. Based on this framework, we propose a task-driven domain adaptation (TDDA) transfer learning method, which can alleviate the degradation of recognition caused by the variance of depression angle between training and testing data. In order to introduce the prior imaging information into the method, simulated SAR data is first obtained by adding a simulated object radar reflectivity to a terrain model of individual point scatters using the known training and testing SAR imaging parameters. Then a domain confusion metric and a supervised classification loss are calculated on simulated data and source training data, respectively, to learn a representation that is semantically meaningful and domain invariant. Comparative experiments on the moving and stationary target acquisition and recognition (MSTAR) dataset demonstrate that the proposed method can obtain better recognition performance than the other methods.

Unsupervised domain adaptation (UDA) aims at learning a classifier for an unlabeled target domain by transferring knowledge from a labeled source domain with a related but different distribution. Most existing approaches learn domain-invariant features by adapting the entire information of the images. However, forcing adaptation of domain-specific variations undermines the effectiveness of the learned features. To address this problem, we propose a novel, yet elegant module, called the deep ladder-suppression network (DLSN), which is designed to better learn the cross-domain shared content by suppressing domain-specific variations. Our proposed DLSN is an autoencoder with lateral connections from the encoder to the decoder. By this design, the domain-specific details, which are only necessary for reconstructing the unlabeled target data, are directly fed to the decoder to complete the reconstruction task, relieving the pressure of learning domain-specific variations at the later layers of the shared encoder. As a result, DLSN allows the shared encoder to focus on learning cross-domain shared content and ignores the domain-specific variations. Notably, the proposed DLSN can be used as a standard module to be integrated with various existing UDA frameworks to further boost performance. Without whistles and bells, extensive experimental results on four gold-standard domain adaptation datasets, for example: 1) Digits; 2) Office31; 3) Office-Home; and 4) VisDA-C, demonstrate that the proposed DLSN can consistently and significantly improve the performance of various popular UDA frameworks.

Aircraft detection in synthetic aperture radar (SAR) imagery is a challenging task in SAR automatic target recognition (SAR ATR) areas due to aircraft’s extremely discrete appearance, obvious intraclass variation, small size, and serious background’s interference. In this article, a single shot detector (SSD), namely, attentional feature refinement and alignment network (AFRAN), is proposed for detecting aircraft in SAR images with competitive accuracy and speed. Specifically, three significant components, including attention feature fusion module (AFFM), deformable lateral connection module (DLCM), and anchor-guided detection module (ADM), are carefully designed in our method for refining and aligning informative characteristics of aircraft. To represent the characteristics of aircraft with less interference, low-level textural and high-level semantic features of aircraft are fused and refined in AFFM thoroughly. The alignment between aircraft’s discrete backscatting points and convolutional sampling spots is promoted in DLCM. Eventually, the locations of aircraft are predicted precisely in ADM based on aligned features revised by refined anchors. To evaluate the performance of our method, a self-built SAR aircraft sliced dataset and a large scene SAR image are collected. Extensive quantitative and qualitative experiments with detailed analysis illustrate the effectiveness of the three proposed components. Furthermore, the topmost detection accuracy and competitive speed are achieved by our method compared with other domain-specific methods, e.g., dense attention pyramid network (DAPN) and pyramid attention dilated network (PADN), and general convolutional neural network (CNN)-based methods, e.g., Feature Pyramid Network (FPN), Cascade R-CNN, SSD, RefineDet, and RepPoints Detector (RPDet).

Unsupervised Domain Adaptation (UDA) aims at learning a classifier for an unlabeled target domain by transferring knowledge from a labeled source domain with a related but different distribution. The strategy of aligning the two domains in latent feature space via metric discrepancy or adversarial learning has achieved considerable progress. However, these existing approaches mainly focus on adapting the entire image and ignore the bottleneck that occurs when forced adaptation of uninformative domain-specific variations undermines the effectiveness of learned features. To address this problem, we propose a novel component called Informative Feature Disentanglement (IFD), which is equipped with the adversarial network or the metric discrepancy model, respectively. Accordingly, the new network architectures, named IFDAN and IFDMN, enable informative feature refinement before the adaptation. The proposed IFD is designed to disentangle informative features from the uninformative domain-specific variations, which are produced by a Variational Autoencoder (VAE) with lateral connections from the encoder to the decoder. We cooperatively apply the IFD to conduct supervised disentanglement for the source domain and unsupervised disentanglement for the target domain. In this way, informative features are disentangled from the domain-specific details before the adaptation. Extensive experimental results on three gold-standard domain adaptation datasets, e.g., Office31, Office-Home and VisDA-C, demonstrate the effectiveness of the proposed IFDAN and IFDMN models for UDA.

Unsupervised Domain Adaptation (UDA) aims to seek an effective model for unlabeled target domain by leveraging knowledge from a labeled source domain with a related but different distribution. Many existing approaches ignore the underlying discriminative features of the target data and the discrepancy of conditional distributions. To address these two issues simultaneously, the paper presents a Transferable Discriminative Feature Mining (TDFM) approach for UDA, which can naturally unify the mining of domain-invariant discriminative features and the alignment of class-wise features into one single framework. To be specific, to achieve the domain-invariant discriminative features, TDFM jointly learns a shared encoding representation for two tasks: supervised classification of labeled source data, and discriminative clustering of unlabeled target data. It then conducts the class-wise alignment by decreasing intra-class variations and increasing inter-class differences across domains, encouraging the emergence of transferable discriminative features. When combined, these two procedures are mutually beneficial. Comprehensive experiments verify that TDFM can obtain remarkable margins over state-of-the-art domain adaptation methods.

Device-free localization (DFL) locates targets without equipping with wireless devices or tag under the Internet-of-Things (IoT) architectures. As an emerging technology, DFL has spawned extensive applications in the IoT environment, such as intrusion detection, mobile robot localization, and location-based services. Current DFL-related machine learning (ML) algorithms still suffer from low localization accuracy and weak dependability/robustness because the group structure has not been considered in their location estimation, which leads to an undependable process. To overcome these challenges, we propose in this work a dependable block-sparse scheme by particularly considering the group structure of signals. An accurate and robust ML algorithm named block-sparse coding with the proximal operator (BSCPO) is proposed for DFL. In addition, a severe Gaussian noise is added in the original sensing signals for preserving network-related privacy as well as improving the dependability of the model. The real-world data-driven experimental results show that the proposed BSCPO achieves robust localization and signal-recovery performance even under severely noisy conditions and outperforms state-of-the-art DFL methods. For single-target localization, BSCPO retains high accuracy when the signal-to-noise ratio exceeds -10 dB. BSCPO is also able to localize accurately under most multitarget localization test cases.

Recently, deep learning based methods have been successfully applied in synthetic aperture radar automatic target recognition (SAR ATR) fields. However, due to the effects of the special structures of aircrafts and the complexity of SAR imaging mechanism, detecting aircrafts accurately in SAR images is still challenging. To alleviate this problem, a novel network called pyramid attention dilated network (PADN) is proposed in this letter. The key component of PADN is the dilated attention block (DAB), which is composed of two submodules - multibranch dilated convolution module (MBDCM) and convolution block attention module (CBAM). In our method, MBDCM is used to enhance the relationship among discrete backscattering features of aircrafts. CBAM is employed to refine redundant information and highlight significant features of aircrafts. A well-designed fine-grained feature pyramid is established by combining the two modules reasonably into DAB when building lateral connections. To alleviate class imbalance, focal loss (FL) is employed to train our network. Experiments on a mixed SAR aircraft data set illustrate the efficiency of the proposed method for aircraft detection.

Unsupervised Domain Adaptation (UDA) aims to learn a classifier for the unlabeled target domain by leveraging knowledge from a labeled source domain with a different but related distribution. Many existing approaches typically learn a domain-invariant representation space by directly matching the marginal distributions of the two domains. However, they ignore exploring the underlying discriminative features of the target data and align the cross-domain discriminative features, which may lead to suboptimal performance. To tackle these two issues simultaneously, this paper presents a Joint Clustering and Discriminative Feature Alignment (JCDFA) approach for UDA, which is capable of naturally unifying the mining of discriminative features and the alignment of class-discriminative features into one single framework. Specifically, in order to mine the intrinsic discriminative information of the unlabeled target data, JCDFA jointly learns a shared encoding representation for two tasks: supervised classification of labeled source data, and discriminative clustering of unlabeled target data, where the classification of the source domain can guide the clustering learning of the target domain to locate the object category. We then conduct the cross-domain discriminative feature alignment by separately optimizing two new metrics: 1) an extended supervised contrastive learning, i.e., semi-supervised contrastive learning 2) an extended Maximum Mean Discrepancy (MMD), i.e., conditional MMD, explicitly minimizing the intra-class dispersion and maximizing the inter-class compactness. When these two procedures, i.e., discriminative features mining and alignment are integrated into one framework, they tend to benefit from each other to enhance the final performance from a cooperative learning perspective. Experiments are conducted on four real-world benchmarks (e.g., Office-31, ImageCLEF-DA, Office-Home and VisDA-C). All the results demonstrate that our JCDFA can obtain remarkable margins over state-of-the-art domain adaptation methods. Comprehensive ablation studies also verify the importance of each key component of our proposed algorithm and the effectiveness of combining two learning strategies into a framework.

Automatically analyzing events in a large amount of text is crucial for situation awareness and decision making. Previous approaches treat event extraction as “one size fits all” with an ontology defined a priori. The resulted extraction models are built just for extracting those types in the ontology. These approaches cannot be easily adapted to new event types nor new domains of interest. To accommodate personalized event-centric information needs, this paper introduces the few-shot Event Mention Retrieval (EMR) task: given a user-supplied query consisting of a handful of event mentions, return relevant event mentions found in a corpus. This formulation enables “query by example”, which drastically lowers the bar of specifying event-centric information needs. The retrieval setting also enables fuzzy search. We present an evaluation framework leveraging existing event datasets such as ACE. We also develop a Siamese Network approach, and show that it performs better than ad-hoc retrieval models in the few-shot EMR setting.

Fault detection is a fundamental requirement for Industrial Internet of Things (IIoT), such as the process industry. This article first reviews the recent studies focusing on applying the fault detection techniques to the IIoT networks. However, we find that numerous studies focus on the resource utilization and workload allocation. The fault detection toward IIoT facilities is still in its immature stage because the existing approaches are not accurate enough for the stringent fault detection in IIoT networks. To this end, we present a novel algorithm, named Gaussian Bernoulli restricted Boltzmann machines (GBRBMs)-based deep neural network (DNN), to transform the fault detection into a classification problem. The real trace-driven experiments show that the proposed scheme outperforms other baseline machine learning methods. We anticipate that this article can inspire blooming studies on the related topics of smart IIoT networks.

We propose a neural network model to estimate word translation probabilities for Cross-Lingual Information Retrieval (CLIR). The model estimates better probabilities for word translations than automatic word alignments alone, and generalizes to unseen source-target word pairs. We further improve the lexical neural translation model (and subsequently CLIR), by incorporating source word context, and by encoding the character sequences of input source words to generate translations of out-of-vocabulary words. To be effective, neural network models typically need training on large amounts of data labeled directly on the final task, in this case relevance to queries. In contrast, our approach only requires parallel data to train the translation model, and uses an unsupervised model to compute CLIR relevance scores.

Device-free localization (DFL), as an emerging technology that locates targets without any attached devices via wireless sensor networks, has spawned extensive applications in the Internet of Things (IoT) field. For DFL, a key problem is how to extract significant features to characterize raw signals with different patterns associated with different locations. To address this problem, in this paper, the DFL problem is formulated as an image classification problem. Moreover, we design a three-layer convolutional autoencoder (CAE) neural network to perform unsupervised feature extraction from raw signals followed by supervised fine-tuning for classification. The CAE combines the advantages of a convolutional neural network (CNN) and a deep autoencoder (AE) in the feature learning and signals reconstruction, which is expected to achieve good performance for DFL. The experimental results show that the proposed approach can achieve a high localization accuracy rate of 100% for a reasonable grid size on the raw real-world data, i.e., the collected raw data without added Gaussian noise, and is robust to noisy data with a signal-to-noise ratio greater than -5 dB. Additionally, its time cost for the classification of a single activity is 4 ms, which is fast enough for the IoT applications. The proposed approach outperforms the deep CNN and AE in terms of localization accuracy and robust ability against noise.

Recently, sparse representation has achieved the impressive performance on target recognition in synthetic aperture radar (SAR) image. However, the unstable and unsupervised optimization of the sparse representation may lead to undesired recognition result. In this paper, a locality-constrained and class-specific sparse representation (LCSR) framework is presented to alleviate these problems. Instead of the sparse constraint, the locality constraint is designed to utilize the local structure information of the training samples. It provides stable representation for the samples with minor variations, which is beneficial to classification. To further improve the recognition performance, the query sample is represented as a linear combination of class-specific galleries based on the supervision of class information. The inference is reached corresponding to the class with the minimum reconstruction error. The experimental results demonstrate the effectiveness and robustness of the proposed method.

As an emerging technology, device-free localization (DFL), using radio frequency (RF) sensor networks to detect targets who do not carry any attached devices, has spawned extensive applications. Many existing works formulate DFL as a classification problem, and a key problem is how to extract discriminative features to characterize the raw wireless signal. In this paper, we present an autoencoder-based deep neural network for feature extraction, moreover, multiple Gaussian Bernoulli restricted Boltzmann machines (GBRBMs) are utilized for pre-training and dimension reduction. Experiment results show that this method of GBRBM-based autoencoder (GBRBM-AE) can achieve a high accuracy and efficient performance, which outperforms the conventional autoencoder. When the dimensions of input data are reduced from 784 to 20 dims, our algorithm can maintain a high accuracy of 97.1% and is robust to noise with SNR = 5dB.

This paper proposes a novel registration method for optical and SAR images which is based on straight line features and mutual information. Firstly, different edge detectors are employed to detect the line segments in both optical and SAR images respectively. Then, through the Hough transform and a straight line fitting and filtration method, the main straight lines of each image are extracted and their intersections are obtained and taken as the candidate matching points. With the RANSAC (RANdom SAmpling Consensus) method, corresponding point pairs (CPPs) are found with these candidate points and a coarse registration between the heterogeneous images is implemented. At last, by using the mutual information of the separated patches generated from the coarse registered images, a fine registration result is finally achieved. The experiment with a pair of X-band air-borne SAR and optical images validates the efficiency and precision of the proposed method.

In this paper, we propose a theoretically new and effective feature for SAR image classification. The new feature combines traditional gray level co-occurrence matrix (GLCM) textural feature and the recent multilevel local pattern histogram (MLPH) feature. It can not only describe intrinsic property of land-cover/land-use surfaces, corresponding to textural information, but it also captures both local and global structural information. Experiments on real SAR images demonstrate that the proposed feature obtains better results than the original GLCM and MLPH features in SAR image classification.

For downward-looking linear array three-dimensional SAR, the resolution in cross-track direction is a curial problem. Hence, compressive sensing algorithm has been used to acquire the superresolution performance in cross-track direction. The limits of the proposed algorithm are investigated in this paper. How accurately can the scattering intensity of the scatterers be estimated? What is the closest separable distance of two scatterers at different levels of SNR? What is the influence of the acquisitions N on the resolution? For all of these questions, the theoretical analysis is given by Cramér-Rao Bound and numerical simulations are proven. The results can be considered as a fundamental bound on parameter estimates.

In this paper, classification via joint sparse representation of the monogenic signal is presented for target recognition in SAR imagery. First, the monogenic signal is performed to capture the characteristics of SAR image. Since it is infeasible to directly apply the raw component to classification due to the high data dimension and redundancy, three augmented feature vectors are defined via uniform downampling of the real part, the imagery part, and the instantaneous phase. The monogenic features are then fed into a recently developed framework, sparse representation-based classification (SRC). Rather than produce individual sparse pattern, this paper generates the similar sparsity pattern for three feature vectors by imposing a mixed norm on the representation matrix. Extensive experiments on MSTAR database demonstrate that the proposed method could significantly improve the recognition accuracy.

In this paper, the classification via nonnegative and local linear regression model is proposed for SAR image-based target recognition. Recently, a simple yet effective method, linear regression for pattern recognition has been presented. By assuming that images from a single-object class lie on a linear subspace, it represents the test image as a linear combination of class-specific galleries. The representation is obtained by solving a typical inverse problem with least-square strategy. Since the negative weights play a counteractive role in reconstruction, it may be unreasonable to generate the negative weights. In addition, those elements close to the test sample should contribute much more than the ones far from the test. Thus this paper limits the feasible set of the representation by nonnegative and locality constraint. The decision is ruled in favor of the class with the minimum reconstruction error. Extensive experiments on MSTAR database demonstrate that the proposed methods significantly improve the accuracy than the standard one.

In this paper, a new azimuth ambiguities removal method is introduced for ship detection by Time-Frequency (TF) analysis. A TF coherence indicator is proposed to filter ghost echoes due to the different TF coherence characteristics between real ship target echoes and ambiguous ones. The effectiveness of this proposed TF coherence indicator for ship detection is demonstrated using single polarimetric spaceborne TerraSAR-X coherent data over the test sea/ocean site in Hongkong, China.

Moment has been widely used for contour matching. To use the moment to achieve contour matching under affine transformations, the affine‐invariant support point set (SPS) should be constructed first. Then, a novel method of acquiring SPS based on the contour projection (SPS‐CP) is proposed here. For an arbitrary selected contour point, the contour is projected onto the line vertical to the vector connecting the contour centroid and the selected point, and the contour points with the sampled projection values are picked up to form the SPS‐CP of the point. SPS‐CP which captures the global structure of the contour is stably affine‐invariant. Experiments on synthetic and real data demonstrate that moments generated from SPS‐CP outperform those generated from SPSs sampled by uniform spacing or affine length.

We reconstruct the multispectral image based on compressive sensing theory. Both spatial domain regularization and transform domain regularization are employed in the proposed objective function. Bregman split method is used to optimize the proposed objective function. In order to making use of the correlation features between different channels of multispectral image, principal component analysis (PCA) is introduced into the shrinkage step of the spatial domain regularization. For further enhance the performance of CS reconstruction, the similarity of wavelet coefficients between different channels are also explored in the shrinkage step of transform domain. We compare the proposed method with some other methods. Experiments validate the better performances of the proposed method, and it is attributed to combine two regularizations and employ the spectral correlation between channels.

Since superpixel takes spatial relationship between pixels into account, which makes the image classification process more understandable and the results more satisfactory, superpixel-based classification methods have been widely studied in recent years. However, due to speckle noise, traditional superpixel generating algorithms still have some drawbacks for synthetic aperture radar (SAR) image. In this letter, we propose a novel superpixel generating algorithm based on pixel intensity and location similarity (PILS) for SAR image. In addition, for the sake of image classification, features of Gabor filters and gray level co-occurrence matrix (GLCM) are extracted from each superpixel. The proposed superpixel generating method has the following three characteristics: (1) the terrain boundaries of SAR image are preserved well; (2) the method has more robustness against speckle noise; and (3) it has high computational efficiency. Experiments on synthetic and real SAR images demonstrate that our method significantly outperforms several state-of-the-art superpixel methods and PILS superpixel-based classification obtains better results than other pixel-based methods.

This paper addresses the Polarimetric synthetic aperture radar (PolSAR) CFAR target detection utilizing the PolSAR synthesis technique. The optimal polarization ellipticity and orientation angles, which maximizes the ratio of the antenna receiver power between target and clutter (SCR), are searched in the co-polarized and cross-polarized channels, and the obtained antenna receiver power is named as the polarization synthesis enhancement (PSE) metric. Using the PSE metric, a data fitting based target detection scheme is presented. First, the Fisher distribution, which has extensive modeling capacity in a large set of clutters, is utilized to fit the distribution of PSE metric. Then, the corresponding “Second Kind Statistics” (SKS) parameter estimator is presented and the numerical solution of the detection threshold is also derived. Afterward, the CFAR detection is implemented. The experimental results demonstrate the enhancement results of PSE metric are almost comparable with that of the Polarimetric Matched Filter (PMF) detector. However, the computation load of PSE metric is generally less than that of the PMF. Moreover, the target detection results show the CFAR detector based on the Fisher distribution can realize the accurate target detection in homogenous and heterogeneous clutter areas.

This letter addresses the polarimetric synthetic aperture radar target detection using the magnitude of the (2, 3) term in the sample averaged coherency matrix. The theoretical analysis demonstrates that such term reveals the difference between the nonreflection symmetric targets and natural clutters. The statistical models for such term are derived within different degrees of homogeneity. Based on the statistical models, an automatic constant-false-alarm-rate detection scheme is completed. The parameter estimation and the solution for the detection threshold are given in detail. Experimental results demonstrate the capability of the proposed approach for detecting ships, oil stores, buildings, etc., in homogeneous and heterogeneous areas.

In this paper, a new classification scheme, which extracts the main scattering mechanism with target scattering similarities, is proposed. This approach not only leads to improved understanding of scattering mechanisms, but also has good performance in discriminating different scattering type of land cover. The NASA/JPL AIRSAR data is used for validating its effectiveness.

In this letter, we propose a novel parameter to measure the scattering similarity between a random scatterer and a canonical scatterer. Compared with the similarity parameter proposed by Yang, the novel parameter not only has some advantages, such as its independence of the spans of a coherence matrix, but also can be applied directly in the case of a random scatterer made up of multiscattering centers. As an example, the novel parameter is adopted to extract some scattering characteristics of a target. With the full polarimetric L-band airborne synthetic aperture radar data, we illustrate the veracity of the novel parameter in measuring scattering similarity and its application in terrain classification.

The Lagrange multiplier method is one of the basic optimization procedures to find the optimum polarizations for the incoherent scattering case. This letter proves for the first time that a fixed relationship exists between the optimum polarization and the Lagrange multiplier. Then, an optimization procedure is proposed to simplify the computational complexity of the Lagrange multiplier method. To speed up the convergence of the proposed procedure, the minimum search intervals are discussed and given theoretically. A numerical example is shown to demonstrate the effectiveness of the proposed procedure.

An adaptive and fast constant false alarm rate (CFAR) algorithm based on automatic censoring (AC) is proposed for target detection in high-resolution synthetic aperture radar (SAR) images. First, an adaptive global threshold is selected to obtain an index matrix which labels whether each pixel of the image is a potential target pixel or not. Second, by using the index matrix, the clutter environment can be determined adaptively to prescreen the clutter pixels in the sliding window used for detecting. The$G^{0}$distribution, which can model multilook SAR images within an extensive range of degree of homogeneity, is adopted as the statistical model of clutter in this paper. With the introduction of AC, the proposed algorithm gains good CFAR detection performance for homogeneous regions, clutter edge, and multitarget situations. Meanwhile, the corresponding fast algorithm greatly reduces the computational load. Finally, target clustering is implemented to obtain more accurate target regions. According to the theoretical performance analysis and the experiment results of typical real SAR images, the proposed algorithm is shown to be of good performance and strong practicability.