In recent years, multivariate time series anomaly detection has become an important research topic in the field of anomaly detection. In Industrial Internet of Things (IIoT) systems, the collected data may be affected by internal failures, external disturbances, or other adverse factors. In such cases, appropriate anomaly detection methods are required to ensure the stable operation of the system. However, existing methods based on reconstruction, prediction, or hybrid approaches often suffer performance degradation when anomalies are present in large amounts of training data, as these anomalies can negatively impact the training process. To address this challenge, we propose a dynamic masking strategy in both temporal and spatial dimensions. We develop a time series imputation framework based on a diffusion model that integrates Graph Neural Network (GNN) and Transformer architectures. This framework, termed Spatial-Temporal Dynamic Mask Diffusion for Anomaly Detection (STDMD-AD), incorporates a dynamic masking mechanism: temporally, reconstruction errors are used to mask data by randomly concealing values with higher errors; spatially, attention is applied to mask channels that are more likely to contain anomalies during training. Experiments on five real-world datasets demonstrate that the proposed method outperforms existing benchmarks and achieves state-of-the-art anomaly detection performance.

The increasing proliferation of unmanned aerial vehicles (UAVs) poses significant challenges to airspace security, necessitating the development of effective detection technologies. Passive detection techniques, such as passive radar, offer key advantages including spectrum efficiency and covert operation. However, passive radars that rely on coherent integration are often computationally expensive and dependent on strong Doppler signatures, rendering them ineffective for detecting low-speed or hovering UAVs. To overcome these limitations, we explore the use of channel state information (CSI) time series to characterize UAV presence and propose a temporal variation network for detecting UAV states, including hovering conditions. Our method utilizes digital terrestrial multimedia broadcast (DTMB) signals, which have wide coverage and high transmission power. By capturing DTMB signals with a single receiver, we reduce the complexity of passive detection systems. First, we perform channel estimation on the received signal to obtain CSI, which is arranged in frame order to form a CSI time series. This enables the modeling of interference channels caused by UAVs. We then propose the Channel Estimation and Temporal Variation Network (CETVNet), which leverages an adaptive noise reduction module and a multi-period feature extraction module to process these series for passive UAV state detection. Finally, a real-world signal dataset is collected using a software-defined radio device to train and evaluate CETVNet. Experimental results demonstrate that CETVNet achieves superior performance compared to state-of-the-art methods.

In building extraction, collecting pixel-level annotations required by fully supervised methods is extremely costly. Image-level weakly supervised methods based on class activation maps (CAMs) effectively reduce the cost and have shown promising progress. However, generating high-quality CAMs remains challenging due to the supervision gap between classification and segmentation tasks. Specifically, image-level supervision causes CAMs to activate only the most discriminative regions, which compromises the integrity of CAMs. Meanwhile, the absence of pixel-level supervision leads to a depletion of spatial information, resulting in imprecise boundaries. In this study, we propose a novel image-level weakly supervised building extraction method based on view consistency, named ViewCAM, to generate high-quality CAMs. The view transformation module is designed to apply view transformations to remote sensing images and the high-dimensional features. Additionally, a feature affinity enhancement module (FAEM) is proposed to capture positional relationships between pixels and low-level features, such as edges and textures, improving boundary fineness. We integrate these two modules into a classification network and incorporate pixel-level supervision using view-consistency constraints. The entire network is then trained in an end-to-end manner, leading to improved integrity and boundary fineness of the seeds generated from CAMs. To verify the effectiveness and robustness of ViewCAM, we conduct experiments on two representative datasets, and the results demonstrate that our proposed method achieves superior CAM integrity and boundary fineness, outperforming state-of-the-art methods.

Automatic Modulation Classification (AMC)-oriented Deep Neural Networks (ADNNs) have received much attention in recent years for their wide range of applications. However, they are vulnerable to attacks. Adversarial Examples (AEs) of modulation signals with added weak perturbations can easily fool ADNNs. The study of AEs on AMC, on one side, can enhance the security of wireless communication systems; on the other side, it can provide an effective defence against potential attacks. Nevertheless, most existing attack methods generate AEs with low transferability. In this paper, we propose a Multiscale Discriminative Attack Method (MDAM) for modulated signals. The method strives to alleviate such transferability issue by destroying discriminative features in multi-layer. Specifically, we utilize interpretable class activation maps to distinguish the discriminative regions, ignoring the noise and focusing on the interference of the discriminative features. Beyond that, we propose a multi-layer activation disruption loss to constrain activations in the middle layers. In so doing, the AEs do not erroneously retain deep features of the original signal. We conduct extensive experiments on RadioML datasets and the local area network (LAN) communication dataset we collected to evaluate the effectiveness of MDAM in both white-box and black-box attack scenarios. The results show that MDAM outperforms existing methods.

Incurable Alzheimer's disease (AD) plagues many elderly people and families. It is important to accurately diagnose and predict it at an early stage. However, the existing methods have shortcomings, such as inability to learn local and global information and the inability to extract effective features. In this paper, we propose a lightweight classification network Local and Global Graph ConvNeXt. This model has a hybrid architecture of convolutional neural network and Transformers. We build the Global NeXt Block and the Local NeXt Block to extract the local and global features of the structural magnetic resonance imaging (sMRI). These two blocks are optimized by adding global multilayer perceptron and locally grouped attention, respectively. Then, the features are fed into the pixel graph neural network to aggregate the valid pixel features using mask attention. In addition, we decoupled the loss by category to optimize the calculation of the loss. This method was tested on slices of the processed sMRI datasets from ADNI and achieved excellent performance. Our model achieves 95.81% accuracy with fewer parameters and floating point operations per second (FLOPS) than other classical efficient models in the diagnosis of AD.

Due to the presence of redundancy and interference, frame sampling is a promising but challenging solution to mitigate the expensive computation of video action recognition. Although the motion prior has shown great potential for frame selection, existing motion-based strategies suffer from limitations in terms of robustness and interpretive evaluation. In this paper, we devise a robust frame sampling strategy called positive motion guided sampler (PMGSampler). It consists of two procedures, local motion capture and global motion statistics. At the local level, we propose two concepts about inter-frame motion amplitude and motion continuity, which helps to perceive the movement of subjects and identify abnormal events that may generate negative pseudo-motion information. Then, through a global analysis of the obtained local motions, the sampler becomes more sensitive to informative frames and robust to outliers. The proposed sampler can be applied to most existing models for improving recognition accuracy. We conduct extensive experiments on four widely-used benchmarks to demonstrate the superiority of our PMGSampler over other methods of the same type. In addition, to analyse how sampled frames influence action recognition, we present a visual interpretation method for video models, termed as spatio-temporal class activation map (STCAM). By introducing spatial and temporal branches, our STCAM is able to visualise the salience of spatio-temporal features. With the help of STCAM, we can further intuitively evaluate the performance of different sampling strategies.

The surge in encrypted network traffic poses a significant challenge to existing cyberspace security measures. Classical traffic identification methods, such as those based on ports or statistical features, are ineffective against encrypted traffic. Deep learning methods offer new avenues for identifying encrypted traffic, but they are highly dependent on labeled data and struggle to adapt to new types of encrypted traffic. Pre-trained models possess a powerful generalization capability due to their training on large-scale unlabeled datasets, and they reduce the dependency on labeled data for specific downstream tasks. In this paper, we present an encrypted traffic classification framework based on pre-trained models and introduce a tokenization method tailored for encrypted traffic called FlowPiece. The experimental results demonstrate that our approach, along with the FlowPiece method, can reduce the parameter count by approximately 93% while maintaining nearly identical performance. This significant reduction in parameters enables the widespread application of encrypted traffic identification methods based on pretrained models.

Silent high-dynamic target recognition using single receiver has garnered significant attention due to its advantages in safety and stealth within military operations. This paper presents an in-depth exploration of a silent high-dynamic target recognition method based on frequency modulation (FM) signals. The proposed method processes the FM signals with a least squares filtering approach and further implements the calculation of Cross-Ambiguity Function (CAF) mapping to achieve target imaging within the CAF spectrum. By capturing FM signals with an antenna array tuned to various orientations, we conducted empirical analysis using actual aircraft in flight as the target for identification. The recognition scheme put forth by our research is capable of precisely determining the velocity of the target and the relative distance parameters between dual base stations, facilitating accurate tracking of targets.

Falls, a common type of accident, especially among the elderly and those with mobility impairments, potentially leading to serious physical injuries and health issues. Fall detection refers to the use of sensors, monitoring equipment, or other technological means to monitor and identify occurrences of falls. Currently, a lot of research on fall detection from various aspects such as vision, wearable devices, and multi-modal. However, current vision-based fall detection datasets are limited to single indoor scenarios and do not consider outdoor scenarios. Therefore, in this paper we propose a multi-perspective indoor and outdoor scenarios fall detection dataset based on campus surveillance, which include both indoor and outdoor campus scenarios. We employed YOLOv5 to carry out experiments on our proposed dataset as a benchmark. Moreover, we undertook comparative experiments against other datasets and assessed the richness of our dataset. The experiment results reveal that our dataset encompasses more diverse scenarios than other vision-based fall detection dataset.

The Specific Emitter Identification (SEI) technology has broad application prospects in the fields of the Internet of Things and cognitive communication and serves as an effective means for device authentication. Most existing SEI methods based on deep learning primarily operate in the real number domain. However, complex numbers naturally represent radio frequency signals, making complex-valued neural network(CVNN) a superior choice for signal representation. To address the limitations of traditional real-valued neural network methods, such as low recognition accuracy and the requirement for a large number of training samples, we propose an SEI method based on CVNN to learn the true RF fingerprint features. In the data preprocessing stage, we employ a signal slicing strategy with a random window step size to enhance data randomness and improve the model’s generalization performance. Given the complex-valued nature of signals, we employ a complex-valued convolutional neural network for feature extraction. Subsequently, we design a feature fusion module based on the complex-valued attention mechanism to eliminate redundant features while preserving radio frequency fingerprint features. Considering the high similarity between emitter signals, we employed a joint loss function based on metric learning that promotes intra-class aggregation and interclass separation. The experimental results demonstrate that the proposed method uncovers the authentic RF fingerprint with high accuracy and robustness under limited training sample conditions.

With the advancement of Internet of Things technology, the need for sophisticated signal modulation classification has intensified, ensuring seamless communication and bolstering security among interconnected devices. In the contemporary complex channel environment, the difficult lies in dealing with a multitude of modulation schemes that exhibit subtle distinctions. Prior knowledge-guided and deep learning methods have complementary strengths in the current context of signal modulation classification. To synthesize the advantages of these two methods, we propose an integrated method of prior knowledge and contrast feature for signal modulation classification, called APFS. APFS integrates prior knowledge from the modulation task with feature information acquired through contrastive learning. Feature extraction guided by prior knowledge accurately captures the key patterns in modulated signals. Contrastive learning reveals the inherent distinctions among various modulation modes by comparing different samples. In the joint feature extraction approach for prior knowledge, each form of prior knowledge is first analyzed independently, and then jointed to extract information from its temporal sequence. The contrast features surpass the constraints of labeling and unearth deeper implicit information. In experiments, we systematically compared the performance of our method with various baselines, as well as combinations of prior knowledge and contrast feature. The results demonstrate the superior performance of our method.

Advances in communication technology have enabled signal modulation classification (SMC) to be widely used in noncooperative identification situations, such as spectrum detection, electronic countermeasures, and target identification. In the face of complex electromagnetic environments and various classification tasks, the class imbalance phenomenon in modulated signal data sets has become a problem that cannot be ignored. For the SMC based on machine learning, the unbalanced training data set will cause the actual decision boundary to shift, thereby reducing the prediction accuracy of minority signals. And for SMC based on deep learning, unbalanced data will lead to distortion of the feature space and affect the extraction of discriminative features. However, the existing modulation classification methods cannot effectively deal with the imbalance problem. This study introduces an oversampling method tailored for modulation signals. Our method balances the data set by synthesizing new samples according to the distribution of signal samples and the distance between samples, which will effectively reduce the impact of the imbalance problem on the classifier. For modulated signals, experimental results show that our method performs better than other oversampling methods. In addition to the SMC task, we test the performance of the proposed method for individual identification of radiation sources on the aircraft communications addressing and reporting system data set. Compared with other comparison methods, our method improves the classification performance the most.

Seamless Internet of Things (IoT) connections expose many vulnerabilities in wireless networks, and IoT devices inevitably face many malicious active attacks. automatic modulation recognition (AMR) is an effective way to combat IoT physical layer threats. In the field of noncollaborative communication, feature representation learning for unlabeled signals is an important task of AMR. However, due to the unavailability of a priori knowledge and the influence of interference during signal transmission, the intercepted unlabeled signals are difficult to perform efficient feature representation. In this article, we propose cooperative contrast learning for unlabeled modulation signal Cooperative Contrast Learning for modulation Signals (CoCL-Sig). Specifically, the CoCL-Sig is trained using both sequence and constellation diagram modalities, and is divided into two parts: 1) modal-level feature representation and 2) instance-level auxiliary feature representation. In modal-level feature representation, two modal projections are matched in the same hyperplane space. To ensure the stability of the feature representation, a sequence auxiliary branch is added to form an instance-level feature representation of the sequence. In addition, the feature representations obtained by the CoCL-Sig can be applied to modulation signals for semi-supervised classification and clustering tasks. We have conducted extensive experiments on two widely used modulation signal data sets, RML2016.10A and RML2016.04C. The results demonstrate the effectiveness of our method in modulation signal feature representation and its superiority compared to other methods.

With the development of deep learning and the increase in the amount of data, general artificial intelligence models have become a popular research area nowadays. When facing a new application scenario, a pretraining general model can often show better performance than models trained with new data on its own. However, because of the specificity of the differences in hyperspectral image data bands, the current hyperspectral image classification (HSIC) field has not proposed a better general model training solution, and it is difficult to utilize the information of the existing hyperspectral datasets for model training in the face of a new scenario. In order to solve this problem, this article proposes a generalized hyperspectral classification model training method, which effectively completes the training of hyperspectral classification models across datasets by adaptive channel module and masked self-supervised pretraining method, and can pretrain and fine-tune hyperspectral classification models using multiple datasets. The adaptive channel module is able to solve the band difference problem of using hyperspectral datasets across datasets, and the masked self-supervised learning method solves the label difference and labeling difficulties of training models across datasets. Experimental results on multiple datasets show that the method proposed in this article can effectively use a large amount of data to complete the pretraining of hyperspectral classification models, and the fine-tuning results on downstream datasets have certain advantages relative to current advanced deep learning methods.

The continual progression and expanding applications of Hyperspectral Imaging (HSI) technology necessitate the development of lightweight HSI classification models that are capable of lifelong learning. However, the computationally-demanding task of training and updating HSI classification models, exacerbated by the substantial number of trainable parameters in feature extractors, remains a substantial challenge. This paper proposes an Attention-based Reservoir Computing (ARC) model to overcome these hurdles. The ARC model utilizes a cross-slicing operation to generate multi-directional inputs, treating the HSI dataset as spatial sequence for processing within a reservoir. For every target pixel, four spatial sequences from various directions are introduced into the reservoir, generating four corresponding outputs. A voting mechanism then evaluates these outputs to yield the final prediction. Additionally, we design an attention-based leaky function for reservoir computing to capture the spatial correlation inherent in HSI data accurately. The attention-based leaky function enables the reservoir state to weigh less on the pixels outside the region of interest (ROI) and have a longer memory for pixels inside the ROI. The ARC was tested on widely used HSI datasets: Indian Pines, PaviaU, and Salinas. It demonstrated competitive lightweight classification performance against state-of-the-art lightweight models by maintaining comparable training time while achieving superior accuracy. Furthermore, the model’s lifelong learning accuracy also showed superior performance compared to existing lifelong learning models, with a one thousand times reduction of parameter-to-be-updated. This work makes the ARC model an effective contender for HSI classification tasks, excelling in both lightweight classification and lifelong learning capacities. The source codes are available publicly at: https://github.com/Waterman-Ann/ARC.

The electromagnetic environments have become more complex with the development of wireless communication technology. Signal modulation classification has attracted extensive attention due to its application in electronic countermeasures and physical layer security threat prevention under complex electromagnetic environments. Excellent classification performance requirements challenge the adaptability of the method and the ability to extract modulation characteristics. This paper proposes an automated machine learning framework, AutoSMC, for signal modulation classification. An adaptive signal augmentation method is proposed to adapt to the network changes during the search process. In order to extract the modulation features effectively, an scalable convolutional random fourier feature block is proposed. Moreover, the initial search space of the framework is given. The Bayesian Optimization is used to drive hyperparameter optimization to achieve AutoSMC and obtain the optimal method state. Great experiments were carried out on RADIOML 2016.10A and RADIOML 2016.10B. Experimental evaluations on these datasets show that our approach AutoSMC achieves state-of-the-art results compared to the most relevant signal modulation classification methods.

In recent years, object localization and detection methods in remote sensing images (RSIs) have received increasing attention due to their broad applications. However, most previous fully supervised methods require a large number of time-consuming and labor-intensive instance-level annotations. Compared with those fully supervised methods, weakly supervised object localization (WSOL) aims to recognize object instances using only image-level labels, which greatly saves the labeling costs of RSIs. In this article, we propose a self-directed weakly supervised strategy (SD-WSS) to perform WSOL in RSIs. To specify, we fully exploit and enhance the spatial feature extraction capability of the RSIs' classification model to accurately localize the objects of interest. To alleviate the serious discriminative region problem exhibited by previous WSOL methods, the spatial location information implicit in the classification model is carefully extracted by GradCAM++ to guide the learning procedure. Furthermore, to eliminate the interference from complex backgrounds of RSIs, we design a novel self-directed loss to make the model optimize itself and explicitly tell it where to look. Finally, we review and annotate the existing remote sensing scene classification dataset and create two new WSOL benchmarks in RSIs, named C45V2 and PN2. We conduct extensive experiments to evaluate the proposed method and six mainstream WSOL methods with three backbones on C45V2 and PN2. The results demonstrate that our proposed method achieves better performance when compared with state-of-the-arts.

The task of hyperspectral image (HSI) classification has attracted extensive attention. The rich spectral information in HSIs not only provides more detailed information but also brings a lot of redundant information. Redundant information makes spectral curves of different categories have similar trends, which leads to poor category separability. In this article, we achieve better category separability from the perspective of increasing the difference between categories and reducing the variation within category, thus improving the classification accuracy. Specifically, we propose the template spectrum-based processing module from spectral perspective, which can effectively expose the unique characteristics of different categories and reduce the difficulty of model mining key features. Second, we design an adaptive dual attention network from spatial perspective, where the target pixel can adaptively aggregate high-level features by evaluating the confidence of effective information in different receptive fields. Compared with the single adjacency scheme, the adaptive dual attention mechanism makes the ability of target pixel to combine spatial information to reduce variation more stable. Finally, we designed a dispersion loss from the classifier's perspective. By supervising the learnable parameters of the final classification layer, the loss makes the category standard eigenvectors learned by the model more dispersed, which improves the category separability and reduces the rate of misclassification. Experiments on three common datasets show that our proposed method is superior to the comparison method.

Imbalanced signal modulation classification is a challenging problem in wireless communication. In this paper, we propose an overall framework from data generation to generated data qualitative analysis. It generates data based on Synthetic Minority Over-Sampling Technique (SMOTE) and evaluates the generated data in terms of amplitude, distribution, and classification validity, namely SMOTE-ADC. The framework aims to generate synthetic data for the minority class to rebalance the dataset and improve the classification performance. SMOTE-ADC creates and defines a matrix of distances between eigenvalues by considering the eigenvalues and their relationships. By comparing the variability of neighboring feature vectors and defining the weights of the distance matrix to make the generated data more close to the real data. For the evaluation of generated data in SMOTE-ADC, we propose three methods to analyze the effect of generated data, comparing the amplitude, distribution, and classification effectiveness of generated data and original data, respectively. Experimental results demonstrate that our approach effectively mitigates the impact of data imbalance and enhances the accuracy of signal modulation classification.

Achieving reliable intervehicle positioning is one of the most fundamental elements for many vehicular applications, including collision avoidance and autonomous driving. Vehicle position is generally provided by a global navigation satellite system (GNSS), which unfortunately suffers from inaccuracy to varying degrees in challenging environments, for example, GNSS outages. In this article, a reliable fusion technique, called non-Gaussian Redheffer weighted least squares ( n GRWLSs), is proposed. This new approach highlights the intervehicle positioning estimation in multi-GNSS outage environments, such as complete, partial, and free GNSS pseudorange outages. The proposed method combines, on the one hand, the benefits of the Gaussian dynamical matrix principle and the Redheffer distribution function for the sparse property in complete GNSS pseudorange outages and, on the other hand, the use of the optimal window size to regulate the data flow generated by both the inertial navigation systems (INSs) and GNSS during a partial GNSS pseudorange outage. During the free GNSS pseudorange outage, the process ignores data from the INS, and instead, GNSS pseudorange information alone will be considered to compute the intervehicle positioning information. Consequently, weighted least squares is used as an intervehicle positioning estimator. To address the pseudorange uncommon and INS measurement noises, the generalized error distribution (GED) is used to estimate the non-Gaussian densities. Finally, road-test experiments are implemented to evaluate the consistency of the proposed approach. The experimental results show that the proposed n GRWLS can accurately estimate the intervehicle positioning under various conditions (free, partial, and complete GNSS pseudorange outages).

Understanding the private car aggregation effect is conducive to a broad range of applications, from intelligent transportation management to urban planning. However, this work is challenging, especially on weekends, due to the inefficient representations of spatiotemporal features for such aggregation effect and the considerable randomness of private car mobility on weekends. In this article, we propose a deep learning framework for a spatiotemporal attention network (STANet) with a neural algorithm logic unit (NALU), the so-called STANet-NALU, to understand the dynamic aggregation effect of private cars on weekends. Specifically: 1) we design an improved kernel density estimator (KDE) by defining a log-cosh loss function to calculate the spatial distribution of the aggregation effect with guaranteed robustness and 2) we utilize the stay time of private cars as a temporal feature to represent the nonlinear temporal correlation of the aggregation effect. Next, we propose a spatiotemporal attention module that separately captures the dynamic spatial correlation and nonlinear temporal correlation of the private car aggregation effect, and then we design a gate control unit to fuse spatiotemporal features adaptively. Further, we establish the STANet-NALU structure, which provides the model with numerical extrapolation ability to generate promising prediction results of the private car aggregation effect on weekends. We conduct extensive experiments based on real-world private car trajectories data. The results reveal that the proposed STANet-NALU outperforms the well-known existing methods in terms of various metrics, including the mean absolute error (MAE), root mean square error (RMSE), Kullback-Leibler divergence (KL), and R2.

Hyperspectral image classification (HSIC) has attracted widespread attention due to its important application in environment alterations and geophysical disaster monitoring. However, surface cultivation is not static as time passes, which leads to different hyperspectral images information collected from the same area at different time periods. Therefore, researchers are currently eager to construct a HSIC model that continuously acquires new classes of data. During the continuous learning process, the model is expected to not only effective in extracting unique spatial-spectral features of the hyperspectral image, but also ensures the ability to maintain the old classes knowledge while learning new data. To achieve this purpose, we propose a method which based on geometric spatial-spectral feature integration network with class incremental learning (GS2FIN-CIL) framework in continuous learning to make the model adaptable to new classes data and not overly forgetting the old classes knowledge during the training process. We conduct extensive experiments with the proposed GS2FIN-CIL method on widely-used hyperspectral datasets including Indian Pines, PaviaU and Salinas. The experimental results show that our GS2FIN-CIL method can achieve significantly improved results compared to current state-of-the-art class incremental learning methods, allowing for efficient adaptation and utilization of spatial-spectral features in processing new classes of hyperspectral images and alleviating the problem of catastrophic forgetting of learned old classes knowledge. The GS2FIN-CIL method could be successfully applied to the challenge of adding new classes data in HSIC task.

Deep learning-based recognition of radio signal modulation has emerged as a current research hotspot with significant practical potential. However, in practical applications, radio modulation signal data acquisition is complicated to obtain, and label samples are costly and time-consuming to meet the data dependence of deep learning. Transfer learning allows pretrained networks to be reused on large-scale datasets, making it a kind of solution for modulation signal recognition in limited data. The method of suppressing small singular values in the feature vector is employed in this paper to realize selective knowledge transfer for modulation signal recognition, while stochastic normalization is employed to replace the batch normalization layer to avoid over-fitting. We tested the stochastic normalized selective knowledge transfer method on the RML2016.10A and RML2016.04C datasets, with an SNR of 6dB signal samples, and found that it can lead to average growth of 15.77% and 10.32% when compared to direct training, and 6.1% and 2.73% when compared to vanilla fine-tuning. In addition, we check up under a variety of SNR conditions to ensure that our method is effective.

Synthetic Aperture Radar is superior to optical sensors in that it can identify ships at all hours and on all days. Deep learning-based object detection relies on huge amounts of data, yet SAR ship images are challenging to obtain and label. A few-shot cross-domain transfer learning approach for SAR image ship detection is used in this paper. It is divided into two stages: the first uses a large volume of optical remote sensing ship images as the source domain training detection framework, and the second employs SAR ship images and optical remote sensing ship images to create a few-shot balanced subset fine-tuning detection framework. Use a metric learning-based prediction box classifier instead of a fully connected prediction box classifier. When fine-tuning the whole detection frame using the metric learning-based pre-diction frame classifier, the experiments show that an AP50 of 55.99% can be reached with only 10 SAR ship images.

Early diagnosis of Alzheimer's disease(AD) is becoming increasingly important in preventing and treating the disease as the world's population ages. We proposed a SMIL-DeiT network for AD classification tasks amongst three groups: Alzheimer's Disease (AD), Mild Cognitive Impairment (MCI), and Normal Cognitive (NC) in this study. Vision Transformer is the fundamental structure of our work. The data pre-training is performed utilizing DINO, a self-supervised technique, whereas the downstream classification task is done with Multiple Instance Learning. Our proposed technique works on the ADNI dataset. We used four performance metrics accuracy rates, precision, recall, and Fl-score in the evaluation, the most important of which was accuracy. The accuracy obtained by our method is higher than the transformer's 90.1% and CNN's 90.8%, reaching 93.2%.

The multimodal representation of signals can give a lot of information in the field of communication modulation signal recognition. Existing approaches can use some of the modes for classification tasks, but they don't take use of the multimodal characteristics of signals well. The approach of signal modulation recognition suggested in this research is based on reinforcement learning and multimodal features. The feature set is obtained via wavelet transform using the 1D features of the signal and the constellation map features, and then the feature set is filtered using reinforcement learning to get a small subset of features to obtain the training feature set. With a small number of extra channels, the accuracy of recognition can be enhanced when utilizing the filtered feature set to categorize the modulation class of the signal. Experiments on the RML2016 dataset validate the effectiveness of our proposed method and achieve good classification performance, which is a pioneering idea for the signal modulation classification problem.

Convolutional neural networks (CNNs) have achieved remarkable performance in driver drowsiness detection based on the extraction of deep features of drivers' faces. However, the performance of driver drowsiness detection methods decreases sharply when complications, such as illumination changes in the cab, occlusions and shadows on the driver's face, and variations in the driver's head pose, occur. In addition, current driver drowsiness detection methods are not capable of distinguishing between driver states, such as talking versus yawning or blinking versus closing eyes. Therefore, technical challenges remain in driver drowsiness detection. In this article, we propose a novel and robust two-stream spatial-temporal graph convolutional network (2s-STGCN) for driver drowsiness detection to solve the above-mentioned challenges. To take advantage of the spatial and temporal features of the input data, we use a facial landmark detection method to extract the driver's facial landmarks from real-time videos and then obtain the driver drowsiness detection result by 2s-STGCN. Unlike existing methods, our proposed method uses videos rather than consecutive video frames as processing units. This is the first effort to exploit these processing units in the field of driver drowsiness detection. Moreover, the two-stream framework not only models both the spatial and temporal features but also models both the first-order and second-order information simultaneously, thereby notably improving driver drowsiness detection. Extensive experiments have been performed on the yawn detection dataset (YawDD) and the National TsingHua University drowsy driver detection (NTHU-DDD) dataset. The experimental results validate the feasibility of the proposed method. This method achieves an average accuracy of 93.4% on the YawDD dataset and an average accuracy of 92.7% on the evaluation set of the NTHU-DDD dataset.

Hyperspectral imaging (HSI) classification has drawn tremendous attention in the field of Earth observation. In the big data era, explosive growth has occurred in the amount of data obtained by advanced remote sensors. Inevitably, new data classes and refined categories appear continuously, and such data are limited in terms of the timeliness of application. These characteristics motivate us to build an HSI classification model that learns new classifying capability rapidly within a few shots while maintaining good performance on the original classes. To achieve this goal, we propose a linear programming incremental learning classifier (LPILC) that can enable existing deep learning classification models to adapt to new datasets. Specifically, the LPILC learns the new ability by taking advantage of the well-trained classification model within one shot of the new class without any original class data. The entire process requires minimal new class data, computational resources, and time, thereby making LPILC a suitable tool for some time-sensitive applications. Moreover, we utilize the proposed LPILC to implement fine-grained classification via the well-trained original coarse-grained classification model. We demonstrate the success of LPILC with extensive experiments based on three widely used hyperspectral datasets, namely, PaviaU, Indian Pines, and Salinas. The experimental results reveal that the proposed LPILC outperforms state-of-the-art methods under the same data access and computational resource. The LPILC can be integrated into any sophisticated classification model, thereby bringing new insights into incremental learning applied in HSI classification.

Hyperspectral images (HSIs) have gained high spectral resolution due to recent advances in spectral imaging technologies. This incurs problems, such as an increased data scale and an increased number of bands for HSIs, which results in a complex correlation between different bands. In the applications of remote sensing and earth observation, ground objects represented by each HSI pixel are composed of physical and chemical non-Euclidean structures, and HSI classification (HIC) is becoming a more challenging task. To solve the above problems, we propose a framework based on a deep attention graph convolutional network (DAGCN). Specifically, we first integrate an attention mechanism into the spectral similarity measurement to aggregate similar spectra. Therefore, we propose a new similarity measurement method, i.e., the mixed measurement of a kernel spectral angle mapper and spectral information divergence (KSAM-SID), to aggregate similar spectra. Considering the non-Euclidean structural characteristics of HSIs, we design deep graph convolutional networks (DeepGCNs) as a feature extraction method to extract deep abstract features and explore the internal relationship between HSI data. Finally, we dynamically update the attention graph adjacency matrix to adapt to the changes in each feature graph. Experiments on three standard HSI data sets, namely, the Indian Pines, Pavia University, and Salinas data sets, demonstrate that the DAGCN outperforms the baselines in terms of various evaluation criteria. For example, on the Indian Pines data set, the overall accuracy of the proposed method achieves 98.61% when the training sample is 10%.

In the field of hyperspectral image (HSI) classification, deep learning has helped achieve great successes. However, most of these achievements are made with very large amounts of labeled training data. Manual annotation of HSIs is labor intensive and time consuming. In practical HSI classification, there may only be a few labeled samples available. To perform HSI classification with a small number of labeled samples, a new few-shot classification model based on adaptive subspaces and featurewise transformation is proposed in this article. First, we design a 3-D local channel attention residual network to obtain the spatial–spectral features of HSIs. Then, a featurewise transformation strategy is introduced to enhance feature diversity to avoid model overfitting problems and to mitigate the impact of cross-domain problems. Finally, a subspace classifier is implemented to construct different subspace categories based on the embedded features of the limited labeled samples. Classification of an HSI sample is performed using spatial projection and a distance metric. The proposed model is trained using the metalearning mechanism to perform few-shot classification of HSIs. Four public datasets are utilized to construct a sufficient few-shot classification task named episodes for training. The other three public datasets are used to test the proposed model. Experiments show that our proposed method can outperform mainstream small sample HSI classification methods.

Recently, optical remote-sensing images have been steadily growing in size, as they contain massive data and complex backgrounds. This trend presents several problems for object detection, for example, increased computation time and memory consumption and more false positives due to the complex backgrounds of large-scale images. Inspired by deep neural networks combined with time-frequency analysis, we propose a time-frequency analysis-based object detection method for large-scale remote-sensing images with complex backgrounds. We utilize wavelet decomposition to carry out a time-frequency transform and then integrate it with deep learning in feature optimization. To effectively capture the time-frequency features, we propose a feature optimization method based on deep reinforcement learning to select the dominant time-frequency channels. Furthermore, we design a discrete wavelet multiscale attention mechanism (DW-MAM), enabling the detector to concentrate on the object area rather than the background. Extensive experiments show that the proposed method of learning from time-frequency channels not only solves the challenges of large-scale and complex backgrounds, but also improves the performance compared to the original state-of-the-art object detection methods. In addition, the proposed method can be used with almost all object detection neural networks, regardless of whether they are anchor-based or anchor-free detectors, horizontal or rotation detectors.

The graph-based hyperspectral image classification (HSIC) method has attracted wide attention because it can extract information with a non-Euclidean structure. Many graph-based HSIC works have achieved good results, but unresolved technical issues remain. For example, many graph nodes lead to high computational costs, and the mining of non-Euclidean structures is not sufficient. To solve these problems, we propose a graph attention network with an adaptive graph structure mining (GAT-AGSM) approach. Specifically, we first propose an HSIC framework with a superpixel feature subdivision (SFS) mechanism. In this framework, the number of nodes in the graph structure is reduced by using superpixel segmentation algorithms, and the SFS mechanism is designed to generate finer classification results. Second, we design the spatial–spectral attention layer with an adaptive graph structure mining (AGSM) mechanism for the graph attention network. The spatial–spectral attention layer can filter information in both spatial and spectral dimensions. The AGSM mechanism requires less manual intervention to dynamically generate non-Euclidean graph structures that better aggregate information. We conduct excessive experiments to compare the proposed GAT-AGSM with seven nongraph methods and three graph-based methods on widely used datasets. On the Indian Pines, Pavia University, and Salinas datasets, compared to the comparison method, the overall accuracy of GAT-AGSM is improved by at least 4.26%, 2.59%, and 1.41%, respectively. Experimental results show that GAT-AGSM has the best performance compared to the baselines in terms of various metrics.

Deep learning has become a mainstream method of hyperspectral image (HSI) classification. Many DL-based methods exploit spatial-spectral features to achieve better classification results. However, due to the complex backgrounds in HSIs, existing methods usually show unsatisfactory performance for the class pixels located on the land-cover category boundary area. In large part, this is because the network is susceptible to interference by the irrelevant information around the target pixel in the training stage, resulting in inaccurate feature extraction. In this paper, a new multibranch transformer architecture (SST-M) that assembles spatial attention and extracts spectral features is proposed to address this problem. The transformer model has a global receptive field and thus can integrate global spatial position information in the HSI cube. Meanwhile, we design a spatial sequence attention model to enhance the useful spatial location features and weaken invalid information. Considering that HSIs contain considerable spectral information, a spectral feature extraction model is designed to extract discriminative spectral features, replacing the widely used PCA method and obtaining better classification results than it. Finally, inspired by semantic segmentation, a mask prediction model is designed to classify all of the pixels in the HSI cube; this guides the neural network to learn precise pixel characteristics and spatial distributions. To verify the effectiveness of our algorithm (SST-M), quantitative experiments were conducted in three well-known datasets, namely, IP, PU, and KSC. The experimental results demonstrate that the proposed model achieves better performance than the other state-of-the-art methods.

In recent years, hyperspectral image classification (HIC) algorithm based on deep learning has been widely studied, and has achieved much better results than traditional algorithms. HIC using small samples has gradually become a research hotspot, and the generative adversarial networks (GANs) have become a brilliant application in this field. However, the HIC results based on GAN methods are poor and volatile, since a single loss function cannot accurately measure the distance between the generated samples and the real samples in different hyperspectral images. To resolve this problem, we propose a novel immune evolutionary generative adversarial network (HIEGAN) via leveraging the evolutionary strategy and immune strategy. Specifically, we enhance the performance of the generator in two ways: 1) HIEGAN uses multiple loss functions for calculation and backpropagation, so as to endow the generator with different parameter values and select the best one as the evolution result each time to enter the next iteration and 2) in the training process, we preserve the optimal generator as memory cells to avoid the performance degradation of the generator. Through these changes, HIEGAN overcame the defects of GAN, improved the stability of GAN, and finally improved classification efficiency. At the same time, in order to alleviate the overfitting problem of depth network under small samples, we change convolution and deconvolution into ghost module to reduce the network parameters. Experiments on three classical datasets validate that HIEGAN has encouraging performance in HIC under small samples.

Unsupervised signal modulation clustering is becoming increasingly important due to its application in the dynamic spectrum access process of 5G wireless communication and threat detection at the physical layer of Internet of Things. The need for better clustering results makes it a challenge to avoid feature drift and improve feature separability. This article proposes a novel separable loss function to address the issue. Besides, the high-level semantic properties of modulation types make it difficult for networks to extract their features. An autoencoder structure based on the random Fourier feature (RffAe) is proposed to simulate the demodulation process of unknown signals. Combined with the separable loss of RffAe (RffAe-S), it has excellent feature extraction ability. Great experiments were carried out on RADIOML 2016.10 A and RADIOML 2016.10 B. Experimental evaluations on these datasets show that our approach RffAe-S achieves state-of-the-art results compared to classical and the most relevant deep clustering methods.

With the fast development of urbanization and motorization, an increasing number of people choose to buy private cars to fulfill their daily travel needs. In particular, many people from various positions of the city drive their cars to specified areas, and then they will stop and stay for a certain period of time, leading to a spatiotemporal evolution of urban area attractiveness (AA). In this paper, we aim at understanding urban AA based on analyses of private car trajectory datasets. Specifically, by extracting point-of-stop (PoS) data from the private car trajectories, we design the variational Bayesian Gaussian mixture models (VBGMM) to deduce the probability density distribution of PoSs and connect it to the variation of AA. We establish a deep learning model based on long short-term memory (LSTM) to capture the evolution of the AA. Furthermore, we integrate dropout in the LSTM method to address challenging issues such as overfitting and time-consuming training of complex neural networks in the AA prediction. We conduct experiments by using real-world private car trajectory data to evaluate the performance of the proposed method. The results validate that our proposed method outperforms existing ones in terms of various metrics. To the authors’ knowledge, our work is the first one to utilize private car trajectory data to study urban area attractiveness, thereby facilitating a new perspective regarding an understanding of human travel behavior and the evolution of urban mobility.

Hyperspectral image has a large amount of data and many feature channels. If the hyperspectral image classification model is too complex, it is easy to cause low efficiency. In order to reduce the amount of network calculation and model parameters, improve the operation efficiency, and make full use of the characteristics of hyperspectral data, this paper uses the Ghost module to reduce the complexity of the model, and combined with the extended morphological profile (EMP) features, proposes a hyperspectral image classification method based on extended morphological profile features and Ghost module (GhostEMP). The experimental results show that the proposed method can improve the efficiency of operation while ensuring the operation efficiency of the network model.

This letter explores how to apply a generative adversarial network (GAN) to the classification of hyperspectral images (HSIs) to obtain a smooth training process and better classification results. To this end, the ideas of the progressive growing GAN (PG-GAN) and Wasserstein generative adversarial network gradient penalty (WGAN-GP) are combined to propose a new method for HSI classification. PG-GAN is optimized from the training process of generating adversarial networks. It gradually increases the depth of the network and the size of the input image, making the training smoother. WGAN-GP is optimized in terms of the loss function. The gradient penalty method is used to solve the problems of vanishing gradient and exploding gradient, making the training more stable. Based on the combination of the two methods, a classifier is added to the model so that it can complete the HSI classification task. The proposed method is evaluated over two publicly available hyperspectral data sets, the Indian Pines and University of Pavia data sets. The results show that the proposed method can achieve good training results with only a small amount of labeled training data.

Benefited from the rapid development of deep learning, object detection in natural image has made great improvements. However, since the size of the optical remote sensing video is very large while the size of airplane is very small, airplane detection in optical remote sensing video still faces a lot of challenges. In this article, we aim at a novel approach for airplane detection in optical remote sensing video. The proposed approach utilizes spatial features from structured forests edge detection and temporal features from neighboring frames. It is capable of circumventing existing challenges and running at a high speed for practical applications. To realize this goal, edge detection results of optical remote sensing video frames are obtained from structured forests edge detection method. Afterwards, improved frames differencing method is utilized to extract temporal features. Finally, airplane detection result is generated by deep neural networks with extracted spatial and temporal features. Our experiments demonstrate that our method has a great breakthrough on the precision and recall of airplane detection in optical remote sensing video.

With the ubiquitous demand for wireless communications, researchers have studied heterogeneous networks (HetNets) for years. Often the HetNets include a macrocell base station (MBS), several sets of macrocell users (MUs), a large number of femtocell base stations (FBSs), and femtocell users (secondary users), where the femtocells help the macrocell system relay the uplink or downlink traffic between the MUs and the MBS. In this article, we propose a novel joint information and energy cooperation method, with the aim of enhancing the spectrum and energy efficiency (EE) for cognitive HetNets. Specifically, the MUs and the femtocells harvest wireless energy from the radio frequency signal transmitted by MBS. By using the harvested energy, femtocells obtain the transmission opportunity to forward the signals of their serving users. We theoretically derive the theoretical expressions of the outage probabilities of the primary link as well as the secondary link. Then, we focus on investigating how to maximize EE by jointly considering time allocation and power control. Furthermore, we formulate the EE maximization problem, which contains the fractional form objective function and the linear inequality constraints and hence is nonconvex. To resolve this, we integrate the Dinkelbach method with convex optimization to derive the tractable and optimal solution. The numerical results demonstrate the simulations well match our theoretical analysis. Moreover, the results validate the feasibility of the proposed method for high-quality transmission without incurring extra energy consumption.

In order to provide a robust estimate of vehicle position in all environments, especially, in challenging urban areas where GPS signals are blocked, a fusion framework based on sparse Gaussian-Wigner prediction (SG-WP) is proposed. This new approach combines the advantages of both the random matrix theory and the sparse property to provide enhanced vehicle localization capabilities. In this method, measurement noises are assumed to be non-Gaussian distributed, and a generalized error distribution is adopted as an approximation to non-Gaussian densities. To ensure the robustness and the stability of the proposed approach, road-test experiments in various scenarios, including free, partial, and complete GPS outages, were performed based on the geometric dilution of precision metric. During complete outages, the SG-WP fuses all available INS measurements to improve the vehicle position prediction, whereas in free outages, only GPS information is processed. Besides, information from both GPS and INS are taken as inputs during partial outages, and the slide window is then introduced to regulate the flow data. The experimental comparison with the existing prediction methods reveals that the proposed method can achieve accurate and reliable positioning for land vehicles in all considered environments when the measurement noises are Gaussian or non-Gaussian distributed.

A road network is a critical aspect of both urban planning and route recommendation. This article proposes an efficient approach to build a fine-grained road network based on sparsely sampled private car trajectory data under complex urban environment. In order to resolve difficulties introduced by low sampling rate trajectory data, we concentrate sample points around intersections by utilizing the turning characteristics from the large-scale trajectory data to ensure the accuracy of the detection of intersections and road segments. In front of complex road networks including many complex intersections, such as the overpasses and underpasses, we first layer intersections into major and minor one, and then propose a simplified representation of intersections and corresponding computable model based on the features of roads, which can significantly improve the accuracy of detected road networks, especially for the complex intersections. In order to construct fine-grained road networks, we distinguish various types of intersections using direction information and detected turning limit. To the best of our knowledge, our road network building method is the first time to give fine-grained road networks based on low-sampling rate private car trajectory data, especially able to infer the location of complex intersections and its connections to other intersections. Last but not the least, we propose an effective parameter selection process for the Density-Based Spatial Clustering of Applications with Noise based clustering algorithm, which is used to implement the reliable intersection detection. Extensive evaluations are conducted based on a real-world trajectory dataset from 1,345 private cars in Futian district, Shenzhen city of China. The results demonstrate the effectiveness of the proposed method. The constructed road network matches close to the one from a public editing map OpenStreetMap, especially the location of the road intersections and road segments, which achieves 92.2% intersections within 20m and 91.6% road segments within 8m.

The proportion of an aurora region in a field of view is an important index to measure the magnetic stress stored in the magnetosphere. Detecting the aurora region is a necessary step to obtain the index. Intensity inhomogeneity, a characteristic of overlaps between the ranges of intensities in segmented regions, has become a challenging issue in the field of auroral oval segmentation. Classical auroral segmentation methods can reasonably detect auroral ovals in clean images. The segmentation quality of these methods deteriorates when auroral oval pixel intensities are not distinct from the background. To reduce the negative influence of intensity inhomogeneity in auroral oval segmentation, a gradually evolved active contour model employing the narrow-band technique instead of using a full region computation is designed. In such case, only the region near auroral oval boundaries has to evolve in each iteration, thus enabling the contour to evolve gradually and saving computational resources. Experimental results demonstrate that the proposed method detects more accurate auroral oval regions than traditional methods in terms of human visual perception and segmentation accuracy.

For hyperspectral imagery (HSI) classification, many works have shown the effectiveness of the spectral-spatial method. However, some previous works using neighboring information assumed that all neighboring pixels make an equal contribution to the central pixel, which is unreasonable for heterogeneous pixels, especially near the boundary of a region. In this letter, a nonlocal self-similarity based on the sparse coding method, followed by the use of a support vector machine classifier, is proposed to improve classification performance. Inspired by the success of nonlocal means, a new nonlocal weighted method is developed to determine the relationship between a test pixel and its neighboring ones. The nonlocal weights are determined by using the spectral angle mapper algorithm, which can exploit the spectral information of surface features. The experiments validate the superiority of our proposed method over existing approaches for HSI classification.

In this letter, we propose a shape-adaptive (SA) reversible integer lapped transform (SA-RLT) method. The new method can deal with arbitrarily shaped image areas while guaranteeing completely reversible integer-to-integer transform. Based on SA-RLT and object-based set partitioned embedded block coder, a new region-of-interest (ROI) compression scheme is designed for 2-D remote sensing images. Numerical experiments reveal that SA-RLT performs better than integer SA discrete wavelet transform, and the new ROI compression scheme performs comparably even better than the JPEG2000-ROI scheme. Advantages in hardware implementation have been preserved by SA-RLT, such as parallel processing and low memory requirement.