The detection of new Multiple Sclerosis (MS) lesions refers to identifying newly appeared or progressing lesion areas within specific time periods from MS-related regions in longitudinal medical images. In recent years, many deep learning methods have been proposed for this field. However, existing methods often fail to fully utilize prior knowledge about MS lesions to suppress interference from non-lesion signals. In this paper, we propose the Lesion-Guided New Lesion Detection Network (LGNLDNet) for enhancing the detection of new MS lesions. Specifically, The model first learns MS lesion-related prior knowledge, enabling effective extraction of lesion-specific features during the global encoding of images. Since globally encoded image features inherently contain a large amount of non-lesion signals, lesion features are prone to being overshadowed by these non-lesion signals when the model analyzes new lesions. Therefore, the model further extracts prior knowledge represented by local lesion features. This prior knowledge is used to construct prompt information. The proposed Lesion-Guided Differential Enhancement Feature Fusion Module (LG-DEFF) then employs the prompt to suppress irrelevant background interference, thus effectively capturing discriminative features of newly emerging MS lesions. Additionally, we introduce the Multi-Level Feature Fusion Module (MLFF), which is cascaded in a pyramidal structure to effectively fuse complementary information from feature maps at different levels. Experimental results on the MICCAI-21 dataset demonstrate that the proposed method outperforms state-of-the-art approaches.

Self-supervised low-dose computed tomography (LDCT) denoising remains challenging due to the difficulty of recovering fine texture details while suppressing structured artifacts. To address these limitations, we propose the Semantic-Guided Artifact-Aware Diffusion model (SGAA). SGAA is designed with a two-stage denoising framework. In the first stage, it leverages multi-scale semantic features of NDCT extracted from a unified latent space to guide the reverse process, thereby mitigating prediction errors caused by domain shift and achieving the removal of random noise in LDCT. In the second stage, the artifact-aware mechanism identifies artifacts by utilizing the attenuation characteristics of unknown noise in the iterative reverse process, so as to eliminate residual artifacts from the first stage. Experiments on the Mayo LDCT dataset demonstrate that SGAA outperforms state-of-the-art methods in both quantitative metrics and visual fidelity when only using NDCT image data.

In clinical practice, MR images with incomplete modalities lead to severe degradation of tumor segmentation performance. We propose an incomplete modality Segmentation network based on shared latent feature compensation (SLFCNet) to address the challenge of robust segmentation in the presence of arbitrary missing modalities. Specifically, SLFCNet includes a set of hybrid encoders, a shared encoder, and a Multimodal correlation Modeling Module (MCMM). The hybrid encoder employs the mamaba mechanism to extract modality-specific features; the shared encoder extracts shared latent features. The MCMM introduces the Transformer to realize effective information fusion of the two types of features. The fused features have richer semantic information and can produce more accurate segmentation results. Comprehensive experiments on BraTS2018 and BraTS2021 datasets verify the effectiveness of our method.

Radiology report generation aims to provide comprehensive clinical descriptions and ease radiologists' workloads. Previous research has explored using knowledge graphs and auxiliary classification tasks to enhance the model's ability to generate accurate reports. However, due to the lack of information in the knowledge graphs or insufficient class label information, these methods fail to provide models with clinical severity information about the same disease at different stages of development, resulting in less accurate reports. To address this issue, we propose a Severity-Guided Radiology Report Generation method (SR2Gen), which guides the model in identifying internal severity variations of the disease from both explicit and implicit dimensions. Specifically, SR2Gen includes two innovative modules: a Knowledge Enhancement Module (KEM) and a Disease Severity-Aware Module (DSAM). First, KEM explicitly guides the report generation model by constructing a knowledge graph containing disease severity information as prior knowledge. Secondly, DSAM enhances the severity-aware classifier using pseudo-labels generated through momentum distillation and further incorporates an adaptive disease severity learning method, implicitly guiding the model to learn disease progression. Extensive experiments and analyses on IU X-Ray and MIMIC-CXR datasets demonstrate that SR2Gen outperforms previous state-of-the-art methods.

The automatic analysis of student behavior in classroom environments poses substantial challenges in computer vision, encompassing fine-grained action recognition, long-term multi-object tracking, and the integration of heterogeneous video data sources. In this paper, we propose a novel multi-modal framework for the spatiotemporal recognition and visualization of student behavior in technology-enhanced classrooms. Our system processes two synchronous video streams: a primary classroom video feed and a secondary screen capture of the instructor's computer screen. The core of our method is a dual-stream processing pipeline. The visual stream performs student detection, classifies seven distinct behaviors using Stu-B-YOLO, an enhanced YOLO-based model, and maintains consistent student identities across 2,700 frames via a coordinate-based tracking algorithm. Simultaneously, the screen analysis stream employs an optical character recognition (OCR) module to detect and temporally localize interactive learning phases, which are identified by specific on-screen elements from the “Rain Classroom” platform. A key contribution is our context-aware fusion module, which dynamically recalibrates the semantic interpretation of certain behaviors—for instance, reclassifying “smartphone usage” as active participation—during these identified interactive intervals. We construct a dataset of 9,000 annotated images to train and evaluate our Stu-BYOLO. Experimental results demonstrate that our framework achieves robust performance in behavior recognition and generates powerful, interpretable visualizations of classroom dynamics, thereby offering a valuable tool for automated educational analytics.

Confronted with the increasing emergency of hyperspectral remote sensing categories in the dynamic environment, traditional classification models that depend on fixed-category labeled data encounter difficulties on new classes recognition. Novel class discovery (NCD) aims to discover unknown class-disjoint novel classes in an unlabeled dataset with the pre-existing knowledge of known classes. Notably, the critical goal of NCD is to ensure recognition accuracy of known classes while identifying new ones. In this paper, we propose a class-relation perceptive distillation with prototype-level clustering prediction network (CRPD-PCP) for NCD of hyperspectral image (HSI). The proposed framework comprises an initial training stage (ITS) and a novel class discovery stage (NCDS) with two essential modules. Specifically, we present the class relation perceptive distillation (CRPD) module, which imposes a similarity constraint on the prediction of the distribution of new class data over the models of two stages. With the CRPD operated on the NCDS, our model effectively captures class relation information in spectral-spatial domain between known and novel classes of HSI to avoid forgetting old knowledge. Besides, we establish the prototype-level clustering prediction (PCP) module to generate high-confidence pseudo-labels for unlabeled novel classes. To be specific, we progressively cluster samples with the same spectral angular distance from the perspective of prototypes, and the self-supervised prototype-level knowledge distillation strategy in PCP facilitates effective identification of new categories. Experiments conducted on four datasets demonstrate that the CRPD-PCP model generates superior performance compared to other NCD methods for HSI. Our code will be released at https://github.com/Chirsycy/CRPD-PCP.git.

Self-attention-based approaches that leverage global context information for hyperspectral image (HSI) classification have gained increasing prominence. Nevertheless, due to the assignment of equivalent attention weight to all the tokens (pixels or patches), the existing self-attention mechanism inadvertently prioritizes the non-label-specified information over the instinct label-specified information, which generates attention shifts and redundancy in HSI classification. To alleviate the mentioned barrier, we propose the center-specific perception transformer network (CP-Transformer), which is the first attempt to perform class-guided attention and filter interference factors for HSI classification feature representation. Specifically, the central-pixel focus attention module (CFA) is presented to compute the label-related attention between the center and other pixels. In this manner, CFA reduces computational complexity and closely aligns with the center-pixel labeling strategy. Besides, the spectral saliency focus attention module (SSFA) is developed to capture the spectral correlation by focusing salient bands to provide a beneficial supplement for spatial features. Moreover, the hierarchical integration network (HIN) constructs the inference network to integrate and rectify spatial-spectral features for HSI classification. The experiment results on four popular HSI datasets demonstrate that the proposed method achieves robust performance compared to other state-of-the-art methods. Our code will be released at https://github.com/Chirsycy/CP-Transformer.

Semantic segmentation in remote sensing images (RSI) assigns unique semantic labels to each pixel and plays a crucial role in real-world applications such as environmental change monitoring, precision agriculture, and economic assessment. Although convolutional neural networks (CNN) and Transformer-based models for semantic segmentation of RSI have achieved remarkable success, existing approaches still struggle to accurately detect weak edges and occluded objects due to the complexity and fuzziness of edges in RSI. To overcome this obstacle, we propose a novel probability-guided edge enhancement network (PEEN) for semantic segmentation of RSI, which is the first attempt to leverage the probability function to guide the segmentation model in performing edge prediction for RSI. Specifically, in the feature extraction stage of PEEN, we present a convolutional self-attention mechanism to enhance the global feature representation of the encoder-decoder network. In the edge enhancement stage of PEEN, we innovatively build an iterative probability-guided edge prediction module to refine edge prediction mathematically and iteratively. With the cooperation of the mentioned two stages, the proposed model yields precise segmentation of the objects and edge portions in RSI. Experiment results and analysis demonstrate that the PEEN model outperforms the existing popular CNN-based and Transformer-based models in semantic segmentation with 85.54% and 88.35% of Mean Intersection Over Union (mIOU) on the Vaihingen and Potsdam test datasets. Our code is available at https://github.com/Zyk517/PEEN.

Determining the occlusal position in CBCT images is a critical step in the digital virtual articulator treatment of Anterior Disc Displacement with Reduction (ADDwR). Current oral segmentation methods primarily rely on supervised learning, which typically requires a large dataset. However, acquiring oral datasets is complex and challenging, making semi-supervised learning more suitable. Current semi-supervised segmentation methods have several limitations. The perturbations used in consistency-based semi-supervised methods are often manually designed, which can introduce negative biases detrimental to training. Furthermore, semi-supervised learning often faces an empirical mismatch between labeled and unlabeled data. When these two data types are handled independently or without alignment, significant information derived from labeled data may not be fully utilized. We propose a novel semi-supervised framework X-Match for oral jawbones segmentation. The X-Match utilizes wavelet transforms to extract low-frequency and high-frequency information for consistency training, reducing the learning bias caused by manual perturbations. Furthermore, it combines labeled with unlabeled data bidirectionally during training, allowing unlabeled data to acquire comprehensive shared features from labeled data. Experimental results demonstrate that our method outperforms baselines and achieves superior performance in oral jawbones dataset.

In video analysis, existing approaches achieve excellent performance under preset conditions, but the scarcity of training videos limits deep learning development. To address this, this paper introduces two external resources to alleviate the video shortage: image data and cross-institutional data. We propose a novel method, IOLCIA, which combines image-only learning and cross-institutional learning to maximize the use of image resources. Additionally, it leverages the differences and similarities in data from various institutions to enhance feature learning. Comparison experiments show that our method outperforms models trained solely on images, with only a slight decrease in effectiveness compared to those trained on videos.

This paper presents a new dual domain network with image prior based on Convolutional Neural Network (CNN) and Transformer simultaneously for CT Metal Artifact Reduction (MAR). Challenges in MAR derive from the following aspects: firstly, the different morphologies of metal artifacts complexify resolving the issue just in a single domain; secondly, albeit many methods excel in quantitative metrics, yet the restored anatomical structures are over-smooth blurring reconstructed CT images; thirdly, MAR demands better performance as a clinical application, but the approaches relying on CNN or Transformer struggle due to CNN’s restricted spatial scope and Transformer’s ignorance to the local details, respectively, that is, CNN focuses on the local information while Transformer emphasizes the global information with higher computational complexity. To address these problems, we put forward MAReraser, a novel dual domain network, to deal with metal artifacts. MAReraser removes metal artifacts in both the projection and image domains, effectively reducing heteromorphic metal artifacts. Moreover, MAReraser introduces the image prior generated by an image prior subnet to refine the quality of reconstructed CT images. The prior subnet is pretrained in an expanded dataset which incorporates CT images corrected by diverse traditional MAR methods, providing extra potential prior knowledge from different perspectives. Further, the network backbone of MAReraser integrates CNN and Transformer, enabling complementary local and global feature extraction and balancing computational complexity. Extensive experiment results demonstrate that our method outperforms several other approaches whether in quantitative metrics or in qualitative visualization results.

Due to limitations of satellite imaging systems, hyperspectral image (HSI) often suffers from incomplete coverage, with certain regions of the study area missing. Data fusion and reconstruction are effective approaches to resolve the contradiction in spatial and spectral domains, where related theories have intensively developed in recent years. However, existing fusion methods are mostly applicable to simulated data and are challenging to apply to real data. In this paper, we propose an unsupervised fusion spatial reconstruction network namely UFSRnet, which not only reconstructs the missing regions of HSI but also learns the differences between heterogeneous data adaptively. Specifically, a sensor radiation deviation correction (SRDC) module is designed to tackle the disparities between heterogeneous data adaptively. The model demonstrates commendable performance across both simulated and real data sets.

Recently, the methods based on self-attention mechanisms have gained increasing prominence in hyperspectral image classification (HSIC). However, the existing self-attention mechanism suffers the challenge of attention shift and redundancy. To address the problem, we propose the center category focusing transformer network (CCSF-Transformer) for HSIC, which is designed to resolve attention shifts and redundancy by balancing the multiple category features. Specifically, the central-category-focused attention mechanism (CFA) is presented in the proposed framework to compute the category-matched attention between the center pixel and neighbor pixels, closely matching the center-pixel style labeling strategy, and reducing the computation complexity by excluding the computation between interference pixels. Besides, the spectral-salient-focused attention module (SFA) is developed to capture the spectral correlation, which concentrates on the salient bands and suppresses the expression of redundant bands. Moreover, the hierarchical integration network (HIN) is built to rectify spatial and spectral features The experiment results on two popular HSI datasets demonstrate that the proposed method achieves robust performance compared to other state-of-the-art methods.

Change detection (CD) in remote sensing imagery is identified as a pivotal task in the field of Earth observation, while it usually confronts the dilemma of intricate data and minor alterations. To address the stated challenge, this letter presents an innovative frequency-temporal attention network for CD (FTAN), which incorporates two advanced modules including the multidimensional convolutional frequency attention module (MCFA) and the interactive attention module (IAM). Specifically, the MCFA module is essential for enhancing sensitivity in CD by merging multiscale spatial and frequency domain features. As a supplement to MCFA, the IAM aggregates category-related tokens and processes cross-attention information from different time phases. The seamless integration of MCFA and IAM empowers the FTAN network with enhanced capabilities to detect minor regions and edges accurately. Experiments on datasets like LEVIR-CD and DSIFN-CD demonstrate superior performance by outperforming existing models in F1 scores and IoU metrics. Our code and pretrained models will be released athttps://github.com/chirsycy/FTAN.

To mitigate the domain shift and enhance the alignment of the spatial-spectral features, this letter proposes a novel dual-intervention-constrained mask-adversary (DICMA) framework for unsupervised domain adaptation (UDA) of hyperspectral image classification (HSIC). Innovatively, DICMA integrates a generator, masker, and bi-classifier within an adversarial framework constrained by a dual intervention mechanism. Specifically, the correlation intervention module (CIM) ensures the preservation and independence of causal spatial-spectral variables, while the knowledge distillation intervention module completes the spatial-spectral generalization with constrained distillation information. Besides, with the collaborative adversarial training strategy, the proposed approach transfers effective knowledge for spatial-spectral feature alignment. Experimental results and analyses demonstrate the effectiveness of the proposed DICMA model, which yields an accuracy of 91.15% in the Pavia University (PaviaU)$\to $Pavia Center (PaviaC). Our code will be released athttps://github.com/Chirsycy/DICMA.

Oriented to adaptive recognition of the new land-cover categories, incremental classification (IC) that aims to complete adaptive classification with continuous learning is urgent and crucial for hyperspectral image classification (HSIC). Nevertheless, deep-learning-based HSIC models adopted the learning paradigm with fixed classes yield unsatisfactory inference in the situation of IC due to the catastrophic forgetting problem. To eliminate the recognition gap and maintain the old knowledge during IC, in this paper, we propose a novel approach called the distillation-constrained prototype representation network (DCPRN) for hyperspectral image incremental classification (HSIIC). The primary goal of DCPRN is to enhance the discriminative capability for recognizing the original classes in HSIIC, while effectively integrating both the original and incremental knowledge to facilitate adaptive learning. Specifically, the proposed framework incorporates a prototype representation mechanism, which serves as a bridge for knowledge transfer and integration between the initial and incremental learning phases of HSIIC. Additionally, we present a dual knowledge distillation module in incremental learning, which integrates discriminative information at both the feature and decision level. In this way, the proposed mechanism enables flexible and dynamic adaptation to new classes and overcomes the limitations of fixed-category feature learning. Extensive experimental analysis conducted on three popular data sets validates the superiority of the proposed DCPRN method compared with other typical HSIIC approaches.

Hyperspectral image classification (HSIC) models have made remarkable progress in the last decade. Nevertheless, the downsized mapping in the convolutional neural network (CNN) and down-sampled mechanism in the transformer-based approach amplify the loss of hidden knowledge in the subpixel that encompasses crucial yet unseen features within a single pixel. Considering this aspect, the mentioned popular solutions for HSIC contradict the inherent characteristic of hyperspectral data. To address this issue, we rethink the size factor in CNN and propose a novel spatial mapping expansion with spectral compression (SMESC) network for HSIC. Specifically, the SMESC builds a mapping expansion network to mine unseen information in subpixels with enlarged feature maps. A channel modulation residual block (CMRB) is developed to compress spectral redundancy and promote salient channels with modulation information. Moreover, we design a multiple-size training strategy to substitute the traditional multiple feature extraction (FE) branches and improve the model adaptation to the different sizes of the testing samples. The extensive experimental results and analysis of four hyperspectral image (HSI) datasets demonstrate the superiority of the proposed architecture compared to other advanced HSIC methods. Our code will be released athttps://github.com/Chirsycy/SMESC.

High-quality Computed Tomography(CT) plays a vital role in clinical diagnosis, but the presence of metallic implants will introduce severe metal artifacts on CT images and obstruct doctors’ decision-making. Many prior researches on Metal Artifact Reduction(MAR) are based on Convolutional Neural Network(CNN). Recently, Transformer has demonstrated phenomenal potential in computer vision. Also, transformer-based methods have been harnessed in CT image denoising. Nevertheless, these methods have been little explored in MAR. To fill the gap, we put forth, to the best of our knowledge, the first transformer-based architecture for MAR. Our method relies on a standard Vision Transformer(ViT). Furthermore, we tap into the progressive tokenization to refrain from the simple tokenization of ViT which gives rise to inability to model the local anatomical information. Additionally, for the sake of facilitating the interaction among tokens, we take advantage of cyclic shift from Swin Transformer. Finally, many experiment results reveal that the transformer-based technique is superior to those on the basis of CNN to some degree.

Hyperspectral images provide spectral curves that reflect the "fingerprint" properties of substances, making them suitable for many applications. Thanks to the rapid development of computing resources, deep learning algorithms can significantly improve the cognitive ability of the network by extracting hidden features, and have been successfully applied to hyperspectral image processing, such as classification and detection. In this paper, a new hyperspectral target detection model based on one-dimensional generative adversarial networks (1D-GAN) is proposed. The proposed 1D-GAN network is designed to extract HSI features, and the probability is calculated accordingly whether the pixel to be detected is a target or background. In order to capture the spatial features, the guided filter is then used to obtain the final detection map. Performance comparison with several state-of-the-art methods demonstrate the effectiveness and efficiency of the proposed 1D-GAN algorithm.

A novel multi-scale fusion maximum entropy subspace clustering (MFMESC) for hyperspectral image (HSI) band selection is proposed in this paper. Subspace clustering is combined as a self-expression layer with stacked convolutional autoencoder, so that subspace clustering working in linear subspaces can deal with complicated HSI data with nonlinear characteristics. Multiple fully-connected linear layers are inserted between the encoder layers and their corresponding decoder layers to promote learning more favorable representations for subspace clustering. A multi-scale fusion module is designed to guide the fusion of multi-scale information extracted from different layers to learn a more discriminative self-expression coefficient matrix. Furthermore, the maximum entropy regularization is introduced in the subspace clustering to promote the connectivity within each subspace. Experimental results demonstrate the superiority of the proposed model against state of-the-art methods.

Unsupervised domain adaptation (UDA) attempts to boost the performance on an unlabeled target domain by transferring knowledge from a labeled source domain. The previous models consider domain-level discrepancy while neglecting content-level distinction. To further decrease the distribution gap between different domains, this letter proposes a novel UDA approach with content-wise alignment for hyperspectral image classification (HSIC). We accomplish feature alignment with content-wise discrepancy reduction through an adversarial framework for the first time. Expressly, the core of the proposed content-wise scheme is integrated with a class-level and style-perceive-level regularized alignment to strengthen the representation of invariant feature. The experimental analysis demonstrates that the proposed model achieves more effective performance than other domain adaptation methods for hyperspectral image (HSI).

Band selection (BS) aims to choose a salient subset implied sufficient information from the numerous bands, which supplies a significantly efficient way to alleviate the barrier of dimensionality disaster for hyperspectral image classification (HSIC). This letter develops a semisupervised BS approach based on dual-constrained low-rank representation BS (DCLRR-BS) with two regularizations for HSIC. To be specific, a low-rank representation model is first proposed with super-pixel and imbalanced class-wise constraints, which are explicitly integrated to improve the performance of the band description. Next, the clusters are built adaptively based on graph theory in an unsupervised manner to rapid selection efficiency. A selection criterion is last designed to highlight the prominent band of each subset cluster to fulfill the BS procedure. Experimental results conducted on four types of classifiers with two real hyperspectral image (HSI) data sets demonstrate that the proposed DCLRR-BS method performs well in the imbalanced HSIC area.

To achieve real-time abundance estimation of hyperspectral images and improve the accuracy and efficiency of estimation, this article proposes a new band processing approach for abundance estimation, to be called sequential band fusion (SBF). To achieve SBF, a new band priority mechanism is proposed. It is derived from the concept of orthogonal subspace projection (OSP) by orthogonalizing undesired targets using projection, while minimizing the variance resulting from the background. By taking advantage of OSP, the interfering effects caused by all undesired targets can be eliminated and then the detector produced by a target of interest can be further used as a measure of prioritizing bands as well as a means of searching bands for this particular target. As a result, two ranking-based band priority criteria (RP), called MaxOSP-RP and MinOSP-RP, and two searching-based band priority criteria (SP), called sequential feed forward band search (SFBS) and sequential backward band search (SBBS), can be derived. We provide a detailed theoretical description and formula derivation of the SBF and combine it with band sequence (BSQ), RP and SP to propose three different fusion mechanisms, SBF-BSQ, SBF-RP and SBF-SP to make the fusion mechanism applied to different scenarios. Experimental results show that the proposed methods performs well for abundance estimation.

This article presents a new approach, called progressive band subset fusion (PBSF) for hyperspectral anomaly detection. Unlike band selection (BS) which selects bands according to band prioritization or band search strategies, PBSF fuses band subsets progressively during data collection processing. It is completely opposite to BS that must be done after data are acquired and then select bands by removing spectral redundancy as post-data processing. To accomplish PBSF, two versions of PBSF are derived: PBSF of the multiple-band subset (PBSF-MBS) and PBSF of uniform BS (PBSF-UBS). In particular, the fusion process takes place in an anomaly detector from a real-time processing perspective. Three approaches are developed to realize PBSF of two-band subsets simultaneously: PBSF-band sequential (PBSF-BSQ), PBSF-RT, and PBSF-zigzag. Extensive experiments demonstrate that PBSF has advantages over BS in many ways.

This article proposes a new approach to hyperspectral band processing for anomaly detection, to be called sequential band fusion (SBF), derived from the band sequential (BSQ) data acquisition format used by a hyperspectral imaging sensor which fuses one single band at a time with previously fused band subset sequentially. In order to realize SBF, four versions, SBF-BSQ, initial band driven SBF (IBD-SBF), band prioritization SBF (BP-SBF), and band selection SBF (BS-SBF), are developed. Furthermore, to validate the sequentially fused results by SBF identical to that produced by combining all joint bands together, its mathematical theory and derivations are also presented. Finally, the full utility of SBF in anomaly detection is demonstrated through extensive experiments.

When predicting data for which limited supervised information is available, hyperspectral target detection methods based on deep transfer learning expect that the network will not require considerable retraining to generalize to unfamiliar application contexts. Meta-learning is an effective and practical framework for solving this problem in deep learning. This article proposes a new meta-learning based hyperspectral target detection using Siamese network (MLSN). First, a deep residual convolution feature embedding module is designed to embed spectral vectors into the Euclidean feature space. Then, the triplet loss is used to learn the intraclass similarity and interclass dissimilarity between spectra in embedding feature space by using the known labeled source data on the designed three-channel Siamese network for meta-training. The learned meta-knowledge is updated with the prior target spectrum through a designed two-channel Siamese network to quickly adapt to the new detection task. It should be noted that the parameters and structure of the deep residual convolution embedding modules of each channel in the Siamese network are identical. Finally, the spatial information is combined, and the detection map of the two-channel Siamese network is processed by the guiding image filtering and morphological closing operation, and a final detection result is obtained. Based on the experimental analysis of six real hyperspectral image datasets, the proposed MLSN has shown its excellent comprehensive performance.

Despite continuing to progress in hyperspectral image classification (HSIC) based on deep learning, the classification accuracy is limited to furtherly improve in the absence of labeled samples. To address this issue, the metric-based prototypical networks for few-shot learning have enjoyed widespread popularity. However, the conventional prototypical networks are vulnerable to the selected examples and fail to accomplish representative predictions for the prototypes in complicated situations. In this paper, we propose a multi-view calibrated prototype-learning framework for few-shot HSIC, which consists of three rectified strategies from different views to improve the robustness of prototypes in the embedding space. Specifically, the calibrated aggregation network is the first presented to calibrate the representations with local patches aggregation for the enhancement of the prototypes. Moreover, to improve the compactness of the intraclass expression, the calibrated metric learning with regularization terms is designed to strengthen the discrimination of the prototypes. Furthermore, we calibrate the feature distribution of supervised samples by transferring statistical knowledge to eliminate the local bias in the test phase. The extensive experimental results and analysis of three hyperspectral image datasets demonstrate the superiority of the proposed architecture compared with other advanced methods.

Hyperspectral image classification (HSIC) methods based on convolutional neural network (CNN) continue to progress in recent years. However, high complexity, information redundancy, and inefficient description still are the main barriers to the current HSIC networks. To address the mentioned problems, we present a spatial-spectral dense CNN framework with a feedback attention mechanism called FADCNN for HSIC in this article. The proposed architecture assembles the spectral-spatial feature in a compact connection style to extract sufficient information independently with two separate dense CNN networks. Specifically, the feedback attention modules are developed for the first time to enhance the attention map with the semantic knowledge from the high-level layer of the dense model, and we strengthen the spatial attention module by considering multiscale spatial information. To further improve the computation efficiency and the discrimination of the feature representation, the band attention module is designed to emphasize the weight of the bands that participated in the classification training. Besides, the spatial-spectral features are integrated and mined intensely for better refinement in the feature mining network. The extensive experimental results on real hyperspectral images (HSI) demonstrate that the proposed FADCNN architecture has significant advantages compared with other state-of-the-art methods.

The current hyperspectral image classification (HSIC) model based on the convolutional neural network for feature extraction and softmax classifier has been prone to the barrier of label prediction with limited samples. Substituting for the enormously complicated work of terrain labeling, few-shot learning provides a popular option for HSIC with very few annotated samples. In this paper, we proposed a novel edge-inferring framework with the meta-learning paradigm for hyperspectral few-shot classification (HSFSC). In which, a graph neural network for similarity measurement is firstly presented to iteratively infer edge labels with the exploitation of instance-level similarity and the distribution-level similarity. Besides, in the meta-training stage, the pixel prediction model and patch prediction model based on edge inferring architecture are concretized jointly to improve the classification accuracy of the test samples. Expressly, at the meta-testing phase, the dynamic task-guided self-diagnosis strategy is developed for the first time to diagnose the samples separability of the current classification task, which is responsible for dynamically assigning the most reliable results based on the generated reliability grade of the sample. The extensive experimental results and analysis of three hyperspectral image datasets demonstrate the superiority of the proposed HSFSC architecture compared with other advanced methods.

Hyperspectral image (HSI) provided with a substantial number of correlated bands causes calculation consumption and an undesirable "dimension disaster" problem for the classification. Band selection (BS) is an effective measure to reduce the information redundancy with the physics spectrum preserved for HSI. Although the existing BS methods have achieved noticeable progress, the correlation between neighbor bands still needs to be mined deeply for an effective selection criterion. This paper proposes a BS approach to collecting the discriminative band subset for hyperspectral image classification (HSIC), which adopts the self-supervised learning paradigm to implement the BS by auxiliary spectrum rebuilding task. In specific, we utilized a Convolutional neural network (CNN) and Graph Convolutional Network (GCN) for the spectral-spatial feature extraction. Next, GCN and CNN are developed for the refinement of the band correlation sequentially. Afterward, the selected bands in terms of the acquired correlation are fed into the presented self-supervised spectrum rebuilding network for spectral reconstruction. Simultaneously, the proposed architecture completed the selection with the optimization of the band reconstruction by a defined loss function. In this way, we supply substitution for selection criterion and path searching through the end-to-end framework. The extensive experimental results and analysis demonstrated that the proposed hybrid architecture provided a competitive band subset for the classification, and the accuracies with different types of classifiers are more effective than the compared BS methods.

This paper proposes a new hyperspectral anomaly detection method based on transferred deep learning and tensor decomposition. Firstly, since there is no labeled input data for training in anomaly detection, the detection model is obtained by training convolutional neural network with transferred learning. Then the model is decomposed to increase the number of convolution layers, that is, the depth of the network, so as to give more accurate results without over fitting. At the same time, the spatial information of the input data is extracted in order to make full use of the existing data for detection. Finally, combining the spectral and spatial information of the current pixel, the detection result is given. Experiments on two hyperspectral datasets show that the proposed algorithm has excellent performance.

This paper proposes a method to improve the performance of the supervised classification from two aspects. Firstly, the global spatial and local spectral similarity is used to extend the labeled sample size (GLS). Secondly, extended subspace projection (ESP) which projects the original image to a lower-dimensional subspace is used to alleviate band redundancy. Finally, the two implements are combined with the sparse representation classifier (SRC) to optimize the hyperspectral image classification (HSIC). The proposed method is named GLSESP. Experimental results on real hyperspectral data set demonstrate the practicality and effectiveness of GLSESP for HSIC tasks.

Low-rank and sparsity-matrix decomposition (LRaSMD) has received considerable interests lately. One of effective methods for LRaSMD is called go decomposition (GoDec), which finds low-rank and sparse matrices iteratively subject to the predetermined low-rank matrix order m and sparsity cardinality k. This article presents an orthogonal subspace-projection (OSP) version of GoDec to be called OSPGoDec, which implements GoDec in an iterative process by a sequence of OSPs to find desired low-rank and sparse matrices. In order to resolve the issues of empirically determining p = m + j and k, the well-known virtual dimensionality (VD) is used to estimate p in conjunction with the Kuybeda et al. developed minimax-singular value decomposition (MX-SVD) in the maximum orthogonal complement algorithm (MOCA) to estimate k. Consequently, LRaSMD can be realized by implementing OSP-GoDec using p and k determined by VD and MX-SVD, respectively. Its application to anomaly detection demonstrates that the proposed OSP-GoDec coupled with VD and MX-SVD performs very effectively and better than the commonly used LRaSMD-based anomaly detectors.

Wealthy spectral information provided by hyperspectral image (HSI) offers great benefits for many applications in hyperspectral data exploitation. However, processing such high-dimensional data volumes that may result in redundant bands due to its high interband correlation will be a challenge. For target detection and classification, this is particularly true since there may only need a relatively small number of bands that respond one particular target of interest well, while most of other bands do not. Band selection (BS) is a major dimensionality reduction technique to remove the redundant bands and selects a few bands to represent the entire image. However, how to eliminate the effect of uninteresting targets with similar spectra on detection of interesting targets is a severe issue arising in target detection for BS. This article develops a new approach called target-constrained interference-minimized BS (TCIMBS) which can be used to select band subset for specific target detection, while annihilating targets of no interest and suppressing interferers and background. Its idea is derived from target-constrained interference-minimized filter (TCIMF). By taking advantage of TCIMF, two band prioritization (BP) criteria called forward minimum variance BP (FMinV-BP) and backward maximum variance BP (BMaxV-BP) along with their three band search-based BS counterparts called sequential forward TCIMBS (SF-TCIMBS), sequential backward TCIMBS (SB-TCIMBS), and improved SB-TCIMBS (SB-TCIMBS*) are derived. The experimental results suggest that TCIMBS can improve the detection accuracy and also achieve better performance in comparison with several state-of-the-art methods.

Most current multi modalities medical image registration approaches are concerned about registering one modality image to another. However, in the real world, medical image registration may be involved in multiple modes, not just two specific modalities. To this end, we propose a multi-contrast modalities medical image registration modal (Star-Reg net). It uses a single generator and discriminator for all contrasts of registrations amount several modalities. Furthermore, the proposed approach is trained in an unsupervised way, which alleviates the requirement of manual annotation data. The experiment on the IXI dataset demonstrates the Star-Reg net effectiveness in multi-contrast modalities medical image registration.

Classification is a major task in hyperspectral image (HSI) processing. This paper develops an approach by taking advantage of low rank matrix derived from the low rank and sparse matrix decomposition (LRSMD) model which decomposes a hyperspectral data matrix X as X = L+S+n where L, S and n are referred to low rank, sparse and noise matrices respectively. The hyperspectral image classification (HSIC) is then performed on the low rank matrix L rather than the original data matrix X where the well-known go decomposition (GoDec) is used to produce such LRSMD model. To determine the two key parameters used in GoDec, the rank of L, m, and the cardinality of the sparse matrix, k the well-known virtual dimensionality (VD) and minimax-singular value decomposition (MX-SVD) methods are used for this purpose. Finally, to demonstrate advantages of using the low rank matrix L, support vector machine (SVM) and an edge-preserving filters (EPF)-based classifiers are implemented to evaluate classification performance.

This letter introduces a new approach to hyperspectral image classification (HSIC), called progressive band selection processing of hyperspectral image classification (PBSP-HSIC), which performs classification in multiple stages in the sense that each stage performs HSIC progressively according to a specifically selected band subset. Interestingly, such PBSP-HSIC offers a rare view of how different classes are classified in progressive stages, which has never been explored in the past. The experimental results also show that PBSP-HSIC performs better than HSIC using full bands.

This paper presents a new band selection approach, called uniform band interval divided band selection (UBIDBS) which uniformly divides a band range into a finite number of band intervals from which a band can be selected from each band interval according to a custom designed band prioritization (BP) criterion. Two BP criteria are introduced. One is derived from orthogonal subspace projection (OSP). The other is based on correlation matrix R originated from constrained energy minimization (CEM). These two criteria allow users to identify a most significant band to be selected in each of band intervals. As a result, it avoids band decorrelation required by BP to remove adjacent high- correlated bands.

Background (BKG) is an integral part of an image and has significant effect and impact on hyperspectral image classification (HSIC). Unfortunately, how to address the BKG issue has not received much attention over the past years. This paper investigates this issue by developing a mixed pixel classifier, iterative constrained energy minimization (ICEM) and a posteriori classification measure, called precision (PR).

Nowadays, the Convolutional Neural Network is successfully applied to the images object detection. When new classes of object emerges, it is popular to adapt the convolutional neural network based detection model through a retraining process with the new classes of samples. Unfortunately, the adapted model can only detect the new classes of objects, but cannot identify the old classes of objects, which is called catastrophic forgetting, also occurring in incremental classification tasks. Knowledge distillation has achieved good results in incremental learning for classification tasks. Due to the dual tasks within object detection, object classification and location at the same time, a straightforward migration of knowledge distillation method cannot provide a satisfactory result in incremental learning for object detection tasks. Hence, this paper propose a new knowledge distillation for incremental object detection, which introduces a new object detection distillation loss, a loss not only for classification results but also for location results of the predicted bounding boxes, not only for all final detected regions of interest but also for all intermediate regions proposal. Furthermore, to avoid forgetting learned knowledge from old datasets, this paper not only employs hint learning to retain the characteristic information of the initial model, but also innovatively uses confidence loss to extract the confidence information of the initial model. A series of experiment results on the PASCAL VOC 2007 dataset verify the effectiveness of the proposed method.

GAN has recently been proved to be able to generate symbolic music in the form of piano-rolls. However, those existing GAN-based multi-track music generation methods are always unstable. Moreover, due to defects in the temporal features extraction, the generated multi-track music does not sound natural enough. Therefore, we propose a new GAN model with self-attention mechanism, DMB-GAN, which can extract more temporal features of music to generate multi-instruments music stably. First of all, to generate more consistent and natural single-track music, we introduce self-attention mechanism to enable GAN-based music generation model to extract not only spatial features but also temporal features. Secondly, to generate multi-instruments music with harmonic structure among all tracks, we construct a dual generative adversarial architecture with multi-branches, each branch for one track. Finally, to improve generated quality of multi-instruments symbolic music, we introduce switchable normalization to stabilize network training. The experimental results show that DMB-GAN can stably generate coherent, natural multi-instruments music with good quality.

This paper presents a class information (CI)-based band selection (BS) approach to hyperspectral image classification (HSIC). It introduces a new concept from an information theory point of view, CI which can be used to determine an appropriate weight imposed on each class of interest. Specifically, two types of criteria, intraclass information criterion (IC) and interclass IC are derived as CI probabilities to measure CI that can be used to determine the number of training samples required to be selected for each class. With such CI-calculated probabilities, another new concept called class self-information (CSI) is also defined for each class that can be further used to define the class entropy (CE) so that CSI and CE can be used to determine the number of bands required for BS, nBS. In order to find desired nBS bands, two types of BS methods based on CSI and CE are custom-designed, called single class signature-constrained BS (SCSC-BS) which utilizes the constrained energy minimization (CEM) to constrain each individual class signature to select bands for a particular class according to its CSI-determined nBS and a multiple class signatures-constrained BS (MCSC-BS) which takes advantage of linearly constrained minimum variance (LCMV) to constrain all class signatures to select CE-determined nBS bands for all classes. These SCSC-BS and MCSC-BS selected bands are then used to perform classification and evaluated by CI-weighted classification measures by real image experiments. The results show that HSIC using judiciously selected partial bands as well as CI-weighted measures can improve HSIC with using full bands.

This paper develops a new approach to band selection for multiple-target detection, called constrained-target band selection (CTBS). Its idea is derived from the concept of constrained energy minimization (CEM) by constraining a target of interest, while minimizing the variance resulting from the background (BKG). By taking advantage of CEM, the variance produced by a target of interest can be further used as a measure of prioritizing bands as well as a means of selecting bands for this particular target. As a result, two CTBS-based band prioritization (BP) criteria, called minimal variance-based BP (MinV-BP) and maximal variance-based BP (MaxV-BP), and two CTBS-based BS methods, called sequential forward CTBS (SF-CTBS) and sequential backward CTBS (SB-CTBS), can be derived for multiple-target detection. Since the bands selected by CTBS vary with targets of interest used to constrain CEM, in order for CTBS to be applied to multiple targets, a new fusion technique, called band fusion selection (BFS), is further developed for CTBS to integrate bands selected by different targets so that CTBS can work for all targets. Unlike most BS methods for target detection which generally simultaneously select a fixed set of bands for all targets of interest, the ideas of constraining multiple-target detection and using BFS are novelty of this paper. Experimental results show that CTBS performs well for multiple-target detection.

In hyperspectral image classification (HSIC), background (BKG) is generally excluded from consideration due to the fact that obtaining complete knowledge of BKG is nearly impossible in reality. Unfortunately, BKG has significant impact on classification and band selection (BS). This paper investigates both issues and presents a novel approach called class signature-constrained BKG suppression (CSCBS) approach to BS for HSIC, where class signatures can be obtained either by a priori or a posteriori knowledge or training samples, and BKG suppression can be accomplished by taking the inverse of the sample correlation matrix R. Its idea takes advantage of the concept of the linearly constrained minimum variance (LCMV) developed from adaptive beamforming by constraining class signatures of interest while minimizing the effect caused by the unknown BKG so as to enhance the classification performance. There are two immediate applications of CSCBS. One is its application to HSIC, in which it becomes a CSCBS classifier. The other is its use of the LCMV-suppressed BKG as a measure to derive the band prioritization (BP) criteria and BS. Experimental results demonstrate that generally CSCBS does not need the full-band set for HSIC since a partial band subset selected by CSCBS-BP/BS can actually improve the classification results using full-band information.

Anomaly detection (AD) generally finds targets that are spectrally distinct from their surrounding neighborhoods but cannot discriminate its detected targets one from another. It cannot even perform classification because there is no prior knowledge about the data. This paper presents a new approach to AD, to be called a posteriori AD for unlabeled anomaly classification where a posteriori indicates that information obtained directly from processing data is used as new information for subsequent data processing. In particular, a posteriori AD uses a Gaussian filter to capture spatial correlation of detected anomalies as a posteriori information which is included as new information for further AD. In doing so, a posteriori AD develops an iterative version of AD, referred to as iterative anomaly detection (IAD), which implements AD by feeding back Gaussian-filtered AD maps in an iterative manner. It then uses an unsupervised target detection algorithm to identify spectrally distinct anomalies that can be used to specify particular anomaly classes. To terminate IAD, an automatic stopping rule is also derived. Finally, it uses identified distinct anomalies as desired target signatures to implement constrained energy minimization (CEM) to classify all detected anomalies into unlabeled classes. The experimental results show that a posteriori AD is indeed very effective in unlabeled anomaly classification.

This paper develops a new Neyman–Pearson detection approach, to be called band-specified virtual dimensionality (BSVD), to estimating the number of bands required by band selection (BS),$n_{\mathrm {BS}}$, as well as finding desired bands at the same time. Its idea is derived from target-specified virtual dimensionality (TSVD) where targets under hypotheses as signal sources in TSVD are replaced with bands as signal sources and the test statistics derived for a Neyman–Pearson detector (NPD) is signal-to-noise ratio (SNR) that is used to derive orthogonal subspace projection (OSP) approach for hyperspectral image classification and dimensionality reduction. Accordingly, the resulting virtual dimensionality is referred to as OSP-based BSVD. Several benefits resulting from BSVD cannot be offered by the traditional BS methods. One is its direct approach to dealing with$n_{\mathrm {BS}}$. Another is no-search strategy needed for finding optimal bands. Instead, it uses NPD to determine and rank desired bands for band prioritization. Most importantly, it determines$n_{\mathrm {BS}}$and finds desired bands simultaneously and progressively.

Anomaly detection (AD) is designed to find targets that are spectrally distinct from their surrounding neighborhood. Unfortunately, commonly used anomaly detectors generally do not take into account its surrounding spatial information. This paper derives an iterative version of anomaly detection, iterative anomaly detection (IAD) to address this issue. Its idea is to use a Gaussian filter to capture spatial information of the anomaly detection map and then feeds back the Gaussian filtered AD map to create a new data cube. The whole process is repeated over again in an iterative manner. When IAD is terminated anomaly representatives are identified and can be used as desired target signatures to implement constrain energy minimization (CEM) so as to classify all detected anomalies. Accordingly, IAD can be considered as anomaly classification.

This paper extends target-constrained interference minimized filter (TCIMF) to multiclass-constrained background suppression classifier (MCBSC) for hyperspectral image classification. In order to capture spatial contextual information MCBSC makes use of a Gaussian filter to feed back a Gaussian-filtered MCBSC map to create a new set of hyperspectral images for MCBSC to be re-implemented again in an iterative manner, referred to as iterative MCBSC (IMCBSC). Finally, it uses Otsu's method to perform hyperspectral image classification. As shown by experiments, MCBSC generally performs better than existing spectral-spatial hyperspectral image classification techniques in terms of several quantitative measures, such as classification rate, false classification rate, precision rate, accuracy rate in addition to overall accuracy (OA) rate.

This paper presents a new approach, called band subset selection (BSS)-based hyperspectral anomaly detection (AD), which selects multiple bands simultaneously as a band subset rather than selecting multiple bands one at a time as the tradition band selection (BS) does, referred to as sequential multiple BS (SQMBS). Its idea is to first use virtual dimensionality (VD) to determine the number of multiple bands, nBS needed to be selected as a band subset and then develop two iterative process, sequential BSS (SQ-BSS) algorithm and successive BSS (SC-BSS) algorithm to find an optimal band subset numerically among all possible nBS combinations out of the full band set. In order to terminate the search process the averaged least-squares error (ALSE) and 3-D receiver operating characteristic (3D ROC) curves are used as stopping criteria to evaluate performance relative to AD using the full band set. Experimental results demonstrate that BSS generally performs better background suppression while maintaining target detection capability compared to target detection using full band information.

Hyperspectral image classification faces various levels of difficulty due to the use of different types of hyperspectral image data. Recently, spectral-spatial approaches have been developed by jointly taking care of spectral and spatial information. This paper presents a completely different approach from a subpixel target detection view point. It implements four stage processes, a preprocessing stage, which uses band selection (BS) and nonlinear band expansion, referred to as BS-then-nonlinear expansion (BSNE), a detection stage, which implements constrained energy minimization (CEM) to produce subpixel target maps, and an iterative stage, which develops an iterative CEM (ICEM) by applying Gaussian filters to capture spatial information, and then feeding the Gaussian-filtered CEM-detection maps back to BSNE band images to reprocess CEM in an iterative manner. Finally, in the last stage Otsu's method is applied to converting ICEM-detected real-valued maps to discrete values for classification. The entire process is called BSNE-ICEM. Experimental results demonstrate BSNE-ICEM, which has advantages over support vector machine-based approaches in many aspects, such as easy implementation, fewer parameters to be used, and better false classification and precision rates.

We conducted a survey indicating the wide variety of computer literacy and computer skills among college students. Most teachers suggest that new teaching strategies should be deployed. To resolve these issues, we propose a strategy for improving learning performance in the Introduction to Computers class based on the concepts of crowdsourcing and flipped classroom. The design strategy attempts to resolve common issues in training teachers, designing teaching activities, and sharing knowledge and teaching experiences simultaneously. To accomplish the above goals, we make use of a crowdsourcing system which serves multiple purposes including (1) the innovation of teaching ideas; (2) the enhancement of instructor engagement and training process; and (3) the improvement of sharing knowledge and teaching experiences. To verify the proposed approach, an experiment involving over 10,000 students will be conducted.

Web services and SOA technologies are growing with a fast rate but still facing many problems due to their heterogeneous nature. This paper, based on OWL-S, presents a rich and extensible ontology framework named OWL-QSP for service compositions. In the framework, Service Type is imported to improve service abstract level, and QoS, situation, context are adopted. Since service discovery, service selection and service execution can adapt to the changing situation, QoS and situation-aware service-based systems are more dynamic and flexible so to better satisfy the users' functional and non-functional requirements. The introducing of policy permits managing WSs at a high level and facilitate reuse. It also presents SMICE, a prototype of the service composition system, and describes its main components with service composition process.

It is evident that a STEP-based PnP (plug-and-play) CAD/CAPP integrative tool is very helpful for product data integration. However, the problem is in the difficulty in developing an integrated PnP tool that can be applied to heterogeneous CAD and CAPP systems. This paper presents the framework of a CAD/CAPP integrative tool with PnP characteristics. This implementation is by a series of model mapping from AP203 to AP224 and between AP214 and AP203/AP224.

Collaborative virtual environment model and role scheme are two research branches in computer science. Collaborative activities need an effective mechanism to support participants' roles and rights 'while role schemes can be applied to this field. In this paper, we present a new generic role-based and agent-oriented model for collaborative virtual environment based on discussion of collaborative activity as an essential role in the research of CVE. It introduces role schemes to establish more efficient collaborative virtual environments. The proposed model includes two important parts: collaborative entity and collaborative event. It also advances intelligent entity in running state and collaborative federation in this model to describe cooperation in collaborative virtual environment.