In semi-supervised medical image segmentation, the use of CutMix in the Mean Teacher architecture is considered an effective strong data augmentation strategy. However, we believe that randomly selecting patches from the source image might mislead the model into learning unexpected feature representations. Therefore, we propose Gradient Saliency-aware CutMix for semi-supervised medical image segmentation (GSC-Seg). Utilizing the gradient from pre-trained models to detect salient regions and then copies and pastes the large gradient areas from labeled data into corresponding areas of unlabeled data based on the gradient, and vice versa, guiding the model to learn more appropriate feature representations. Furthermore, we propose a gradient augmentation strategy, which generates disruptions in the gradient through the network itself and enhances the gradient representation abilities of the network. Experiment results show that our approach achieves the state-of-the-art performance on three medical image segmentation datasets. Code is available at https://github.com/UESTC-Med424-JYX/GSC-Seg.

Medical images often contain sensitive information, and one typical security measure is to encrypt medical images prior to storage and analysis. A number of solutions, such as those utilizing deep learning, have been proposed for medical image encryption and decryption. However, our research shows that deep learning-based encryption models can potentially be vulnerable to backdoor attacks. In this paper, a backdoor attack paradigm for encryption and decryption network is proposed and corresponding attacks are respectively designed for encryption and decryption scenarios. For attacking the encryption model, a backdoor discriminator is adopted, which is randomly trained with the normal discriminator to confuse the encryption process. In the decryption scenario, a number of subnetwork parameters are replaced and the subnetwork can be activated when detecting the trigger embedded into the input (encrypted image) to degrade the decryption performance. Considering the model performance degradation due to parameter replacement, the model pruning is also adopted to further strengthen the attacking performance. Furthermore, the image steganography is adopted to generate invisible triggers for each image; subsequently, improving the stealthiness of backdoor attacks. Our research on designing backdoor attacks for encryption and decryption network can serve as an attacking mode for such networks, and provides another research direction for improving the security of such models. This research is also one of the earliest works to realize the backdoor attack on the deep learning based medical encryption and decryption network to evaluate the security performance of these networks. Extensive experimental results show that the proposed method can effectively threaten the security performance both for the encryption and decryption network.

Most existing semantic segmentation methods primarily employ supervised learning with discriminative models. Although these methods are straightforward, they overlook the modeling of underlying data distributions. In this paper, we propose a novel medical image segmentation framework called Diff-SFCT based on Diffusion Model. We formulate semantic segmentation as a generative problem for segmentation masks, replacing the conventional pixel-wise discriminative learning with a latent prior learning process to produce more accurate segmentation results. Diff-SFCT employs a backbone network combining Convolutional Neural Network (CNN) and Transformer, and utilizes the local perception of CNN and the global information modeling capability of Transformer. In Diff-SFCT, we design a Semantic Encoder that effectively extracts fine-grained semantic features from real images. Meanwhile, we propose a novel Spatial-Frequency Cross Transformer (SFCT) framework, which can effectively model and interact the global features of the diffuse noise mask and the real semantic features, reducing the domain gap between the two and enhancing the model’s representational capacity. Additionally, to preserve spatial and frequency information in the diffusion model, we design a Spatial-Frequency Attention Module (SFAM) as part of the Convolutional Block. This module improves the model’s spatial and frequency perception abilities while incurring negligible computational overhead. Experimental results evince that our DiffSFCT substantially outperforms other segmentation methods, exhibiting remarkable performance across various medical image segmentation datasets.

Few-shot learning (FSL) is a method that does not require a large number of labeled samples and has the ability to identify unseen samples. Recently, many excellent FSL methods have been developed for remote sensing scene classification (RSSC). However, for complex remote sensing scenes, the following problems have not been well addressed: 1) the order of local features and the spatial correlation between features have not been taken into consideration and 2) the current metric methods are mainly based on Euclidean space (ES), and discriminative ability is limited. Hence, we propose a framework for RSSC with FSL by combining Hilbert feature sequence and Lie group metric space (HFS-LGMS). Specifically, we adopt a lightweight four-layer convolutional neural network (CNN) as a feature extractor. For the last feature map, we regard each pixel as a multidimensional local feature. Then we organize all the features into a feature sequence along the Hilbert curve. To represent the correlation among features, we introduce the autoregressive moving average model (AMAM) to model the observed features and their spatial states. Since the parameters of the model are matrices, which are directly processed as vectors will affect performance. Therefore, we transform the matrix parameter space into the Lie group manifold space (LGMS) and introduce the Lie group intrinsic mean (LGIM) to represent the prototype. The distance between the Lie groups is measured by the geodesic distance. The similarity is measured on Lie group space. Finally, the effectiveness of HFS-LGMS is demonstrated on three public datasets.

Deep neural networks driven by large amounts of annotated samples have been successfully applied in remote sensing scene classification (RSSC). On most common public datasets, the accuracies of scene recognition tasks have been close to saturation. A learning paradigm that specifically tackles this issue has emerged, i.e., few-shot learning (FSL). FSL methods for natural image recognition are developing rapidly. Recently, these methods have been widely used in RSSC. Compared with natural images, the problems of intra-class differences and inter-class similarities in RSSC are more serious, which hinders the further development of FSL methods for RSSC. To address this issue, we propose a multi-manifold metric learning framework for RSSC with FSL. Specifically, we use a lightweight convolutional neural network as a feature extraction block. To further enhance the representation ability of features, we embed the feature map into two heterogeneous and complementary Riemannian manifold geometric structures, i.e. Grassmannian manifold and symmetric positive definite (SPD) manifold. Then, to facilitate the measurement, we introduce the Riemann kernel function to embed two heterogeneous manifold structures into the high-dimensional Hilbert space for fusion. Finally, we design a learnable distance metric scheme that can be optimized according to the divergence of inter-class pairs and intra-class pairs, thereby reducing the impact of large intra-class differences and high inter-class similarity on scene recognition. We verify the effectiveness of MMML on three commonly used datasets. The results show that the accuracy of our proposed method is from 1.21% to 3.12% higher than the state-of-the-art methods.

Scene classification is of great significance to understand the semantics and extract the information of high spatial resolution remote sensing image. The bag-of-visual-words( BOVW) model is an effective method to understand the semantic content of images. It is widely used in scene classification of remote sensing image.The existed BOVW models usually use single feature or multiple features (such as spectrum, texture, and shape) to describe visual words. However, when considering multiple low-level feature fusion strategies, most methods only combine them by simple accumulation or concatenation which can not fully learn the relationship between different features.In this letter, a visual bag-of-words scene classifier based on regional covariance (RCOVBOVW) is proposed. This method can naturally fuse multiple related features, and the covariance calculation itself has the filtering ability, which can also reduce the dimension of the features and have high efficiency. Experiments have been conducted on two public and challenging datasets (UC Merced and NWPU-RESISC45), and the results show that our proposed method outperforms the most state-of-the-art methods of remote sensing image scene classification.

The existing convolutional neural network (CNN) models have shown excellent performance in remote sensing scene classification. However, the structure of such models is becoming more and more complex, and the learning of low-level features is difficult to interpret. To address this problem, in this study, we introduce lie group machine learning into the CNN model, try to combine both approaches to extract more distinguishing ability and effective features, and propose a novel network model, namely, the lie group regional influence network (LGRIN). First, manifold space samples of the lie group are obtained by mapping, and then, the features of the lie group are extracted after the operations of image decomposition and integral image calculation. Second, the multidilation pooling is integrated into the CNN architecture. At the same time, the image regional influence network module is designed to guide the attention of the classification model by using the regional-level supervision of the decomposition. Finally, the fusion features are classified, and the predicted results are obtained. Our model takes full advantage of regional influence, lie group kernel function, and lie group feature learning. Moreover, our model produces satisfactory performance on three public and challenging data sets: Aerial Image Dataset (AID), UC Merced, and NWPU-RESISC45. The experimental results verify that, compared with the state-of-the-art methods, this method is more explanatory and achieves higher accuracy.

Remote sensing scene classification is used to label specific semantic categories for images. The current methods have achieved competitive performances, but they are only for Euclidean space samples. As a result, their representations are not robust for non-Euclidean space samples, which affects the classification accuracy. In this letter, we introduce the Lie group manifold into the traditional feature representation method and propose a novel intrinsic mean representation method within the Lie group. At the same time, the kernel function based on the sample of the Lie group is designed to further improve the robustness and accuracy of classification. In addition, our method achieves satisfactory performance on two public and challenging remote sensing data sets of UC Merced and NWPU-RESISC45.

The key problem of remote sensing image classification is to understand the semantic content of the images effectively. Up to now, most methods are to improve the accuracy of the final classification through the convolutional neural networks model. However, the structure of the whole model is complex and contains a large number of parameters. To address this problem, in this study, we proposed a lightweight method of the intrinsic mean within Lie groups, which can achieve high accuracy. We apply the computational advantage of Lie group manifold space and design the kernel function that is suitable for both Lie group sample and vector sample. Experiments are performed on two publicly available and challenging data sets [aerial image dataset (AID) and Northwestern Polytechnical University-REmote Sensing Image Scene Classification, contains 45 scene classes (NWPU-RESISC45)], and the results show that our method is more accurate than the state-of-the-art methods.