Beyond behavioral interaction records, multimedia recommendation scenarios possess abundant semantic signals, which provide excellent data support for user interest mining. Recently, the multimodal enhanced interaction graph has been actively explored and has achieved great progress. However, these methods overlook the capability disparity of various modalities in learning users' interests and lack the ability to explore the hierarchical relationships of interests in modality, resulting in suboptimal recommendation performance. Therefore, this work investigates intra-modality hierarchical learning and inter-modality guidance, proposing a hyperbolic self-distillation (HSD) model for multimedia recommendation. In each modality space, HSD introduces a hyperbolic propagation to filter users' hierarchical interests from the interaction graph effectively. Inter-modality interests are aligned further by a two-level self-distillation strategy to designate multimodal interactions to teach single-modal learning, aiming at teaching and learning to promote each other. Extensive experiments on four public datasets demonstrate that the proposed HSD outperforms leading baselines for multimedia recommendation, verifying the effectiveness of hierarchical propagation and two-level self-distillation in mining users' hierarchical interests.

In the domain of recommendations, previous works often retrieve items through sampling strategies from the database to gather negative signals for exploring implicit feedback. However, because of extremely sparse records, the existing items used as negative samples may not sufficiently support the interacted items in depicting the diverse interests of users. Consequently, the generation of negative samples needs to be explored in recommendation systems. In this study, we propose an interest-disentangled contrastive sample generation (IDCG) model to enhance interest modeling by contrasting interacted items with the generated samples for recommendation. Specifically, we decouple the interacted items of users into positively relevant and irrelevant factors of interest, providing a valuable clue to learn negatively relevant factors in personalized interests. Then, negative samples are generated by merging the learned negatively relevant factors and irrelevant factors. At this point, a two-level contrast is constructed between positive and negative samples and between the relevant factors of positives and negatives, providing auxiliary collaborative signals to debias and alleviate the interaction sparsity issue. Extensive experiments on three real datasets demonstrate the effectiveness of IDCG in generating targeted and meaningful negative samples from the perspective of disentangling relevant factors to promote interest modeling for recommendation.

In this paper, we advocate CPN-FedSL, a novel and flexible Federated Split Learning (FedSL) framework over Computing Power Network (CPN). We build a dedicated model to capture the basic settings and learning characteristics (e.g., training flow, latency and convergence). Based on this model, we introduce Resource Usage Effectiveness (RUE), a novel performance metric integrating training utility with system cost, and formulate a multivariate scheduling problem that maximizes RUE by comprehensively taking client admission, model partition, server selection, routing and bandwidth allocation into account (i.e., mixed-integer fractional programming). We design Refinery, an efficient approach that first linearizes the fractional objective and non-convex constraints, and then solves the transformed problem via a greedy based rounding algorithm in multiple iterations. Extensive evaluations corroborate that CPN-FedSL is superior to the standard and state-of-the-art learning frameworks (e.g., FedAvg and SplitFed), and besides Refinery is lightweight and significantly outperforms its variants and de facto heuristic methods under a variety of settings.

Contrastive self-supervised learning (CSSL) is a promising method in extracting effective features from unlabeled data. It performs well in image-level tasks, such as image classification and retrieval. However, the existing CSSL methods are not suitable for pixel-level tasks, e.g., change detection (CD), since they ignore the correlation between local patches or pixels. In this paper, we firstly propose a multi-cue contrastive self-supervised learning (MC-CSSL) method to derive dense features for change detection. Besides data augmentation, the MC-CSSL takes advantage of more cues based on the semantic meaning and temporal correlation of local patches. Specially, the positive pair is built from local patches with the similar semantic meaning or temporal ones with the same geographic location. The assumption is that local patches belonging to the same kind of land-covering tend to share similar features. Secondly, the affinity matrix is truncated and introduced to extract change information between two temporal patches obtained from different types of sensors. As a result, some initial unchanged pixels are selected to serve as the supervision for mapping the dense features into a consistent space. Based on the distance between all bi-temporal pixels in the consistent space, a difference image (DI) is generated and more unchanged pixels can be available. The dense feature mapping and unchanged pixel updating proceed alternately. The proposed CD method is evaluated in both homogeneous and heterogeneous cases and the experimental results demonstrate its effectiveness and priority after comparison with some existing state-of-the-art methods. The source code will be available at https://github.com/Yang202308/ChangeDetection_CSSL.

Convolutional neural networks (CNNs) have superior feature learning capabilities with large numbers of labeled samples. The reality is that labeling these samples is costly in terms of human labor. Existing data augmentation methods alleviate the scarcity of labeled samples. However, these methods are not suitable for synthetic aperture radar (SAR) images, owing to special imaging mechanisms and observational objects. The generative SAR images by existing augmented methods show structure distortion. To address this issue, we introduce a curvelet adversarial augmented neural network (CA2NN) for SAR image classification. Specifically, an$\text{A}^{2}$NN is established, which consists of two generative streams and one discriminative stream. In the generative stream, through the mutual transformation between the whole and partial images, more new samples with structural consistency are generated to augment the limited labeled data. In the discriminative stream, these generated samples show certain appearance variations after adversarial training based on the novel joint discriminant criterion. Simultaneously, given the multiscale and multidirectional nature of SAR images, we construct discretized curvelet in 2-D space, aiming to extract the singularity features and avoid overfitting. By integrating curvelet kernels into$\text{A}^{2}$NN, CA2NN can automatically generate more representative features adapting to complex terrain, while greatly reducing the complexity of the network. Experiments are conducted on the SAR images with large-scale and complex scenes, suggesting that the proposed approach significantly improves the classification performance with few labeled samples.

Contrastive self-supervised learning (CSSL) has achieved promising results in extracting visual features from unlabeled data. Most of the current CSSL methods are used to learn global image features with low-resolution that are not suitable or efficient for pixel-level tasks. In this paper, we propose a coarse-to-fine CSSL framework based on a novel contrasting strategy to address this problem. It consists of two stages, one for encoder pre-training to learn global features and the other for decoder pre-training to derive local features. Firstly, the novel contrasting strategy takes advantage of the spatial structure and semantic meaning of different regions and provides more cues to learn than that relying only on data augmentation. Specifically, a positive pair is built from two nearby patches sampled along the direction of the texture if they fall into the same cluster. A negative pair is generated from different clusters. When the novel contrasting strategy is applied to the coarse-to-fine CSSL framework, global and local features are learned successively by forcing the positive pair close to each other and the negative pair apart in an embedding space. Secondly, a discriminant constraint is incorporated into the per-pixel classification model to maximize the inter-class distance. It makes the classification model more competent at distinguishing between different categories that have similar appearance. Finally, the proposed method is validated on four SAR images for land-cover classification with limited labeled data and substantially improves the experimental results. The effectiveness of the proposed method is demonstrated in pixel-level tasks after comparison with the state-of-the-art methods.

Due to the complementary properties of different types of sensors, change detection between heterogeneous images receives increasing attention from researchers. However, change detection cannot be handled by directly comparing two heterogeneous images since they demonstrate different image appearances and statistics. In this article, we propose a deep pyramid feature learning network (DPFL-Net) for change detection, especially between heterogeneous images. DPFL-Net can learn a series of hierarchical features in an unsupervised fashion, containing both spatial details and multiscale contextual information. The learned pyramid features from two input images make unchanged pixels matched exactly and changed ones dissimilar and after transformed into the same space for each scale successively. We further propose fusion blocks to aggregate multiscale difference images (DIs), generating an enhanced DI with strong separability. Based on the enhanced DI, unchanged areas are predicted and used to train DPFL-Net in the next iteration. In this article, pyramid features and unchanged areas are updated alternately, leading to an unsupervised change detection method. In the feature transformation process, local consistency is introduced to constrain the learned pyramid features, modeling the correlations between the neighboring pixels and reducing the false alarms. Experimental results demonstrate that the proposed approach achieves superior or at least comparable results to the existing state-of-the-art change detection methods in both homogeneous and heterogeneous cases.

Supervised change detection methods always face a big challenge that the current scene (target domain) is fully unlabeled. In remote sensing, it is common that we have sufficient labels in another scene (source domain) with a different but related data distribution. In this article, we try to detect changes in the target domain with the help of the prior knowledge learned from multiple source domains. To achieve this goal, we propose a change detection framework based on selective adversarial adaptation. The adaptation between multisource and target domains is fulfilled by two domain discriminators. First, the first domain discriminator regards each scene as an individual domain and is designed for identifying the domain to which each input sample belongs. According to the output of the first domain discriminator, a subset of important samples is selected from multisource domains to train a deep neural network (DNN)-based change detection model. As a result, not only the positive transfer is enhanced but also the negative transfer is alleviated. Second, as for the second domain discriminator, all the selected samples are thought from one domain. Adversarial learning is introduced to align the distributions of the selected source samples and the target ones. Consequently, it further adapts the knowledge of change from the source domain to the target one. At the fine-tuning stage, target samples with reliable labels and the selected source ones are used to jointly fine-tune the change detection model. As the target domain is fully unlabeled, homogeneity- and boundary-based strategies are exploited to make the pseudolabels from a preclassification map reliable. The proposed method is evaluated on three SAR and two optical data sets, and the experimental results have demonstrated its effectiveness and superiority.

Supervised deep neural networks (DNNs) have been extensively used in diverse tasks. Generally, training such DNNs with superior performance requires a large amount of labeled data. However, it is time-consuming and expensive to manually label the data, especially for tasks in remote sensing, e.g., change detection. The situation motivates us to resort to the existing related images with labels, from which the concept of change can be adapted to new images. However, the distributions of the related labeled images (source domain) and unlabeled new images (target domain) are similar but not identical. It impedes a change detection model learned from source domains being well applied to the target domain. In this paper, we propose a transferred deep learning-based change detection framework to solve this problem. It consists of pretraining and fine-tuning stages. In the pretraining process, we propose two tasks to be learned simultaneously, namely, change detection for the source domain with labels and reconstruction of the unlabeled target data. The auxiliary task aims to reconstruct the difference image (DI) for the target domain. DI is an effective feature, such that the auxiliary task is of much relevance to change detection. The lower layers are shared between these two tasks in the training process. It mitigates the distribution discrepancy between the source and target domains and makes the concept of change from the source domain adapt to the target domain. In addition, we evaluate three modes of the U-net architecture to merge the information for a pair of patches. To fine-tune the change detection network (CDN) for the target domain, two strategies are exploited to select the pixels that have a high possibility of being correctly classified by an unsupervised approach. The proposed method demonstrates an excellent capacity for adapting the concept of change from the source domain to the target domain. It outperforms the state-of-the-art change detection methods via experimental results on real remote sensing data sets.

Segmentation of medical images is an important component for diagnosis and treatment of diseases using medical imaging technologies. However, automated accurate medical image segmentation is still a challenge due to the difficulties in finding a robust feature descriptor to describe the object boundaries in medical images. In this paper, a new normal vector feature profile (NVFP) is proposed to describe the local image information of a contour point by concatenating a series of local region descriptors along the normal direction at that point. To avoid trapping by false boundaries caused by nonboundary image features, a modified scale invariant feature transform (SIFT) descriptor is developed. The number and locations of sample points for building NVFP are determined for each contour point, which are constrained by the neighboring anatomical structures and the statistical consistency of the training features. NVFP is incorporated into a model based method for image segmentation. The performance of our proposed method was demonstrated by segmenting prostate MR images. The segmentation results indicated that our method can achieve better performance compared with other existing methods.