Multimodal fusion is an effective solution for holistic brain tumor diagnosis; however, it faces challenges under missing modalities. Traditional multi-encoder architectures can easily capture modality-specific features, while single-encoder architectures readily obtain modality-shared features. The reverse, however, is challenging. In this paper, we propose a two-stage dual-view prototype learning framework to extract the modality-specific feature and the class-specific feature simultaneously. In the first stage, we utilize the Transformer decoder to learn the modality-prototypes that are used to optimize the modality reconstruction task. A masked autoencoder is introduced to generate shared features of incomplete modalities. The learned modality-prototypes that contain modality-specific features are blended with the modality-share features for the reconstruction process. In the second stage, we learn the class-prototypes through the Transformer decoder to generate a segmentation mask through voxel-to-prototype comparison. A masked modality strategy is introduced to handle random modality absence during training. Furthermore, modality-view and class-view contrastive learning strategies are developed to enhance prototype learning. We conduct experiments on BraTS2020 and BraTS2018; the experimental results demonstrate the superior performance of our model under various missing modality scenarios. On BraTS2020, our model achieves DSC improvements of 5.9% for ET, 0.5% for TC, and 0.2% for WT compared to state-of-the-art methods. Notably, in the challenging T1C modality missing scenario, our model achieved clinically significant gains of 9.5% for ET and 1.8% for TC. The code is available at https://github.com/Xiheran/MCPL.

In the Industrial Internet of Things (IIoT), collaborative intelligence among distributed devices is essential for achieving autonomy and robustness, especially when facing non-IID and out-of-distribution (OOD) data. Deep neural networks have achieved significant success in various applications, but their prediction confidence often degrades on OOD data, which is critical in IIoT environments with heterogeneous sources. Centralized OOD detection methods assume data is centrally stored and require a large number of real OOD samples, which are impractical and costly in federated learning due to data silos and privacy issues. To address the above challenges, we formulate the new problem of OOD detection on heterogeneous data in a federated learning framework. We propose a novel multi-task optimal model named FOOD that improves OOD accuracy through optimal transport theory in a distributed manner with data privacy protection. Specifically, FOOD generates pseudo-OOD samples based on optimal transport theory and purifies training samples to enhance classification accuracy. We use the Wasserstein distance to measure the similarity between in-distribution and out-of-distribution samples and generate heterogeneous pseudo-OOD samples among different clients. FOOD is a plug-and-play plugin that can improve deep neural models' performance without introducing extra overhead. Experiments on OOD datasets show that FOOD significantly enhances OOD detection and classification on several public OOD datasets, AUROC improved by 1.69%, AUPR by 1.75%, and ACC by 4.67%. Using optimal transport theory, our work provides a practical approach to improving data generalization in generative models.

Federated domain generalization (FDG) enables collaborative learning across distributed devices to build a global prediction model capable of generalizing to diverse environments. While existing methods perform well on homogeneous data distributions, they struggle with performance degradation caused by data drift in heterogeneous settings. To address this challenge, we propose FedFAT, a novel method that leverages frequency domain adaptive interpolation to mitigate data drift effectively. FedFAT allows each client to adaptively exchange amplitude information for improved generalization while retaining phase information locally to ensure privacy. The interpolation process-including amplitude interpolation size, mask position, and fusion ratio-is dynamically determined based on the difference between the client's amplitude and that of the shared library. Additionally, we introduce amplitude normalization to align features across clients by batch-normalizing images from multi-source distributions, thereby reducing the negative impact of data drift. Building on these uniform features, weight perturbation is applied to ensure consistent low loss for local models. Extensive experiments and ablation studies on medical image analysis tasks, including MRI prostate segmentation and breast cancer tissue classification, demonstrate the superiority of FedFAT in handling data drift and improving generalization performance. Our results underscore the critical role of frequency domain adaptive interpolation in addressing data drift and enhancing the robustness of federated domain generalization.

Semi-supervised learning (SSL) has been prevailed in medical image analysis field because it leverage unlabeled data for training powerful models without incurring extra high annotation cost. However, the existing SSL models face the challenge of performance degradation in open-set scenario where both in-distribution (ID) and out-of-distribution (OOD) samples are mixed in unlabeled data. The existing two-phases methods treat OOD detection and semi-supervised classification as two independent tasks which fail to reveal their mutual reinforcement ability. In this research, we introduce a joint optimization framework designed to enhance both the semi-supervised classification task and the OOD detection task through an iterative process. Specifically, our approach employs a unified model to assess the likelihood of images being OOD sample, subsequently filtering these instances from the pool of unlabeled data. The model parameters update and the OOD detection are optimized alternately. Additionally, to avoid the model overconfidence, we introduce logit normalization (Logit-Norm) loss to provide more reliable predictions. To validate the effectiveness of our method, we use the ISIC2018 dataset as the ID dataset and mix OOD samples from other medical image datasets to train classification model. Experimental results demonstrate that our method successfully mitigates the influence of OOD data on semi-supervised medical image classification performance while also improving OOD detection performance. The proposed framework successfully addresses the challenges posed by OOD samples in semi-supervised learning, offering a promising solution for medical image classification tasks that involve OOD data.

Transformers have shown great potential in vision tasks such as semantic segmentation. However, most of the existing transformer-based segmentation models neglect the cross-attention between pixel features and class features which impedes the application of transformers. Inspired by the concept of object queries in k-means Mask Transformer, we develop cluster learning and contrastive cluster assignment (CCA) for medical image segmentation in this paper. The cluster learning leverages the object queries to fit the feature-level cluster centers. The contrastive cluster assignment is introduced to guide the pixel class prediction using the cluster centers. Our method is a plug-in and can be integrated into any model. We design two networks for supervised segmentation tasks and semi-supervised segmentation tasks respectively. We equip the decoder with our proposed modules for the supervised segmentation to improve the pixel-level predictions. For the semi-supervised segmentation, we enhance the feature extraction capability of the encoder by using our proposed modules. We conduct comprehensive comparison and ablation experiments on public medical image datasets (ACDC, LA, Synapse, and ISIC2018), the results demonstrate that our proposed models outperform state-of-the-art models consistently, validating the effectiveness of our proposed method. The source code is accessible at https://github.com/zhujinghua1234/CCA-Seg.

The emergence of lightweight models provides a new solution for the classification of Diabetic Retinopathy (DR). These lightweight models have a fewer parameters and low computational complexity. However, existing lightweight models in Diabetic Retinopathy (DR) still face challenges: Convolutional Neural Networks (CNNs) are limited by their receptive fields and lack modeling capabilities over long distances. Although the models based on Transformer can capture global information, the usually have high computational complexity. Recent lightweight models that try to combine the strengths of CNN and Transformer tend to have a simple serial architecture that fails to take full advantage of both. To address these challenges, we propose a new lightweight network, MobileP2VT, for the classification of diabetic retinopathy. The model uses lightweight convolution and Transformer parallel processing to capture local details and global information of the image. In addition, we have designed a new attention mechanism, Sparse Token Attention (SPTA), which reduces the amount of attention computation in Transformer. Experimental results on Eyepacs and APTOS 2019 datasets demonstrate the high efficiency of MobileP2VT.

The general goal of the recommender system is to provide users with precise and personalized recommendations, rather than popular items. However, popularity bias widely exists in existing recommendation models, especially in Session-Based Recommendation(SBR) since the short-term users’ behaviors are much more susceptible to data skew and popularity. Unfortunately, existing SBR methods fail to explicitly take the popularity bias into consideration. Moreover, existing methods of eliminating popularity bias for sequence recommendation are not suitable for SBR scenarios where anonymous users’ information is hard to obtain. Thus, effective methods to eliminate popularity bias especially designed for SBR are needed. In this paper, we leverage causal inference theory to analyze the impact of item popularity by establishing a causal graph and then eliminating popularity bias through counterfactual reasoning. Specifically, we first construct a causal graph to describe the important causal relationship in the recommendation process and find that popularity bias is caused by the direct influence path from the item to the ranking score. Then we model the impact of causal relationships on the recommendation score through multi-task training. Finally, we eliminate the impact of item popularity by removing the direct impact of items according to counterfactual reasoning. We integrate the proposed popularity-eliminating method into the recently outstanding SBR model COTREC and conduct extensive experiments on two real datasets to validate that the casual inference to deal with popularity bias could further improve the recommendation accuracy and our model outperforms the SOTA SBR models consistently.

Federated Learning is a distributed learning paradigm that allows multiple organizations or devices to train a global model collaboratively in a privacy-preserving manner. However, there still exists privacy leakage risks due to the curious or dishonest server. In this paper, we propose a novel server-free federated learning paradigm named MChain-SFFL, which utilizes parallel multi-chain aggregation to mitigate privacy leakage risks and enhance convergence speed. First, MChain-SFFL randomly selects multiple users as chain heads. Then, MChain-SFFL utilizes parallel multi-chain communication mechanism to transmit the masked local model parameters. Finally, every chain head computes the model update for that chain and sends it to the other users. Upon receiving updates from all other chains, each user aggregates the received parameters to generate the model update for the current round. We validate the superiority of our method in accuracy and convergence speed on both image datasets and text datasets. Experimental results show that MChain-SFFL achieves superior privacy protection without impairing model accuracy and exhibits robustness to Non-IID data.

With the staggering growth of data on modern IoT devices, privacy concerns on IoT networks are becoming increasingly prominent. Thus, federated learning emerged. It trains the model by enabling multiple participants to employ data from multiple parties, making data accessible but not visible. However, user data may encounter issues with data privacy leakage in the existing federated learning environment. Thus, this article suggests BFLPP, a privacy protection federated learning framework based on blockchain and committee consensus in IoT devices. It leverages blockchain to verify local updates, rather than a centralized server and uses it to generate and store global models. It also implements local differential privacy to further secure data privacy. In this paper, we use committee nodes to validate the model parameters, and if we receive enough replies during the validation process. It will submit and upload the validated updates to the update block, then trigger the smart contract to aggregate updates and broadcast it to the nodes for the next training round. Eventually, this paper employs the BFLPP framework on convolutional neural networks to conduct experiments on the MNIST dataset, grounded on blockchain and federated learning. The experimental results indicate the security and effectiveness of the BFLPP framework.

Sequential recommendation models user behavior dynamically based on historical interactions. However, data sparsity affects the performance of sequential recommendation consistently. Existing research mainly uses contrastive learning and auxiliary information to alleviate data sparsity. But existing contrastive learning methods only perform contrastive learning on interaction sequences. Auxiliary information enriches item representations, especially auxiliary intentions are widely adopted in the sequential recommendation. But the intent information is also sparse. And it is necessary to perform contrastive learning of the intention sequences corresponding to the interaction sequences. Previous research only uses contrastive learning and auxiliary intentions independently, which can result in general recommendation efficiency. How to combine contrastive learning and auxiliary intentions organically becomes more and more challenging. Therefore, we propose sparse sequential recommendation with interactions and intentions contrastive Learning, namely I2CL, which solves the data sparsity problem by employing both contrastive learning and auxiliary intentions in the sequential recommendation. First, the intent representation is learned by selecting the intent information from a large amount of auxiliary information to compensate for the deficiency that the current sequential recommendation only focuses on recent items due to data sparsity. In this paper, the auxiliary intentions use the seller and classification information of items for user interaction, which can effectively capture users’ long-term preferences. Second, for reasons of learning high-quality representations, we utilize a contrastive learning framework to extract self-supervised signals from raw user behavior sequences and intent sequences and optimize user representation models so as to improve sequential recommender systems. Finally, a multitask training strategy is adopted to optimize recommendations jointly through parameter and structure sharing. Experiments are conducted on two sparse recommendation datasets to demonstrate the effectiveness of the proposed model in this paper.

With the rapid development of social production and the economy, environmental problems have increasingly become prominent. Electric vehicles are very popular due to their characteristics of zero pollution emmissions, which has led to the increasing scale of electric vehicles. As the number of electric vehicles increases, the problem of traffic congestion has become more and more serious, and the difficulty of charging has become a problem for people. How to solve the timeliness and uncertainty of electric vehicle charging and reduce the electric vehicle’s charging costs are two challenges in the new energy field. In this paper, we focus on the real-time electric vehicle charging problem with the consideration of road conditions and weather influence. By constructing state, action, system reward, and state transition functions, the problem of electric vehicle charging scheduling is formulated as a Markov Decision Process. We propose a Soft Actor-Critic algorithm based on deep reinforcement learning to dynamically learn the optimal charging strategy with the aim of minimizing charging time and battery power consumption for users, to improve the charging experience. In addition, we design a deep learning model for real-time electricity price prediction to assist intelligent charging decisions and further save charging costs for users. Numerical experimental results verify the effectiveness and superiority of our proposed method.

With the increasing scale of data to be processed by the recommendation system and the rapid growth of the number of users and commodities, the problems of data sparsity and cold start become more and more severe. Many studies have found that introducing implicit social relationships into recommendation can alleviate the above problems. Among these studies, the method of identifying implicit friends for each user in heterogeneous information networks based on meta-paths achieves good performance. However, meta paths need to be preset artificially and require strong professional domain knowledge, which makes it impossible to capture semantic information accurately and effectively. In this paper, we propose a novel social recommendation model termed as SILK-BPR which is an extension of the Self Guided Walk(SILK) model to by bypass the meta-paths and identify implicit friends for recommendation. Specially, SILK-BPR first establishes a guidance matrix to record the transition probability from user nodes to interest nodes. And then conducts a self-guided walking on the matrix to mine the implicit friends which are classified into different types according to the appear times in different sequences. Finally, these implicit friends are integrated into an enhanced social Bayesian Personalized Ranking model for Top-N recommendation. Experimental results on real-world datasets demonstrate that SILK-BPR significantly outperforms state-of-the-art methods.

Building an effective and friendly human-machine dialogue system is one of the major challenges in Artificial Intelligence. This work proposes a new model named Graph and Attention Matching Network (AGMN) for response selection in retrieval-based dialogue system. AGMN model consists of two parts: cross attention mechanism and knowledge representation extractor. Specifically, the cross attention mechanism is exploited to obtain the dual representation from context and response words because these representations can provide the useful matching information for determining whether the next utterance is suitable response or not. Besides, the domain knowledge relationships which are extracted from Linux manuals are incorporated into the word representation by graph attention mechanism. Experimental results on Ubuntu Dialogue Corpus showed that both cross attention mechanism and domain knowledge can contribute to the performance of response selection and the AGMN model proposed in this paper outperforms the state-of-art approaches.

To solve the data sparseness and cold start problems in collaborative filtering (CF) based recommender systems (RS), various complex algorithms are proposed to extract and integrate explicit or implicit information of data for the recommendation. In this paper, we propose to aggregate and transmit the rich semantic information with the help of knowledge graph (KG) that is regarded as one of the main sources of auxiliary information. Specifically, we first propose a Neural Graph Collaborative Filtering to construct and aggregate information. And then we build a scalable and end-to-end knowledge-aware graph collaborative filtering model named KGCF. In KGCF, neighbourhood information in KG is encoded to construct information in a complex new way. And the information from neighbours are merged with a personalized bias calculated by attention mechanism based on KG. In order to extend the interacted items and capture the high-level semantic information of KG, multiple KGCF layers stacked is used in KGCF. Experimental results on three real data sets indicate that the KGCF model proposed in this paper is superior to the existing models in terms of accuracy and can also effectively solve the data sparsity problem of RS.

Due to the fast development of intellectual devices and wireless technology, the mobile crowdsensing (MCS) technology has gradually turn into an effective means of sensing and collecting information about the surrounding environment in real-time. Task allocation is one of the essential questions in MCS. Previous task assignment methods only directly assign the task to the workers nearby the tasks, without paying attention to the location change of the workers and the tasks, which increase workers' traveling cost and platform cost. Instead, our goal in this paper is to consider predicted workers and tasks to improve overall utility. The dynamic task allocation proposed by us is a novel optimization problem. To tackle this problem, the paper leverages the semi-Markov model to predict the position distribution of workers and tasks. Moreover, the connection probability between worker and task is calculated under time constraint. Eventually, we design a prediction-based task allocation algorithm (PBTA), which can derive the maximum overall system utility and lowest traveling cost. Through experiments, we evaluate our approach extensively using two large-scale real-world datasets. The experimental results validate the effectiveness and efficiency of our proposed scheme.

Due to the fast development of the Internet technology, a variety of social applications have emerged. The users of these social applications are expanding, forming a large-scale social network. In complex and huge social networks, there are a lot of valuable information waiting to be utilized, so that service providers can provide better services for users. On the other hand, to solve the challenge of information overload brought with the communication technology, the recommendation system is becoming more and more important in many fields. In the context of the social network, a recommendation system can better discover the relationship between users and preferences, thus giving a more accurate recommendation. However, most of the traditional recommendation systems mainly focus on providing personalized recommendation service to a single user which are not suitable for group recommendation such as recommendation for a team or a family. In this paper, we proposed a new method based on trust relationship in social network and attention mechanism of deep learning to solve the group recommendation problem. Specifically, we use attention mechanism to capture different group members's preference weight and we consider trust relation among group members when learning group preference in order to capture more information. Through this method, the accuracy of the recommended results can be further improved. The experimental results on Film Trust dataset show that the proposed algorithm in this paper is able to improve the quality of recommendation compared with the existing group recommendation methods.

Point-Of-Interest (POI) recommendation plays an important role in Location-Based Social Networks(LBSN), which is widely used in popular attraction recommendations and travel route planning applications. The traditional recommendation algorithms fail to make full use of social relationships, user check-in distribution features and geographic information because they only use a simple linear function to model the above features. In order to solve the existing problems, we propose a recommendation framework-NCFT(Neural Collaborative Filtering Tree), which can fuse various side information. In the NCFT model, we propose an unsupervised user check-in distribution feature extractor, namely CD-Ex, which unsupervised learning user checkin distribution features. We also propose to build a user-based collaborative filtering tree and item-based collaborative filtering tree, and use the idea of messaging to learn deep representations of users and items. In these two modules, we use multi-head attention and vanilla attention to learn the representations of users and POIs. As for the user social relationship, we use the user's friends in the user-based collaborative tree to assign weights and aggregate his friends' features. The experimental results show that our model has a significant improvement in AUC and F1 compared to other models.

Influence maximization refers to the number of influenced nodes reach to maximum by finding a set of seed nodes in social network. Due to peoples' excessive concentration of attention, sometimes, negative effects will appear during the influence propagation. However, the negative effects on influence propagation are not involved in the existing research work. In this paper, we consider the influence equilibrium problem in mobile social network. The goal of influence equilibrium is to balance the influential range of different kinds of seed sets through the way of node scheduling. Our goal is to reduce the negative effects by transforming the status of influence from excessive concentration to relatively balanced. There are two main challenges: (1) which nodes should be scheduled in mobile social network in order to achieve a balanced effect. (2) how to schedule the nodes. To address the two challenges, we estimate the randomness of nodes and then schedule the nodes which have high randomness. The entropy-based random walk select balance algorithm(RS_B) is proposed in this paper. To achieve the balanced effect, random walk procedure is adopted to choose the sets that can receive the scheduled nodes during the scheduling phase. However, new imbalance may arise during this procedure. Thus, the entropy-based max-min balance algorithm(MM_B) is proposed to deal with the new imbalance caused by RS_B algorithm. Theoretical analysis and experimental results show that the RS_B and MM_B algorithm can balance the influence range and MM_B performs better than RS_B.

Considering limited storage space, limited energy and other constraints of sensors, a query mechanism based on data caching in Multi-Sink sensor networks is proposed. This mechanism extractes the common subtree of a network firstly, and then caches a sink's query results at the common root of a common subtree according to certain cashing strategies. When some other sinks submit the same query, it only needs to send the cached query results to the query sink. This data sharing mechanism based on common subtree caching can reduce communication cost obviously and improve query efficiency. In order to achieve a higher level of data sharing, we propose a Loop Removing Algorithm (LR) which can enlarge the size of a common subtree effectively. Simulation results indicate that the LR and the data caching based query processing method proposed in this paper can reduce the average energy consumption and response time of queries significantly.