Multiagent reinforcement learning (MARL) has shown strong performance in cooperative tasks. However, most existing approaches are designed for single-task scenarios and struggle to adapt to complex and dynamic environments. Multitask MARL methods aim to improve adaptability by sharing policies across tasks, but they often suffer from negative transfer due to conflicting task-specific knowledge. To address this, we propose dual-policy fusion for multitask MARL (DPF-MTMARL), which explicitly integrates a shared policy for leveraging common knowledge and task-specific policies for capturing task-specific information. Specifically, in DPF-MTMARL, we propose a learning method to efficiently train the task-specific policies and provide corresponding theoretical analysis. Additionally, we derive the theoretical conditions for decentralizing the joint policy and enforce these conditions through a regularization term during training. Extensive experiments demonstrate that DPF-MTMARL significantly outperforms state-of-the-art baselines in both homogeneous and heterogeneous task sets, effectively mitigating negative transfer and enabling robust multitask learning.

Seeking high-quality representations with latent variable models (LVMs) to reveal the intrinsic correlation between neural activity and behavior or sensory stimuli has attracted much interest. In the study of the biological visual system, naturalistic visual stimuli are inherently high-dimensional and time-dependent, leading to intricate dynamics within visual neural activity. However, most work on LVMs has not explicitly considered neural temporal relationships. To cope with such conditions, we propose Time-Evolving Visual Dynamical System (TE-ViDS), a sequential LVM that decomposes neural activity into low-dimensional latent representations that evolve over time. To better align the model with the characteristics of visual neural activity, we split latent representations into two parts and apply contrastive learning to shape them. Extensive experiments on synthetic datasets and real neural datasets from the mouse visual cortex demonstrate that TE-ViDS achieves the best decoding performance on naturalistic scenes/movies, extracts interpretable latent trajectories that uncover clear underlying neural dynamics, and provides new insights into differences in visual information processing between subjects and between cortical regions. In summary, TE-ViDS is markedly competent in extracting stimulus-relevant embeddings from visual neural activity and contributes to the understanding of visual processing mechanisms. Our codes are available at https://github.com/Grasshlw/Time-Evolving-Visual-Dynamical-System.

Sequential recommendation has been receiving increasing attention from researchers. Existing sequential recommendation models leverage deep learning models to capture sequential features. However, these methods ignore confounders in the recommendation process, which can lead the model to learn incorrect correlations and fail to accurately capture users' true preferences. Moreover, these methods rely on extensive interaction sequences, but sequential data often suffers from sparsity issues. To address these limitations, this paper proposes a P reference- a ware C ausal I ntervention and Counter f a c tual Data Augmentation ( Pacific ) framework to enhance sequential recommendation. Initially, we model the causal graph of sequential recommendation and categorize user preferences into global long-term preferences, local long-term preferences, and short-term preferences. Then, we introduce the front-door criterion to eliminate the interference of confounders and design different self-attention mechanisms to estimate the causal effects, aiming to capture users' true preferences. In addition, based on counterfactual thinking, we design a counterfactual data augmentation module to generate enriched sequences. Experimental results on four real-world datasets demonstrate the superiority of our proposed approach over state-of-the-art sequential recommendation methods.

Deep neural networks (DNNs) are widely used models for investigating biological visual representations. However, existing DNNs are mostly designed to analyze neural responses to static images, relying on feedforward structures and lacking physiological neuronal mechanisms. There is limited insight into how the visual cortex represents natural movie stimuli that contain context-rich information. To address these problems, this work proposes the long-range feedback spiking network (LoRaFB-SNet), which mimics top-down connections between cortical regions and incorporates spike information processing mechanisms inherent to biological neurons. Taking into account the temporal dependence of representations under movie stimuli, we present Time-Series Representational Similarity Analysis (TSRSA) to measure the similarity between model representations and visual cortical representations of mice. LoRaFB-SNet exhibits the highest level of representational similarity, outperforming other well-known and leading alternatives across various experimental paradigms, especially when representing long movie stimuli. We further conduct experiments to quantify how temporal structures (dynamic information) and static textures (static information) of the movie stimuli influence representational similarity, suggesting that our model benefits from long-range feedback to encode context-dependent representations just like the brain. Altogether, LoRaFB-SNet is highly competent in capturing both dynamic and static representations of the mouse visual cortex and contributes to the understanding of movie processing mechanisms of the visual system. Our codes are available at https://github.com/Grasshlw/SNN-Neural-Similarity-Movie.

Spiking Neural Networks (SNNs) provide a sparse spike-driven mechanism which is believed to be critical for energy-efficient deep learning. Mixture-of-Experts (MoE), on the other side, aligns with the brain mechanism of distributed and sparse processing, resulting in an efficient way of enhancing model capacity and conditional computation. In this work, we consider how to incorporate SNNs’ spike-driven and MoE’s conditional computation into a unified framework. However, MoE uses softmax to get the dense conditional weights for each expert and TopK to hard-sparsify the network, which does not fit the properties of SNNs. To address this issue, we reformulate MoE in SNNs and introduce the Spiking Experts Mixture Mechanism (SEMM) from the perspective of sparse spiking activation. Both the experts and the router output spiking sequences, and their element-wise operation makes SEMM computation spike-driven and dynamic sparse-conditional. By developing SEMM into Spiking Transformer, the Experts Mixture Spiking Attention (EMSA) and the Experts Mixture Spiking Perceptron (EMSP) are proposed, which performs routing allocation for head-wise and channel-wise spiking experts, respectively. Experiments show that SEMM realizes sparse conditional computation and obtains a stable improvement on neuromorphic and static datasets with approximate computational overhead based on the Spiking Transformer baselines.

Spiking Transformers, which integrate Spiking Neural Networks (SNNs) with Transformer architectures, have attracted significant attention due to their potential for low energy consumption and high performance. However, there remains a substantial gap in performance between SNNs and Artificial Neural Networks (ANNs). To narrow this gap, we have developed QKFormer, a direct training spiking transformer with the following features: i) _Linear complexity and high energy efficiency_, the novel spike-form Q-K attention module efficiently models the token or channel attention through binary vectors and enables the construction of larger models. ii) _Multi-scale spiking representation_, achieved by a hierarchical structure with the different numbers of tokens across blocks. iii) _Spiking Patch Embedding with Deformed Shortcut (SPEDS)_, enhances spiking information transmission and integration, thus improving overall performance. It is shown that QKFormer achieves significantly superior performance over existing state-of-the-art SNN models on various mainstream datasets. Notably, with comparable size to Spikformer (66.34 M, 74.81\%), QKFormer (64.96 M) achieves a groundbreaking top-1 accuracy of **85.65\%** on ImageNet-1k, substantially outperforming Spikformer by **10.84\%**. To our best knowledge, this is the first time that directly training SNNs have exceeded 85\% accuracy on ImageNet-1K.

Among various value decomposition-based multiagent reinforcement learning (MARL) algorithms, the overall performance of the multiagent system is represented by a scalar global Q value and optimized by minimizing the temporal difference (TD) error with respect to that global Q value. However, the global Q value cannot accurately model the distributed dynamics of the multiagent system, since it is only a simplified representation for different individual Q values of agents. To explicitly consider the correlations between different cooperative agents, in this article, we propose a distributional framework and construct a practical model called distributional multiagent cooperation (DMAC) from a novel distributional perspective. Specifically, in DMAC, we view the individual Q value for the executed action of a random agent as a value distribution, whose expectation can further represent the overall performance. Then, we employ distributional RL to minimize the difference between the estimated distribution and its target for the optimization. The advantage of DMAC is that the distributed dynamics of agents can be explicitly modeled, and this results in better performance. To verify the effectiveness of DMAC, we conduct extensive experiments under nine different scenarios of the StarCraft Multiagent Challenge (SMAC). Experimental results show that the DMAC can significantly outperform the baselines with respect to the average median test win rate.

In recent years, session-based recommendation (SBR), which seeks to predict the target user’s next click based on anonymous interaction sequences, has drawn increasing interest for its practicality. The key to completing the SBR task is modeling user intent accurately. Due to the popularity of graph neural networks (GNNs), most state-of-the-art (SOTA) SBR approaches attempt to model user intent from the transitions among items in a session with GNNs. Despite their accomplishments, there are still two limitations. First, most existing SBR approaches utilize limited information from short user–item interaction sequences and suffer from the data sparsity problem of session data. Second, most GNN-based SBR approaches describe pairwise relations between items while neglecting complex and high-order data relations. Although some recent studies based on hypergraph neural networks have been proposed to model complex and high-order relations, they usually output unsatisfactory results due to insufficient relation modeling and information loss. To this end, we propose a category-aware lossless heterogeneous hypergraph neural network (CLHHN) in this article to recommend possible items to the target users by leveraging the category of items. More specifically, we convert each category-aware session sequence with repeated user clicks into a lossless heterogeneous hypergraph consisting of item and category nodes as well as three types of hyperedges, each of which can capture specific relations to reflect various user intents. Then, we design an attention-based lossless hypergraph convolutional network to generate sessionwise and multi-granularity intent-aware item representations. Experiments on three real-world datasets indicate that CLHHN can outperform the SOTA models in making a better tradeoff between prediction performance and training efficiency. An ablation study also demonstrates the necessity of CLHHN’s key components.

Deep artificial neural networks (ANNs) play a major role in modeling the visual pathways of primate and rodent. However, they highly simplify the computational properties of neurons compared to their biological counterparts. Instead, Spiking Neural Networks (SNNs) are more biologically plausible models since spiking neurons encode information with time sequences of spikes, just like biological neurons do. However, there is a lack of studies on visual pathways with deep SNNs models. In this study, we model the visual cortex with deep SNNs for the first time, and also with a wide range of state-of-the-art deep CNNs and ViTs for comparison. Using three similarity metrics, we conduct neural representation similarity experiments on three neural datasets collected from two species under three types of stimuli. Based on extensive similarity analyses, we further investigate the functional hierarchy and mechanisms across species. Almost all similarity scores of SNNs are higher than their counterparts of CNNs with an average of 6.6%. Depths of the layers with the highest similarity scores exhibit little differences across mouse cortical regions, but vary significantly across macaque regions, suggesting that the visual processing structure of mice is more regionally homogeneous than that of macaques. Besides, the multi-branch structures observed in some top mouse brain-like neural networks provide computational evidence of parallel processing streams in mice, and the different performance in fitting macaque neural representations under different stimuli exhibits the functional specialization of information processing in macaques. Taken together, our study demonstrates that SNNs could serve as promising candidates to better model and explain the functional hierarchy and mechanisms of the visual system.

Deep reinforcement learning (DRL) based recommender systems are suitable for user cold-start problems as they can capture user preferences progressively. However, most existing DRL-based recommender systems are suboptimal, since they use the same policy to suit the dynamics of different users. We reformulate recommendation as a multitask Markov Decision Process, where each task represents a set of similar users. Since similar users have closer dynamics, a task-specific policy is more effective than a single universal policy for all users. To make recommendations for cold-start users, we use a default policy to collect some initial interactions to identify the user task, after which a task-specific policy is employed. We use Q-learning to optimize our framework and consider the task uncertainty by the mutual information regarding tasks. Experiments are conducted on three real-world datasets to verify the effectiveness of our proposed framework.

The sequential recommendation (also known as the next-item recommendation), which aims to predict the following item to recommend in a session according to users’ historical behavior, plays a critical role in improving session-based recommender systems. Most of the existing deep learning-based approaches utilize the recurrent neural network architecture or self-attention to model the sequential patterns and temporal influence among a user's historical behavior and learn the user's preference at a specific time. However, these methods have two main drawbacks. First, they focus on modeling users’ dynamic states from a user-centric perspective and always neglect the dynamics of items over time. Second, most of them deal with only the first-order user-item interactions and do not consider the high-order connectivity between users and items, which has recently been proved helpful for the sequential recommendation. To address the above problems, in this article, we attempt to model user-item interactions by a bipartite graph structure and propose a new recommendation approach based on a Position-enhanced and Time-aware Graph Convolutional Network (PTGCN) for the sequential recommendation. PTGCN models the sequential patterns and temporal dynamics between user-item interactions by defining a position-enhanced and time-aware graph convolution operation and learning the dynamic representations of users and items simultaneously on the bipartite graph with a self-attention aggregator. Also, it realizes the high-order connectivity between users and items by stacking multi-layer graph convolutions. To demonstrate the effectiveness of PTGCN, we carried out a comprehensive evaluation of PTGCN on three real-world datasets of different sizes compared with a few competitive baselines. Experimental results indicate that PTGCN outperforms several state-of-the-art sequential recommendation models in terms of two commonly-used evaluation metrics for ranking. In particular, it can make a better trade-off between recommendation performance and model training efficiency, which holds great potential for online session-based recommendation scenarios in the future.

Fountain code, as a linear random code without a fixed bit rate, can be introduced into multi-path transmission control protocol (TCP) to improve throughput and reduce the bottleneck effect caused by path quality diversity. However, the multiple-hop transmission of data and huge feedback delay of acknowledge character (ACK) will lead to excessive transmission of fountain-coded packets on multiple paths, posing serious privacy concerns. To tackle with this problem, we propose a secure cross-locking enabled multi-route fountain coding scheme in this paper which can utilize the diversity of transmission quality among different paths to improve the security of the multi-route fountain code transmission system. Specifically, the fountain-coded packet data on the superior path of the legal receiver is employed to protect the fountain-coded packets on the inferior path, forming a cross-interlocking structure between multiple paths. This design can reduce the probability that the eavesdropper receives enough fountain-coded packets and decode successfully. Besides, the data received by the legal receiver is used to support implicit transmission of control information, further reducing the interception probability of eavesdropper. The simulation results finally verify the effectiveness of this scheme against eavesdropping attack.

In this paper, we investigate the issue of secure image transmission over point-to-point flat-fading channels under an eavesdropper. We propose an adaptive noise aggregation (ANA) scheme to worsen the eavesdropper’s reception by exploiting the inherent noise of wireless channels. Particularly, data packets to be sent are equally separated into multiple groups which are then correlated with each other for transmission in different time slots. An ANA scheme is proposed to encode even and odd packets in such a way that the correlation between data packets can be adaptively adjusted according to the average signal-to-noise ratio (SNR) fed back by the legal receiver. We prove that this scheme can significantly improve the receiving signal quality of legitimate receiver under low SNR, while paralyzing the decoding process of eavesdropper. Finally, simulation results show the superiority of our scheme over existing noise aggregation schemes.

Deep reinforcement learning (DRL)-based recommender systems have recently come into the limelight due to their ability to optimize long-term user engagement. A significant challenge in DRL-based recommender systems is the large action space required to represent a variety of items. The large action space weakens the sampling efficiency and thereby, affects the recommendation accuracy. In this article, we propose a DRL-based method called deep hierarchical category-based recommender system (DHCRS) to handle the large action space problem. In DHCRS, categories of items are used to reconstruct the original flat action space into a two-level category-item hierarchy. DHCRS uses two deep Q -networks (DQNs): 1) a high-level DQN for selecting a category and 2) a low-level DQN to choose an item in this category for the recommendation. Hence, the action space of each DQN is significantly reduced. Furthermore, the categorization of items helps capture the users' preferences more effectively. We also propose a bidirectional category selection (BCS) technique, which explicitly considers the category-item relationships. The experiments show that DHCRS can significantly outperform state-of-the-art methods in terms of hit rate and normalized discounted cumulative gain for long-term recommendations.

Next (or successive) point-of-interest (POI) recommendation, which aims to predict where users are likely to go next, has recently emerged as a new research focus of POI recommendation. Most of the previous studies on next POI recommendation attempted to incorporate the spatiotemporal information and sequential patterns of user check-ins into recommendation models to predict the target user's next move. However, few of the next POI recommendation approaches utilized the social influence of each user's friends. In this study, we discuss a new topic of next POI recommendation and present a deep attentive network for social-aware next POI recommendation called DAN-SNR. In particular, the DAN-SNR makes use of the self-attention mechanism instead of the architecture of recurrent neural networks to model sequential influence and social influence in a unified manner. Moreover, we design and implement two parallel channels to capture short-term user preference and long-term user preference as well as social influence, respectively. By leveraging multi-head self-attention, the DAN-SNR can model long-range dependencies between any two historical check-ins efficiently and weigh their contributions to the next destination adaptively. We also carried out a comprehensive evaluation using large-scale real-world datasets collected from two popular location-based social networks, namely, Gowalla and Brightkite. Experimental results indicate that the DAN-SNR outperforms seven competitive baseline approaches regarding recommendation performance and is highly efficient among six neural-network-based methods, four of which utilize the attention mechanism.

Next point-of-interest (POI) recommendation, also known as a natural extension of general POI recommendation, is recently proposed to predict user's next destination and has attracted considerable research interest. It focuses on learning users’ sequential patterns of check-in behavior and on training personalized recommendation models using different types of contextual information. Unfortunately, most of the previous studies failed to incorporate the spatiotemporal contextual information, which plays a critical role in analyzing user check-in behavior, into recommending the next POI. In recent years, embedding learning and recurrent neural network (RNN) based approaches show promising performance for modeling sequential patterns of check-in behavior in next POI recommendation. However, not all of the historical check-in records contribute equally to the next-step check-in behavior. To provide better next POI recommendation performance, we first proposed a spatiotemporal long and short-term memory (ST-LSTM) network. By feeding the spatiotemporal contextual information into the LSTM network in each step, ST-LSTM can model the spatial and temporal information better. Also, we developed an attention-based spatiotemporal LSTM (ATST-LSTM) network for next POI recommendation. By using the attention mechanism, ATST-LSTM can focus on the relevant historical check-in records in a check-in sequence selectively using the spatiotemporal contextual information. Besides, we conducted a comprehensive performance evaluation using large-scale real-world datasets collected from two popular location-based social networks, namely Gowalla and Brightkite. Experimental results indicated that the proposed ATST-LSTM network outperformed two state-of-the-art next POI recommendation approaches regarding three commonly-used evaluation metrics.

Social networks are widespread and important in our daily life. Finding communities and reveal node characteristics in community are crucial to understand the network structure and function. Many methods based on Nonnegative matrix factorization (NMF

Although recommendation techniques have achieved distinct developments over the decades, the data sparseness problem of the involved user-item matrix still seriously influences the recommendation quality. Most of the existing techniques for recommender systems cannot easily deal with users who have very few ratings. How to combine the increasing amount of different types of social information such as user generated content and social relationships to enhance the prediction precision of the recommender systems remains a huge challenge. In this paper, based on a factor graph model, we formalize the problem in a semi-supervised probabilistic model, which can incorporate different user information, user relationships, and user-item ratings for learning to predict the unknown ratings. We evaluate the method in two different genres of datasets, Douban and Last.fm. Experiments indicate that our method outperforms several state-of-the-art recommendation algorithms. Furthermore, a distributed learning algorithm is developed to scale up the approach to real large datasets.