In recent years, deep multi-agent reinforcement learning (MARL) has demonstrated remarkable potential in solving complex cooperative tasks by enabling decentralized yet efficient coordination among agents. However, during decentralized training, agent policy updates induced by different joint action samples may conflict, leading to gradient interference that hinders convergence and the emergence of coordinated behavior. In this paper, we analyze and empirically validate the phenomenon of gradient interference. To address this, we then propose Gradient-Protected Value Decomposition (GPVD), a novel MARL framework that explicitly protects the gradient signals of optimal collaborative actions by suppressing the impact of interfering actions. GPVD employs a dynamic gradient protection mechanism that identifies optimal collaborative joint actions and reweights the loss to attenuate gradients from non-collaborative interfering actions. To effectively identify high-value collaborative actions, we apply SimHash-based state grouping to discover consistent collaboration patterns across similar states. Furthermore, a count-based intrinsic reward is incorporated to encourage exploration and improve the coverage of potentially optimal joint actions. Experiments on challenging multi-agent benchmarks demonstrate that GPVD achieves faster convergence, stronger coordination, and greater training stability compared to state-of-the-art value decomposition methods.

Community detection is an essential approach for understanding and analyzing complex networks. Traditional clustering methods in the framework of machine learning are utilized for community detection based on the accessible adjacent matrix of complex networks. The adjacent matrix commonly reveals the topological structures of networks in a binary form, the numerical characteristic of which may not receive enough attention in community detection. In this paper, a class of weighted nonnegative matrix factorization including semi-supervised weighted least square errors-based nonnegative matrix factorization (SWLSENMF) and semi-supervised weighted KL divergence-based nonnegative matrix factorization (SWKLNMF) is proposed. The novel approach exponentially weights the nonnegative matrix factorization to approximate the adjacent matrix in accordance with its numerical characteristic. In addition, the part of a prior information about the complex network community is used in semi-supervised machine learning for further improving detection accuracy, which is demonstrated through numerical examples on real-world networks. The experiment results show that the proposed SWLSENMF and SWKLNMF perform better than traditional semi-supervised least square errors-based nonnegative matrix factorization (SLSENMF) and semi-supervised KL divergence-based nonnegative matrix factorization (SKLNMF).

Despite the success of Multi-Agent Reinforcement Learning (MARL) algorithms in cooperative tasks, previous works, unfortunately, face challenges in heterogeneous scenarios since they simply disable parameter sharing for agent specialization. Sequential updating scheme was thus proposed, naturally diversifying agents by encouraging agents to learn from preceding ones. However, the exploration strategy in sequential scheme has not been investigated. Benefiting from updating one-by-one, agents have the access to the information from preceding agents. Thus, in this work, we propose to exploit the preceding information to enhance exploration and heterogeneity sequentially. We present Multi-Agent Divergence Policy Optimization (MADPO), equipped with mutual policy divergence maximization framework. We quantify the policy discrepancies between episodes to enhance exploration and between agents to heterogenize agents, termed intra-agent and inter-agent policy divergence. To address the issue that traditional divergence measurements lack stability and directionality, we propose to employ the conditional Cauchy-Schwarz divergence to provide entropy-guided exploration incentives. Extensive experiments show that the proposed method outperforms state-of-the-art sequential updating approaches in two challenging multi-agent tasks with various heterogeneous scenarios. Source code is available at \url{https://github.com/hwdou6677/MADPO}.

The multivariate autoregressive (MVAR) model is widely used in describing the dynamics of nonlinear systems, in which the estimates of model parameters and underlying states can be achieved by dual extended Kalman filter (DEKF). However, when the measurements are corrupted by complicated non-Gaussian noises, the DEKF based on the minimum mean-square error (MMSE) criterion may provide biased estimates. In the present article, we develop a novel dual Kalman-type filter, referred to as DEKF under minimum error entropy (MEE) with fiducial points (MEEFs-DEKF) to deal with the non-Gaussian noises. First, the equivalent state-space model and parameter-space model are presented based on the MVAR model. Then, an optimality criterion based on MEE with fiducial points (MEEFs) is applied in the batch-mode regression equation to improve the robustness. Finally, a fixed-point iteration algorithm gives the posterior estimates of state and parameter. Simulation results confirm that the proposed MEEF-DEKF can achieve excellent performance in various noises with different distributions, especially in heavy-tailed and multimodal noises.

Spectrum sharing in a cognitive radio system involves a secondary user updating transmit power for sharing spectrum with a primary user. The deep$Q$-network in the framework of deep reinforcement learning achieves transmit power control by a deep neural network for learning a nonlinear mapping from states to$Q$-values. Since a deep neural network is confronted with noise susceptibility, the deep$Q$-network produces deteriorative network parameters and volatile$Q$-values in the presence of contaminated states. In view of the positive effect of kernel least mean square (KLMS) for signal smoothing, we combine KLMS with the deep$Q$-network for smoothing network-generated outputs. Since an inappropriate step size of KLMS causes under-smoothing or over-smoothing issues, a weighting procedure using past$Q$-values is proposed for cooperating with KLMS. We assess the incremental ratio of the success rate of the smooth deep$Q$-network to that of the deep$Q$-network (RSR) in cognitive radios. Simulations show that RSR has averaged almost over 30% at the early stage of power control. In particular, the maximum RSR reaches almost over 80% or 180% at different scenarios of power control for the primary user. In addition, the smooth deep$Q$-network achieves an improved success rate in comparison with other algorithms.

To date, most linear and nonlinear Kalman filters (KFs) have been developed under the Gaussian assumption and the well-known minimum mean square error (MMSE) criterion. In order to improve the robustness with respect to impulsive (or heavy-tailed) non-Gaussian noises, the maximum correntropy criterion (MCC) has recently been used to replace the MMSE criterion in developing several robust Kalman-type filters. To deal with more complicated non-Gaussian noises such as noises from multimodal distributions, in this article, we develop a new Kalman-type filter, called minimum error entropy KF (MEE-KF), by using the minimum error entropy (MEE) criterion instead of the MMSE or MCC. Similar to the MCC-based KFs, the proposed filter is also an online algorithm with the recursive process, in which the propagation equations are used to give prior estimates of the state and covariance matrix, and a fixed-point algorithm is used to update the posterior estimates. In addition, the MEE extended KF (MEE-EKF) is also developed for performance improvement in the nonlinear situations. The high accuracy and strong robustness of MEE-KF and MEE-EKF are confirmed by experimental results.

This paper presents a novel nonlinear adaptive filter method, namely, Hammerstein adaptive filter with single feedback under minimum mean square error (HAF-SF-MMSE). A single delayed output is incorporated into the estimation of the current output based on minimum mean square error criterion, and therefore the history information of output is considered. Moreover, hybrid learning rates and adaptive learning rates with normalization factors are designed to guarantee the convergence and stability of HAF-SF-MMSE. Compared with the traditional Hammerstein adaptive filter which usually consists of a nonlinear filter followed by a linear part, HAF-SF-MMSE can achieve a faster convergence rate and higher filter accuracy. Theoretical analysis regarding convergence behavior is performed to acquire a sufficient condition on the convergence of weight. Simulation results show the excellent filtering performance of the proposed HAF-SF-MMSE.

In this paper, the maximum correntropy (MC) criterion is used as the cost function in the online sequential extreme learning machine (OS-ELM) algorithm and constraint OS-ELM (COS-ELM) algorithm, generating the proposed OS-ELM based on maximum correntropy (OS-ELM-MC) and COS-ELM based on maximum correntropy (COS-ELM-MC). In comparison with OS-ELM and COS-ELM, the proposed OS-ELM-MC and COS-ELM-MC present superior performance in non-Gaussian noise environments and almost the same performance in Gaussian noise environments. As an important parameter, the hidden node number is also discussed by simulations in this paper. Simulations on the examples of Mackey-Glass (MG) chaotic time series prediction and nonlinear regression validate the efficiency of the proposed OS-ELM-MC and COS-ELM-MC.