Attributing to the strong reasoning capability, behavior models in artificial intelligence for games play a crucial role in creating gaming experiences. For further enhancing the reasoning capability of behavior models, we propose a novel Switch, Reason, and Revise (SRR) framework, which integrates them with Large Language Model (LLM). The SRR framework contains three core components. The component of Dual-Track Experiential Reasoning fully utilizes the agent experiences for reasoning. The component of Block-Retrieval-Augmented Thoughts adaptively determines the granularity of information retrieval for external sources. The component of Self-Reliant Thinking System Switch increases the reasoning speed by model switching and performs the model switching automatically upon the LLM. Together, these three components can strengthen the agent reasoning capability in complex tasks. Experimental results in the Pokémon battle environment demonstrate the effectiveness and efficiency superiority of SRR over the rival methods. Furthermore, we conduct an exploratory study to reveal the potential of the SRR-empowered agent for guiding new players in Pokémon battle games.

Multiagent reinforcement learning (MARL) methods have good application performances and prospects in cooperative tasks. To improve the capability of agent policy learning in new scenarios, some methods transfer the learned policy knowledge to new scenarios. However, most methods only focus on the knowledge transfer of individual agent policies, neglecting the credit assignment among agents in cooperative tasks, which results in a transfer bias of cooperative policies. In this paper, we propose a novel method, transferable policy generation and credit assignment (TPGCA) network for cooperative MARL. TPGCA can transfer the entire MARL model by the constructed transferable$Q$-value network and mixing network. Specifically, in TPGCA, to enhance the effectivity and transferability of agent policies, we design the correspondence network between observations and actions (COA) on the basis of transformer and gated recurrent unit (GRU). To implement the reliable credit assignment and diminish the transfer bias, we devise the role-based joint$Q$-value decomposition network (RVD) that can evaluate the contributions of agents from different observation perspectives. Experimental results in various micro-management scenarios on StarCraft multiagent challenge (SMAC) and multiagent particle environment (MPE) sufficiently demonstrate the effectiveness and transferability of TPGCA.

In reinforcement learning, the agents need to sufficiently explore the environment and efficiently exploit the existing experiences before finding the solution to the tasks, particularly in cooperative multi-agent scenarios where the state and action spaces grow exponentially with the number of agents. Hence, enhancing the exploration ability of agents and improving the utilization efficiency of experiences are two critical issues in cooperative multi-agent reinforcement learning. We propose a novel method called Generative Adversarial Imitation Learning Policy Gradients (GAILPG). The contributions of GAILPG are as follows: (a) we integrate generative adversarial self-imitation learning into the multi-agent actor-critic framework to improve the utilization efficiency of experiences, thus further assisting the policy training; (b) we design a new curiosity module to enhance the exploration ability of the agents. Experimental results on the StarCraft II micromanagement benchmark demonstrate that GAILPG surpasses state-of-the-art policy-based methods and is even on par with the value-based methods. And the ablation experiments validate the reasonability of the discriminator module and the curiosity module encapsulated in our method.

Knowledge distillation stands as a capable technique for transferring knowledge from a larger to a smaller model, thereby notably enhancing the smaller model’s performance. In the recent past, data augmentation has been employed in contrastive learning based knowledge distillation techniques yielding superior results. Despite the significant role of data augmentation, its value remains underappreciated within the domain of knowledge distillation, with no in-depth analysis in the literature thus far. To make up for this oversight, we conduct a multi-perspective theoretical and experimental analysis on the role that data augmentation can play in knowledge distillation. We summarize the properties of data augmentation and list the core findings as follows. (a) Our investigations validate that data augmentation significantly boosts the performance of knowledge distillation on the tasks of image classification and object detection. And this holds true even if the teacher model lacks comprehensive information about the augmented samples. Moreover, our novel J oint D ata A ugmentation (JDA) approach outperforms single data augmentation in knowledge distillation. (b) The pivotal role of data augmentation in knowledge distillation can be theoretically explained via Sharpness-Aware Minimization. (c) The compatibility of data augmentation with various knowledge distillation methods can enhance their performance. In light of these observations, we propose a new method called C osine C onfidence D istillation (CCD) for more reasonable knowledge transfer from augmented samples. Experimental results not only demonstrate that CCD becomes the state-of-the-art method with less storage requirement on CIFAR-100 and ImageNet-1k, but also validate the superiority of CCD over DIST on the object detection benchmark dataset, MS-COCO.

Credit assignment is a critical problem in cooperative Multi-Agent Reinforcement Learning (MARL). To address this problem, current studies mainly rely on the intrinsic reward, which is directly summed with the global reward to generate a total reward. However, such kinds of intrinsic reward functions ignore the dependence among agents and inevitably limit the adaptivity and effectiveness of MARL methods. In this paper, we propose a novel method, Attention-based Intrinsic Reward Mixing Network (AIRMN), for credit assignment in MARL. Specifically, we design a new intrinsic reward network on the basis of the attention mechanism, in order to enhance the effectiveness of teamwork. Besides, we devise a new mixing network that combines the intrinsic and extrinsic rewards in a nonlinear and dynamic manner, so as to adapt the total reward to the variation of the environment. Experimental results on the battle games of StarCraft II demonstrate that AIRMN outperforms the state-of-the-art methods in terms of the average test win rate, and also validate that AIRMN can dynamically return the precise intrinsic reward to each agent based on their contributions to the team cooperation, thereby better dealing with the credit assignment problem.

Multiagent reinforcement learning (MARL) has made remarkable progress in recent years. However, in most MARL methods, agents share a policy or value network, which is easy to result in similar behaviors of agents, and thus, limits the flexibility of the method to handle complex tasks. To enhance the diversity of agent behaviors, we propose a novel method, making decisions from different perspectives (MDDP). This method enables agents to switch flexibly between different policy roles and make decisions from different perspectives, which can improve the adaptability of policy learning in complex scenarios. Specifically, in MDDP, we design a new self-attention and gated recurrent unit (GRU)-based dueling architecture network (SG-DAN) to estimate the individual$Q$-values. SG-DAN contains two components: 1) the new self-attention-based role-switching network (SAR) and the capable GRU-based state value estimation network (GSE). SAR takes charge of action advantage estimation and GSE is responsible for state value estimation. Experimental results on the challengingStarCraftII micromanagement benchmark not only verify the modeling reasonability of MDDP but also demonstrate its performance superiority over the related advanced approaches.

Recent researches combine domain adaptation methods with elaborate feature extractors to better learn domain-invariant and discriminative features for cross-subject Electroencephalogram(EEG)-based emotion recognition. Existing models only utilize domain adaptation to constrain spatial learning or temporary learning, though the domain shift will possibly appear in both spatial and temporal learning stages. And some models simply treat the different subjects in the source domain as a whole, ignoring the data structure of the source domain. Motivated by the above problems, we design a novel model, Spatial-Temporal Constraint Learning (STCL), which adopts the Multi-Layer Perceptron (MLP) and Transformer Encoder for spatial and temporal features learning, respectively. In the spatial learning stage, we design Multi-Subject Prototypes Alignment (MSPA), which treats different subjects as different domains. In the temporary learning stage, we utilize the adversarial training strategy which treats different subjects in the source domain as a whole to further narrow down the domain gap. In addition, to improve the representative ability of our model in the target domain, we put forward Target Samples Selective Strategy (TSSS) which selects the samples from the target domain with reliable pseudo-labels for training STCL. The cross-subject experiments on two benchmark datasets have demonstrated the effectiveness of our model. Our model achieves the accuracies of 83.43 %, 78.36 %, and 80.09 % for three sessions respectively on SEED, 60.51 % for valence classification, and 63.68 % for arousal classification on DEAP.

Emotion plays an important role in human cognition and behavior. How to recognize emotions based on physiological signals has attracted increasing research interests worldwide up to date. Both traditional eastern medicine and modern western medicine have confirmed the existence of relationship between human emotionality and heart activity. However, in practice, emotion recognition only using Electrocardiogram (ECG) signals seems quite challenging, not only due to the severe noise interferences and the serious data variations, but also because of the ambiguous relationship between emotional states and ECG data. Such difficulty can even be compared to finding needles in a haystack. As an innovative endeavor to deal with the issue of only-ECG-based emotion recognition, this paper has proposed a novel solution from the perspective of weak signal classification. The proposed solution extracts the static-dynamic representation under the principle of Yin-Yang balance from the heartbeat data, and then utilizes the set-based collaborative measurement upon the thought of data coopetition to classify these features for recognizing emotions. Experimental results have demonstrated the effectiveness, efficiency and adaptiveness of the solution for uncovering the potential relationship between emotions and ECG. Thus, this proposal has also illuminated a promising research direction for the general problem of weak signal classification.

Electroencephalogram (EEG)-based emotion recognition has gradually become a research hotspot. However, the large distribution differences of EEG signals across subjects make the current research stuck in a dilemma. To resolve this problem, in this article, we propose a novel and effective method, Multi-Source Feature Representation and Alignment Network (MS-FRAN). The effectiveness of proposed method mainly comes from three new modules: Wide Feature Extractor (WFE) for feature learning, Random Matching Operation (RMO) for model training, and Top- h ranked domain classifier selection (TOP) for emotion classification. MS-FRAN is not only effective in aligning the distributions of each pair of source and target domains, but also capable of reducing the distributional differences among the multiple source domains. Experimental results on the public benchmark datasets SEED and DEAP have demonstrated the advantage of our method over the related competitive approaches for cross-subject EEG-based emotion recognition.

Multiple-shot person re-identification tackles the problem to build the correspondences between sets of human images obtained from distributed cameras. It is challenging due to large within-class variations and small between-class differences, caused by the changing of human appearance and environment. Existing methods for addressing this issue include designing the representation to capture the within-set correlation, or crafting the measure to explore the between-set separation. This paper proposes a novel set based matching model called “Locality Based Discriminative Measure (LBDM)”, in which the discriminative potentiality of a new set-to-set distance is exploited by using the learned local metric field. As experimentally demonstrated, the proposal remarkably outperforms state-of-the-art schemes on public benchmark datasets.

Person re-identification concerns about the problem of recognizing people across space (captured by different cameras) and/or over time gaps. Though recently the literature on it grows rapidly, all the proposed solutions have treated it as a normal classification or ranking problem. In this paper, however, we argue that it is in fact a natural transfer learning problem, thus it's valuable and also necessary to investigate how the progress on transfer learning could benefit the research on it. We present so far the first study on justifying the effectiveness of a representative transfer learning methodology: feature-based inductive transfer learning, for person re-identification. Extensive experiments on standard datasets with typical methods result in several important findings.

In this paper we propose a simple and effective solution to the important and challenging problem of across-camera person re-identification. We focus on the common case in video surveillance where multiple images or video frames are available for each person. Instead of exploring new features, the proposed approach aims at making a better use of such images/frames. It builds a collaborative representation over all the gallery images (of known person individuals) to best approximate the query images (containing an unknown person) via affine combinations. The approximation is measured by the nearest point distance between the two affine hulls constructed by the query images and gallery images, respectively. By enforcing the sparsity of the samples used for approximating the two nearest points, the relative importance of the gallery images belonging to different persons has the ability to reveal the identity of the querying person. Extensive experiments on public benchmark datasets demonstrate that the proposed approach greatly outperforms the state-of-the-art methods.

Person re-identification tackles the problem whether an observed person of interest reappears in a network of cameras. The difficulty primarily originates from few samples per class but large amounts of intra-class variations in real scenarios: illumination, pose and viewpoint changes across cameras. So far, proposals in the literature have treated this either as a matching problem focusing on feature representation or as a classification/ranking problem relying on metric optimization. This paper presents a new way called Common-Near-Neighbor Analysis, which to some extent combines the strengths of these two methodologies. It analyzes the commonness of the near neighbors of each pair of samples in a learned metric space, measured by a novel rank-order based dissimilarity. Our method, using only color cue, has been tested on widely-used benchmark datasets, showing significant performance improvement over the state-of-the-art.