Human-in-the-loop reinforcement learning integrates human expertise to accelerate agent learning and provide critical guidance and feedback in complex fields. However, many existing approaches focus on single-agent tasks and require continuous human involvement during the training process, significantly increasing the human workload and limiting scalability. In this paper, we propose HARP (HumanAssisted Regrouping with Permutation Invariant Critic), a multi-agent reinforcement learning framework designed for group-oriented tasks. HARP integrates automatic agent regrouping with strategic human assistance during deployment, enabling and allowing non-experts to offer effective guidance with minimal intervention. During training, agents dynamically adjust their groupings to optimize collaborative task completion. When deployed, they actively seek human assistance and utilize the Permutation Invariant Group Critic to evaluate and refine human-proposed groupings, allowing non-expert users to contribute valuable suggestions. In multiple collaboration scenarios, our approach is able to leverage limited guidance from non-experts and enhance performance. The project can be found at https://github.com/huawen-hu/HARP.

Machine learning models have achieved widespread success but often inherit and amplify historical biases, resulting in unfair outcomes. Traditional fairness methods typically impose constraints at the prediction level, without addressing underlying biases in data representations. In this work, we propose a principled framework that adjusts data representations to balance predictive utility and fairness. Using sufficient dimension reduction, we decompose the feature space into target-relevant, sensitive, and shared components, and control the fairness–utility trade-off by selectively removing sensitive information. We provide a theoretical analysis of how prediction error and fairness gaps evolve as shared subspaces are added, and employ influence functions to quantify their effects on the asymptotic behavior of parameter estimates. Experiments on both synthetic and real-world datasets validate our theoretical insights and show that the proposed method effectively improves fairness while preserving predictive performance.

The remarkable empirical performance of distributional reinforcement learning~(RL) has garnered increasing attention to understanding its theoretical advantages over classical RL. By decomposing the categorical distributional loss commonly employed in distributional RL, we find that the potential superiority of distributional RL can be attributed to a derived distribution-matching entropy regularization. This less-studied entropy regularization aims to capture additional knowledge of return distribution beyond only its expectation, contributing to an augmented reward signal in policy optimization. In contrast to the vanilla entropy regularization in MaxEnt RL, which explicitly encourages exploration by promoting diverse actions, the novel entropy regularization derived from categorical distributional loss implicitly updates policies to align the learned policy with (estimated) environmental uncertainty. Finally, extensive experiments verify the significance of this uncertainty-aware regularization from distributional RL on the empirical benefits over classical RL. Our study offers an innovative exploration perspective to explain the intrinsic benefits of distributional learning in RL.

The Motor Imagery (MI) decoding based on electroencephalogram (EEG), has promising applications. However, most current methods face two main issues: (1) They usually rely on convolutional neural networks to extract temporal features of MI signals without fully considering the brain’s functional connectivity during MI tasks. (2) They lack analysis and recognition of MI features slices and non-tasks slices within EEG signals, leading to poor generalization and robustness. To address these problems, we propose a novel deep learning model based on graph neural network to learn spatial features between multiple electrode channels and integrate the brain’s functional connectivity features. Additionally, it restructures time slices features segmented by the sliding time window algorithm to enhance MI temporal features in EEG signal. Therefor our model achieves the fusion of spatial and temporal features. To enhance the convergence effect of the model, we introduce electrode channel spatial positions as prior knowledge to initialize the parameters of the graph convolutional network parameters. Experimental evaluations on the publicly available EEG MI dataset from BCI Competition IV 2a show that our model achieves a four-class cross-session classification accuracy of 82.38%. Compared with other methods, our model yields the best results, demonstrating its superiority. Furthermore, the results indicate that the spatial feature obtained through our model bears resemblance to the brain functional connectivity patterns identified during MI tasks. To conclude, the fusion of spatial and temporal features with graph model shows the great application potential for EEG MI signals decoding and other EEG analysis.

The brain-computer interface (BCI) based on electroencephalogram (EEG) motor imagery (MI) decoding demonstrates promising application potential. However, the domain shift between training and testing data significantly impacts the model’s decoding efficacy. Domain adaption (DA) has been developed to address this problem recently. Nevertheless, existing DA methods have two limitations. One is that the extracted features are noisy, and the other is that they only align the distribution of features, which leads to limited generalization ability of the model. In this paper, we propose a novel smooth conditional domain adversarial training framework for solving the motor imagery decoding problem under domain shift. The framework uses interactive frequency convolution and channel attention mechanism as feature extractors to obtain effective features, and integrates smooth conditional domain adversarial training with batch spectral penalty to align the joint distribution of features and classes. At the same time, self-iterative training is implemented by generating pseudo-labels and selective outlier removal. Experimental results demonstrate that our proposed framework achieves 80.67% and 86.17% average accuracy in the BCI IV 2a and 2b respectively for cross-session experiments, achieving the best results compared with other methods, proving that the framework can improve the classification ability on the target domain while transferring effective features.

Electroencephalogram (EEG) motor imagery decoding, serving as the primary non-invasive modality for exploring brain-computer interfaces, has gained increasing attention. Previous research has achieved significant breakthroughs in the extraction and classification of features related to motor imagery. However, effectively integrating temporal-frequency patterns and capturing long-term dependencies across the entire sequence remain open challenges. To address these issues, we propose a novel Temporal-Frequency Hierarchical Transformer Network (TF-HiTNet) for EEG motor imagery classification. TF-HiTNet leverages a hierarchical transformer architecture and a feature fusion module to effectively extract and integrate temporal and frequency features from EEG signals. This approach captures both local features within EEG segments and global patterns across segments, while simultaneously considering information in both the time and frequency domains. Evaluation on the BCI4-2A and GigaDB datasets demonstrates the effectiveness of TF-HiTNet, achieving an average performance of 79.7% and 83.1%, respectively. Our experiments validate that the hierarchical transformer architecture can effectively learn the relationships between low-level and high-level features, while the time-frequency fusion module significantly improves the accuracy of motor imagery classify-cation.

Enze Shi, Lei Ding, Linglong Kong, Bei Jiang. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

As generative large language models (LLMs) such as ChatGPT gain widespread adoption in various domains, their potential to propagate and amplify social biases, particularly in high-stakes areas such as the labor market, has become a pressing concern. AI algorithms are not only widely used in the selection of job applicants, individual job seekers may also make use of generative LLMs to help develop their job application materials. Against this backdrop, this research builds on a novel experimental design to examine social biases within ChatGPT-generated job applications in response to real job advertisements. By simulating the process of job application creation, we examine the language patterns and biases that emerge when the model is prompted with diverse job postings. Notably, we present a novel bias evaluation framework based on Masked Language Models to quantitatively assess social bias based on validated inventories of social cues/words, enabling a systematic analysis of the language used. Our findings show that the increasing adoption of generative AI, not only by employers but also increasingly by individual job seekers, can reinforce and exacerbate gender and social inequalities in the labor market through the use of biased and gendered language.

Electroencephalogram (EEG) is one of the most widely used brain computer interface (BCI) approaches. Despite the success of existing EEG approaches in brain state recognition studies, it is still challenging to differentiate brain states via explainable and generalizable deep learning approaches. In other words, how to explore meaningful and distinguishing features and how to overcome the huge variability and overfitting problem still need to be further studied. To alleviate these challenges, in this work, a multiple random fragment search-based multilayer recurrent neural network (MRFS-MRNN) is proposed to improve the differentiating performance and explore meaningful patterns. Specifically, an explainable MRNN module is proposed to capture the temporal dependences preserved in EEG time series. Besides, a MRFS module is designed to cut multiple random fragments from the entire EEG signal time course to improve the effectiveness of brain state differentiating ability. MRFS-MRNN is concatenatedto effectively overcome the huge variabilities and overfitting problems. Experiment results demonstrate that the proposed MRFS-MRNN model not only has excellent differentiating performance, but also has good explanation and generalization ability. The classification accuracies reach as high as 95.18% for binary classification and 89.19% for four-category classification on the individual level. Similarly, 95.53% and 85.84% classification accuracies are obtained for the binary and four-category classification on the group level. What's more, 94.28% and 85.43% classification accuracies of binary and four-category classifications are achieved for predicting brand new subjects. The experiment results showed that the proposed method outperformed other state-of-the-art (SOTA) models on the same underlying data and improved the explanation and generalization ability.

5G and Industrial Internet are bringing a variety of applications with on-time and reliable demands. Cyclic queuing and forwarding (CQF), a well-known mechanism defined by IEEE 802.1 Qch in Time Sensitive Network (TSN), achieves deterministic end-to-end latency and jitter without complex gating calculations. However, most of the current work ignores the prevalence of hybrid networks with different link rates, resulting in low bandwidth utilization and high latency for single-cycle CQF. In this paper, we propose a multi-cycle CQF to address the transmission in multi-link-rate networking, reducing deterministic end-to-end latency and improving link bandwidth utilization. In addition, we formulate the scheduling constraints, being of guiding significance for designing the transmission of multi-link-rate networks, and we design an online scheduling algorithm based on it. We compare the proposed scheme with the single-cycle CQF online scheduling algorithm in hierarchical multi-link-rate networking scenarios, and the evaluation shows that our algorithm achieves better end-to-end ultra-low latency (38.9% reduction) with a smaller schedulability gap compared with single-cycle CQF.