Federated Learning (FL) enables joint training across distributed clients using their local data privately. Federated Multi-Task Learning (FMTL) builds on FL to handle multiple tasks, assuming model congruity that identical model architecture is deployed in each client. To relax this assumption and thus extend real-world applicability, we introduce a novel problem setting, Hetero-Client Fed-erated Multi-Task Learning (HC-FMTL), to accommodate diverse task setups. The main challenge of HC-FMTL is the model incongruity issue that invalidates conventional aggregation methods. It also escalates the difficulties in model aggregation to deal with data and task heterogeneity inherent in FMTL. To address these challenges, we pro-pose the$FedHCA^{2}$framework, which allows for federated training of personalized models by modeling relationships among heterogeneous clients. Drawing on our theoretical insights into the difference between multi-task and federated optimization, we propose the Hyper Conflict-Averse Aggregation scheme to mitigate conflicts during encoder updates. Additionally, inspired by task interaction in MTL, the Hyper Cross Attention Aggregation scheme uses layer-wise cross attention to enhance decoder interactions while alleviating model incongruity. Moreover, we employ learnable Hyper Aggregation Weights for each client to customize personalized parameter updates. Extensive experiments demon-strate the superior performance of$FedHCA^{2}$in various HC-FMTL scenarios compared to representative methods. Code is available at https://github.com/innovator-zero/FedHCA2.

Federated Learning (FL) has emerged as a promising approach to enable collaborative learning among multiple clients while preserving data privacy. However, cross-domain FL tasks, where clients possess data from different domains or distributions, remain a challenging problem due to the inherent heterogeneity. In this paper, we present UNIDEAL, a novel FL algorithm specifically designed to tackle the challenges of cross-domain scenarios and heterogeneous model architectures. The proposed method introduces Adjustable Teacher-Student Mutual Evaluation Curriculum Learning, which significantly enhances the effectiveness of knowledge distillation in FL settings. We conduct extensive experiments on various datasets, comparing UNIDEAL with state-of-the-art baselines. Our results demonstrate that UNIDEAL achieves superior performance in terms of both model accuracy and communication efficiency. Additionally, we provide a convergence analysis of the algorithm, showing a convergence rate of $O\left( {\frac{1}{T}} \right)$ under non-convex conditions.

This study presents the Task-level Heterogeneous Federated Learning (TH-FL), a novel paradigm that fuses the principles of Federated Learning (FL) and Multi-Task Learning (MTL). In the TH-FL scenario, each client can learn an indefinite number of tasks, which may vary in type and originate from distinct domains. We introduce a unique baseline model, BARTENDER, that integrates a Conditional Prompt (CP) module. This module encodes task-specific and domain-specific information, enabling the model to generate tailored outputs based on the encoding inputs. This innovative strategy not only minimizes the communication costs associated with FL but also enhances model generalization across a variety of task types. Through extensive experiments, we establish that the BARTENDER model surpasses traditional multi-decoder architecture models across diverse scenarios. We also explore the influence of the parameter decoupling strategy on model training and outline the assumptions necessary for achieving a $O\left( {1/\sqrt T } \right)$ convergence speed in the TH-FL scenario.

Community detection on attributed graphs with rich semantic and topological information offers great potential for real-world network analysis, especially user matching in online games. Graph Neural Networks (GNNs) have recently enabled Deep Graph Clustering (DGC) methods to learn cluster assignments from semantic and topological information. However, their success depends on the prior knowledge related to the number of communities K, which is unrealistic due to the high costs and privacy issues of acquisition. In this paper, we investigate the community detection problem without prior K, referred to as K-Free Community Detection problem. To address this problem, we propose a novel Deep Adaptive and Generative model~(DAG) for community detection without specifying the prior K. DAG consists of three key components, i.e., a node representation learning module with masked attribute reconstruction, a community affiliation readout module, and a community number search module with group sparsity. These components enable DAG to convert the process of non-differentiable grid search for the community number, i.e., a discrete hyperparameter in existing DGC methods, into a differentiable learning process. In such a way, DAG can simultaneously perform community detection and community number search end-to-end. To alleviate the cost of acquiring community labels in real-world applications, we design a new metric, EDGE, to evaluate community detection methods even when the labels are not feasible. Extensive offline experiments on five public datasets and a real-world online mobile game dataset demonstrate the superiority of our DAG over the existing state-of-the-art (SOTA) methods. DAG has a relative increase of 7.35% in teams in a Tencent online game compared with the best competitor.

The innovative Federated Multi-Task Learning (FMTL) approach consolidates the benefits of Federated Learning (FL) and Multi-Task Learning (MTL), enabling collaborative model training on multi-task learning datasets. However, a comprehensive evaluation method, integrating the unique features of both FL and MTL, is currently absent in the field. This paper fills this void by introducing a novel framework, FMTL-Bench, for systematic evaluation of the FMTL paradigm. This benchmark covers various aspects at the data, model, and optimization algorithm levels, and comprises seven sets of comparative experiments, encapsulating a wide array of non-independent and identically distributed (Non-IID) data partitioning scenarios. We propose a systematic process for comparing baselines of diverse indicators and conduct a case study on communication expenditure, time, and energy consumption. Through our exhaustive experiments, we aim to provide valuable insights into the strengths and limitations of existing baseline methods, contributing to the ongoing discourse on optimal FMTL application in practical scenarios. The source code can be found at https://github.com/youngfish42/FMTL-Benchmark.

Traditional multi-task learning often relies on explicit task interaction mechanisms to enhance multi-task performance. However, these approaches encounter challenges such as negative transfer when jointly learning multiple weakly correlated tasks. Additionally, these methods handle encoded features at a large scale, which escalates computational complexity to ensure dense prediction task performance. In this study, we introduce a Task-Interaction-Free Network (TIF) for multi-task learning, which diverges from explicitly designed task interaction mechanisms. Firstly, we present a Scale Attentive-Feature Fusion Module (SAFF) to enhance each scale in the shared encoder to have rich task-agnostic encoded features. Subsequently, our proposed task and scale-specific decoders efficiently decode the enhanced features shared across tasks without necessitating task-interaction modules. Concretely, we utilize a Self-Feature Distillation Module (SFD) to explore task-specific features at lower scales and the Low-To-High Scale Feature Diffusion Module (LTHD) to diffuse global pixel relationships from low-level to high-level scales. Experiments on publicly available multi-task learning datasets validate that our TIF attains state-of-the-art performance.

Image-level weakly supervised semantic segmentation faces challenges in accurately capturing boundaries and representing intricate details due to the absence of pixel-level supervision. Constrained by the enormous number of pixels, pixel-level propagation has difficulty in capturing the long-range dependency, particularly in small, isolated regions. To this end, we introduce a novel approach of self-supervised segmentation integrated with superpixel, and develop a network called superpixel guided network (SPGNet) to simultaneously perform superpixel generation and segmentation mask prediction. Significantly, our framework facilitates mutual supervised learning between the segmentation branch and the superpixel branch. The superpixel guides the predicted mask for improved boundary location, while the latter provides supervision on superpixel through superpixel center generation (SCG) and union boundary extraction (UBE). Furthermore, we propose superpixel context fusion (SCF) to generate compact pseudo masks and capture long-range dependency. Experimental results demonstrate that the proposed SPGNet achieves outstanding performance on the PASCAL VOC 2012 segmentation benchmark

Data-efficient learning on graphs (GEL) is essential in real-world applications. Existing GEL methods focus on learning useful representations for nodes, edges, or entire graphs with "small" labeled data. But the problem of data-efficient learning for subgraph prediction has not been explored. The challenges of this problem lie in the following aspects: 1) It is crucial for subgraphs to learn positional features to acquire structural information in the base graph in which they exist. Although the existing subgraph neural network method is capable of learning disentangled position encodings, the overall computational complexity is very high. 2) Prevailing graph augmentation methods for GEL, including rule-based, sample-based, adaptive, and automated methods, are not suitable for augmenting subgraphs because a subgraph contains fewer nodes but richer information such as position, neighbor, and structure. Subgraph augmentation is more susceptible to undesirable perturbations. 3) Only a small number of nodes in the base graph are contained in subgraphs, which leads to a potential "bias" problem that the subgraph representation learning is dominated by these "hot" nodes. By contrast, the remaining nodes fail to be fully learned, which reduces the generalization ability of subgraph representation learning. In this paper, we aim to address the challenges above and propose a Position-Aware Data-Efficient Learning framework for subgraph neural networks called PADEL. Specifically, we propose a novel node position encoding method that is anchor-free, and design a new generative subgraph augmentation method based on a diffused variational subgraph autoencoder, and we propose exploratory and exploitable views for subgraph contrastive learning. Extensive experiment results on three real-world datasets show the superiority of our proposed method over state-of-the-art baselines.

In this paper, we propose an environment semantics aided wireless communication framework to reduce the transmission latency and improve the transmission reliability, where semantic information is extracted from environment image data, selectively encoded based on its task-relevance, and then fused to make decisions for channel related tasks. As a case study, we develop an environment semantics aidednetwork architecturefor mmWave communication systems, which is composed of a semantic feature extraction network, a feature selection algorithm, a task-oriented encoder, and a decision network. With images taken from street cameras and user’s identification information as the inputs, the environment semantics aided network architecture is trained to predict the optimal beam index and the blockage state for the base station. It is seen that without pilot training or costly beam scans, the environment semantics aided network architecture can realize extremely efficient beam prediction and timely blockage prediction, thus meeting requirements for ultra-reliable and low-latency communications (URLLCs). Simulation results demonstrate that compared with existing works, the proposed environment semantics aided network architecture can reduce system overheads such as storage space and computational cost while achieving satisfactory prediction accuracy and protecting user privacy.

In this paper, we develop a dynamic detection network (DDNet) based detector for multiple-input multiple-output (MIMO) systems. By constructing an improved DetNet (IDetNet) detector and the OAMPNet detector as two independent network branches, the DDNet detector performs sample-wise dynamic routing to adaptively select a better one between the IDetNet and the OAMPNet detectors for every samples under different system conditions. To avoid the prohibitive transmission overhead of dataset collection in centralized learning (CL), we propose the federated averaging (FedAve)-DDNet detector, where all raw data are kept at local clients and only locally trained model parameters are transmitted to the central server for aggregation. To further reduce the transmission overhead, we develop the federated gradient sparsification (FedGS)-DDNet detector by randomly sampling gradients with elaborately calculated probability when uploading gradients to the central server. Based on simulation results, the proposed DDNet detector consistently outperforms other detectors under all system conditions thanks to the sample-wise dynamic routing. Moreover, the federated DDNet detectors, especially the FedGS-DDNet detector, can reduce the transmission overhead by at least 25.7% while maintaining satisfactory detection accuracy.

Achieving adequate precision in deep learning based communications often requires large network architectures, which results into unacceptable time delay and power consumption. This paper introduces the dynamic neural network (DyNN) into the design of wireless communications systems. DyNN allocates different samples with computation resources on demand by preforming dynamic inferences, thereby reducing the redundant computational cost and enhancing the network efficiency. We design a dynamic depth architecture that allows samples to adaptively skip layers with various dynamic strategies, from which we further develop aconfidence criterion baseddynamicimproved DetNet (CD-IDetNet) and apolicy network baseddynamicimproved DetNet (PD-IDetNet) for multiple-input multiple-output (MIMO) detection. Specifically, in CD-IDetNet, a confidence criterion is adopted to control samples exiting early, while in PD-IDetNet, policy networks are trained by reinforcement learning to selectively skip layers for varying samples. Simulation results demonstrate that CD-IDetNet and PD-IDetNet detectors can respectively reduce 17.4% and 31.1% computational costs while preserving the full accuracy of IDetNet. Desirable tradeoffs between accuracy and computational complexity can be further achieved by fine-tuning the hyper-parameters of CD-IDetNet and PD-IDetNet. Moreover, over-the-air (OTA) tests are conducted to validate the effectiveness of the proposed detectors in practical systems.

In this paper, we propose a novel deep unsupervised learning-based approach that jointly optimizes antenna selection and hybrid beamforming to improve the hardware and spectral efficiencies of massive multiple-input-multiple-output (MIMO) downlink systems. By employing ResNet to extract features from the channel matrices, two neural networks, i.e., the antenna selection network (ASNet) and the hybrid beamforming network (BFNet), are respectively proposed for dynamic antenna selection and hybrid beamformer design. Furthermore, a deep probabilistic subsampling trick and a specially designed quantization function are respectively developed for ASNet and BFNet to preserve the differentiability while embedding discrete constraints into the network structures. With the aid of a flexibly designed loss function, ASNet and BFNet are jointly trained in a phased unsupervised way, which avoids the prohibitive computational cost of acquiring training labels in supervised learning. Simulation results demonstrate the advantage of the proposed approach over conventional optimization-based algorithms in terms of both the achieved rate and the computational complexity.

Existing deep learning (DL) based downlink channel prediction algorithms for frequency division duplex (FDD) massive multiple-input multiple-output (MIMO) systems mainly utilize single-source sensing information, e.g., the uplink channels, to predict the downlink channels. With the aid of multi-source sensing information (MSI) in communication systems, this paper explores deep multimodal learning (DML) technologies to improve the accuracy of downlink channel prediction. By leveraging various modality combinations and fusion levels, we design several DML based architectures for downlink channel prediction, which can also be easily extended to other communication problems like beam prediction. Simulation results demonstrate that the proposed DML based architectures can effectively exploit the constructive and complementary information of multimodal sensory data, thus achieving better performance than existing works.

Existing work in intelligent communications has recently made preliminary attempts to utilize multi-source sensing information (MSI) to improve the system performance. However, the research on MSI aided intelligent communications has not yet explored how to integrate and fuse the multimodal sensory data, which motivates us to develop a systematic framework for wireless communications based on deep multimodal learning (DML). In this paper, we first present complete descriptions and heuristic understandings on the framework of DML based wireless communications, where core design choices are analyzed in the view of communications. Then, we develop several DML based architectures for channel prediction in massive multiple-input multiple-output (MIMO) systems that leverage various modality combinations and fusion levels. The case study of massive MIMO channel prediction offers an important example that can be followed in developing other DML based communication technologies. Simulation results demonstrate that the proposed DML framework can effectively exploit the constructive and complementary information of multimodal sensory data to assist the current wireless communications.

Source number detection is a critical problem in array signal processing. Conventional model-driven methods e.g., Akaikes information criterion and minimum description length, suffer from severe performance degradation when the number of samples is small or the signal-to-noise ratio is low. In this letter, we exploit the model-aided based deep neural network to estimate the source number. Specifically, we propose two eigenvalue based networks, i.e., a regression network (ERNet) and a classification network (ECNet), for source number detection, where the eigenvalues of the received signal covariance matrix and the source number are used as the input and the label of the networks, respectively. Furthermore, ERNet and ECNet can be easily generalized to handle coherent sources by adopting, e.g., the forward-backward spatial smoothing technique. Numerical results are included to showcase the remarkable improvements of ERNet and ECNet over the existing methods.

Artificial intelligence (AI) based downlink channel state information (CSI) prediction for frequency division duplexing (FDD) massive multiple-input multiple-output (MIMO) systems has attracted growing attention recently. However, existing works focus on the downlink CSI prediction for the users under a given environment and is hard to adapt to users in new environment especially when labeled data is limited. To address this issue, we formulate the downlink channel prediction as a deep transfer learning (DTL) problem, and propose the direct-transfer algorithm based on the fully-connected neural network architecture, where the network is trained in the manner of classical deep learning and is then fine-tuned for new environments. To further improve the transfer efficiency, we propose the meta-learning algorithm that trains the network by alternating inner-task and across-task updates and then adapts to a new environment with a small number of labeled data. Simulation results show that the direct-transfer algorithm achieves better performance than the deep learning algorithm, which implies that the transfer learning benefits the downlink channel prediction in new environments. Moreover, the meta-learning algorithm significantly outperforms the direct-transfer algorithm, which validates its effectiveness and superiority.

Recently, orbital angular momentum (OAM) based vorticose communication has attracted much attention because of its potential to significantly increase the spectrum efficiency (SE) of wireless communications. However, the multiple radio frequency (RF) chains used for multiple OAM modes lead to an unexpected cost for wireless vorticose communications. To reduce the high cost of RF chains for multiple OAM modes and the high complexity required for signal processing, we first propose the mode modulation (MM) based OAM system to allow multiple OAM modes sharing a common RF chain, which can not only reduce the hardware cost, but also boost the SE by introducing the mode as an additional dimension for data transmission. To solve the problem of how to maximize the SE of MM based OAM systems with the limited RF chains, we develop the equal-probability mode modulation (EMM) scheme, where the OAM modes are selected with equal probability and the signal is transmitted though the activated OAM modes. Moreover, we develop the Huffman coding based adaptive mode modulation (AMM) scheme, which can adaptively choose the OAM modes to further increase the SEs of OAM based vorticose communications. We also develop the OAM- water-filling power allocation policies for both EMM and AMM schemes to achieve the maximum SEs for OAM based vorticose communications. Numerical results are presented to show that the MM can offer the mode dimension for vorticose communications and the AMM scheme can achieve larger SE than the EMM scheme. Also, our developed power allocation policies can further increase the SEs for the MM based OAM communications.