Federated learning (FL) is increasingly adopted to combine knowledge from clients in training without revealing their private data. In order to improve the performance of different participants, personalized FL has recently been proposed. However, considering the non-independent and identically distributed (non-IID) data and limited bandwidth at clients, the model performance could be compromised. In reality, clients near each other often tend to have similar data distributions. In this work, we train the personalized edge-based model in the client-edge-server FL. While considering the differences in data distribution, we fully utilize the limited bandwidth resources. To make training efficient and accurate at the same time, An intuitive idea is to learn as much useful knowledge as possible from other edges and reduce the accuracy loss incurred by non-IID data. Therefore, we devise Grouping Hierarchical Personalized Federated Learning (GHPFL). In this framework, each edge establishes physical connections with multiple clients, while the server physically connects with edges. It clusters edges into groups and establishes client-edge logical connections for synchronization. This is based on data similarities that the nodes actively identify, as well as the underlying physical topology. We perform a large-scale evaluation to demonstrate GHPFL’s benefits over other schemes.

Intrusion Detection Systems (IDS) integrated with Machine Learning (ML) techniques have proven to be effective defenses against the increasing cybersecurity attacks in the Artificial Intelligence of Things (AIoT) domain. Privacy concerns have prompted the emergence of Federated Learning (FL) as a promising solution for AIoT intrusion detection. Despite their potential, FL-based IDSs still face challenges related to class-imbalanced data and Non-Independent and Identically Distributed (non-IID) data among AIoT devices. These challenges hinder FL from learning meaningful features from the data, thus impeding the convergence of the learning process. To tackle these issues, this paper proposes a Clustering-enabled Federated Meta-Training (CFMT) framework for AIoT intrusion detection. The proposed CFMT framework effectively addresses the negative impact of imbalanced and non-IID data. Specifically, we design a data-and model-agnostic meta-sampler that adaptively balances local datasets, thereby mitigating the data imbalance problem. Additionally, we propose a dynamic clustering algorithm that selectively eliminates the local models affected by the training state bias caused by non-IID data, thereby addressing the non-IID data issue. Extensive case studies on two real-world datasets demonstrate the superior performance of the proposed CFMT framework compared to existing solutions, including federated non-IID algorithms and federated imbalanced learning algorithms, in terms of IDS performance. Our code and data are available at https://gitee.com/mindspore/models/tree/master/research/cv/HDFL-IDS-Meta.

In federated learning (FL), clients may have diverse objectives, and merging all clients' knowledge into one global model will cause negative transfer to local performance. Thus, clustered FL is proposed to group similar clients into clusters and maintain several global models. In the literature, centralized clustered FL algorithms require the assumption of the number of clusters and hence are not effective enough to explore the latent relationships among clients. In this paper, without assuming the number of clusters, we propose a peer-to-peer (P2P) FL algorithm namedPANM. InPANM, clients communicate with peers to adaptively form an effective clustered topology. Specifically, we present two novel metrics for measuring client similarity and a two-stage neighbor matching algorithm based Monte Carlo method and Expectation Maximization under the Gaussian Mixture Model assumption. We have conducted theoretical analyses ofPANMon the probability of neighbor estimation and the error gap to the clustered optimum. We have also implemented extensive experiments under both synthetic and real-world clustered heterogeneity. Theoretical analysis and empirical experiments show that the proposed algorithm is superior to the P2P FL counterparts, and it achieves better performance than the centralized cluster FL method.PANMis effective even under extremely low communication budgets.

Consider the problem of out-of-distribution prediction given data from multiple environments. While a sufficiently diverse collection of training environments will facilitate the identification of an invariant predictor, with an optimal generalization performance, many applications only provide us with a limited number of environments. It is thus necessary to consider adapting to distribution shift using a handful of labeled test samples. We propose a constrained Bayesian approach for this task, which restricts to models with a worst-group training loss above a prespecified threshold. Our method avoids a pathology of the standard Bayesian posterior, which occurs when spurious correlations improve in-distribution prediction. We also show that on certain high-dimensional linear problems, constrained modeling improves the sample efficiency of adaptation. Synthetic and real-world experiments demonstrate the robust performance of our approach.

Federated learning (FL) has recently emerged as an innovative paradigm to train models among distributed agents. Conventional FL considers the center as an aggregator and trains from distributed data, while the collected global information at the center is not effectively utilized. Thus, the restricted information from local observations may limit the model accuracy. If FL can introduce data sets from the network server, the distributed models may be largely improved by the extra global information. Since network agents may not be completely trusted, the center cannot directly broadcast its raw data for security concern. Then, how to combine the central sets with FL? In this article, we propose to add a learning model at the center, which obtains the central sets as input. The outputs can be transmitted to network agents and integrated into local models instead of the raw data. The central and local models could be trained to form an integration for intelligent inference. Then, what is the integrated performance gain comparing with the original horizontal FL (HFL) and how to implement it? To figure out these two problems, we propose the vertical-HFL (VHFL) scheme, where models of the center and agents are trained collaboratively. We further analyze its convergence and the related communication channel, proposing the theoretical bounds to guide the network implementation of VHFL. Some simulation results will demonstrate the effectiveness of our proposed VHFL scheme. It is expected that VHFL will be an important block for the next generation of smart services.

Fault diagnosis in an open world refers to the diagnosis tasks that need to cope with previously unknown faults in the online stage. It faces a great challenge yet to be addressed—that is, the online data of unknown faults may be classified as normal samples with a high probability. In this article, we develop an effective solution for this challenge by using supervised contrastive learning to learn a discriminative and compact embedding for the known normal situation and fault situations. Specifically, in addition to contrasting a given sample with other instances as is the case in conventional contrastive learning methods, our training scheme contrasts the normal samples with negative augmentations of themselves. The negative out-of-distribution data is generated by the Soft Brownian Offset sampling method to simulate the previously unknown faults. Computational experiments are conducted on the Tennessee Eastman Process benchmark dataset and a practical plasma etching process dataset. The proposed method achieves significant improvement compared with four existing methods under three open-set fault diagnosis circumstances, i.e., balanced open-set fault diagnosis, imbalanced fault diagnosis, and few-shot fault diagnosis. This demonstrates its great potentials in real world fault diagnosis applications.

Intelligent fault diagnosis is essential to guarantee the safe operation of industrial processes. And an important issue is how to develop a method to tackle the dilemma where we can only collect limited fault samples. In this paper, we propose a two-stage method based on supervised contrastive learning for imbalanced fault diagnosis tasks. We utilize the supervised contrastive learning technique as it has shown a powerful representation learning ability in previous works. The computational experiments on the Tennessee Eastman dataset show that our proposed two-stage method can achieve improved performance when compared to existing methods.

Federated learning (FL) has recently emerged as a transformative paradigm that jointly train a model with distributed devices while avoiding the need for central data collection. Due to the limited observation range, the devices only contain local information, which limits the quality of trained models. In this case, combining the global information into FL may be helpful. However, in horizontal FL, the central agency only acts as a model aggregator without utilizing its global observation. Meanwhile, the global data may not be directly transmitted to agents for data security. Then how to utilize the global observation residing in the central agency while protecting its safety thus rises up as an important problem in FL. In this paper, we develop a vertical-horizontal federated learning (VHFL) scheme, where the global feature is shared with the agents in a procedure similar to that of vertical FL. It is shown by experiments that the proposed VHFL could enhance the accuracy compared with horizontal FL while protecting the central data from being announced.

Recently, federated learning (FL) has been treated as an important and promising learning scheme in IoT, enabling devices to jointly learn a model without sharing their data sets. Different from centralized training on some collected data sets, FL training suffers a lot of constraints from limited resources in the network. Therein, the bandwidth and package loss restrict interactions in training. Meanwhile, the highly distributed data sets and limited computation could also affect its convergence. To figure out the specific impact, we analyze the convergence rate of FL training considering both communication and training. Further taking in training costs in terms of time and power, the closed-form optimal settings for communication networks are proposed with principles to assist the parameter selection. The results build a bridge between AI and communication, giving us an intuitive knowledge of how the background system could influence the distributed training process.

Federated learning (FL) has recently emerged as an important and promising learning scheme in IoT, enabling devices to jointly learn a model without sharing their raw data sets. As FL does not collect and store the data centrally, it requires frequent model exchange through the wireless network. However, since the aggregation in FL can be partially participated with synchronized frequency, its communication pattern is different from the conventional network. Therein, limited bandwidth and package loss restrict interactions in training. Thus, the network scheduling could largely affect the FL convergence. To figure out the specific effects, we analyze the convergence rate of FL regarding the joint impact of communication and training. Combining it with the network model, we formulate the optimal scheduling problem for FL implementation. The theoretical results could guide the hyper-parameter design in the network and explain the principle of how the wireless communication could influence the FL training process.

Heavy data load and wide cover range have always been crucial problems for big data processing in Internet of Things (IoT). Recently, mobile-edge computing (MEC) and unmanned aerial vehicle base stations (UAV-BSs) have emerged as promising techniques in IoT. In this article, we propose a three-layer online data processing network based on the MEC technique. On the bottom layer, raw data are generated by distributed sensors with local information. Upon them, UAV-BSs are deployed as moving MEC servers, which collect data and conduct initial steps of data processing. On top of them, a center cloud receives processed results and conducts further evaluation. For online processing requirements, the edge nodes should stabilize delay to ensure data freshness. Furthermore, limited onboard energy poses constraints to edge processing capability. In this article, we propose an online edge processing scheduling algorithm based on Lyapunov optimization. In cases of low data rate, it tends to reduce edge processor frequency for saving energy. In the presence of a high data rate, it will smartly allocate bandwidth for edge data offloading. Meanwhile, hovering UAV-BSs bring a large and flexible service coverage, which results in a path planning issue. In this article, we also consider this problem and apply deep reinforcement learning to develop an online path planning algorithm. Taking observations of around environment as an input, a CNN network is trained to predict action rewards. By simulations, we validate its effectiveness in enhancing service coverage. The result will contribute to big data processing in future IoT.

Heavy data load and wide cover range have always been crucial problems for internet of things (IoT). However, in mobile-edge computing (MEC) network, edge data can be partly processed at the edge. In this paper, a MEC-based big data analysis network is discussed, where distributed raw data are collected and processed by edge servers. The edge servers are supposed to split out a large sum of redundant data and transmit extracted information to the center cloud for further analysis. However, for consideration of the limited edge computation capability, part of the raw data may be directly transmitted to the cloud. To manage limited resources in an online manner, we propose an algorithm based on Lyapunov optimization, which jointly optimizes the policy involving edge processor frequency, transmission power and bandwidth allocation. The algorithm aims at stabilizing data processing delay while saving energy without knowing probability distributions of data sources. The proposed network management algorithm may contribute to big data processing in future IoT.

Model change detection is studied, in which there are two sets of samples that are independently and identically distributed (i.i.d.) according to a pre-change probabilistic model with parameter θ, and a post-change model with parameter θ', respectively. The goal is to detect whether the change in the model is significant, i.e., whether the difference between the pre-change parameter and the post-change parameter ∥θ - θ'∥2is larger than a pre-determined threshold ρ. The problem is considered in a Neyman-Pearson setting, where the goal is to maximize the probability of detection under a false alarm constraint. Since the generalized likelihood ratio test (GLRT) is difficult to compute in this problem, we construct an empirical difference test (EDT), which approximates the GLRT and has low computational complexity. Moreover, we provide an approximation method to set the threshold of the EDT to meet the false alarm constraint. Experiments with linear regression and logistic regression are conducted to validate the proposed algorithms.

In this paper, we focus on the mutual information, which can characterize the transmission ability because it shows correlation between channel input and channel output. Shannon entropy and mutual information are the cornerstones of information theory. In addition, Chernoff information is another fundamental channel information measure, and it describe the maximum achievable exponent of the error probability in hypothesis testing. Uased on alternating conditional expectation (ACE) algorithm, we decompose these two mutual information. In fact, their decomposition results are similar in big data prespective. In this sense, these two kinds of mutual information are just different measures of the same information quantity. This paper also deduces that the channel performance only depends on channel parameters and the decomposition results of a new proposed mutual information should agree with the impact of the parameters.

This paper considers the placement of unmanned aerial vehicle base stations (UAV-BSs) with criterion of minimum UAV-recall-frequency (UAV-RF), indicating the energy efficiency of mobile UAVs networks. Several different power consumptions, including signal transmit power, on-board circuit power and the power for UAVs mobility, and the ground user density are taken into account. Instead of conventional empirical stochastic models, this paper utilizes a pattern formation system to track the instable and non-ergodic time-varying nature of user density. We show that for a single time-slot, the optimal placement is achieved when the transmit power of UAV-BSs equals their on-board circuit power. Then, for multiple time-slot duration, we prove that the optimal placement updating problem is an integer nonlinear programming coupled with an inherent integer linear programming. Since the original problem is NP-hard and cannot be solved with conventional recursive methods, we propose a sequential-Markov-greedy-decision strategy to achieve near minimal UAV-RF in polynomial time. Furthermore, we prove that the increment of UAV-RF caused by inaccurate predicted user density is proportional to the generalization error of learned patterns. Here, in regions with large area, high-rise buildings, or low user density, large sample sets are required for effective pattern formation.

This paper mainly considers the compressed storage problem for big data in wireless communication systems, where the message importance is taken into account. Similar to Shannon Entropy and Renyi Entropy, we first define a non-parametric message important measure (NMIM) as a measure for message importance. It can characterize the uncertainty of random events. It is proved that it can sufficiently describe the two key characters of big data: rare events finding and large diversities of events. Based on NMIM, we propose an effective compressed encoding mode for data storage in wireless communication systems. Numerical simulation results show that using our developed strategy takes up very little storage space without losing too much message importance.

In the downlink of high-speed railway communication systems equipped with distributed transmit antennas, high mobility leads to fast time-varying received signal to noise ratios. In this case, dynamic time-domain power allocation and antenna selection could be jointly optimized to improve the system energy efficiency. This paper considers this problem in such a simple way where dynamic switching between multiple-input-multiple-output and single-input-multiple-output is allowed and exclusively utilized, while the sparse scattering terrains and delay-sensitive traffic flows are taken into account. The original optimization problem is a typical mixed integer nonlinear programming (MINLP) problem. Instead of conventional iteration based methods, such as the extended cutting plane method, we propose a low-complexity and direct solution by exploiting the physical nature of original problem, which can be utilized in real time. Theoretical results show that our proposed method can be viewed as the generalization of channel-inversion associated with transmit antenna selection. Also, compared with methods without dynamic antenna selection, our method significantly decreases the average transmit power.

In this paper, we present a practical simple location-aided umbrella-shaped beamforming scheme with transmit diversity of massive Multiple-input Multiple-output (MIMO) system for high speed railway scenarios. Unlike conventional schemes which combines space-time block coding (STBC) with adaptive beamforming or orthogonal switched beamforming, our scheme needs neither uplink channel covariance matrix (UCCM) nor downlink CCM (DCCM) but precalculates the beamforming weights with the help of train location information, which can be completed through pure off-line calculation and therefore reduce system implementation complexity. A closed-form solution of power allocation optimization is derived and the performance of our scheme is verified with simulations from the perspectives of instantaneous received signal-to- noise ratio (SNR), bit error rate (BER) and handover success probability. It indicates that the performance of our scheme approaches to the combination scheme of STBC and adaptive beamforming (STBC-ABF) without introducing any on-line system complexities.

High mobility may destroy the orthogonality of subcarriers in OFDM systems, resulting in inter-carrier interference (ICI), which may greatly reduce the service quantity of high speed railway (HSR) wireless communications. This paper focuses on ICI mitigation in the HSR downlinks with distributed transmit antennas. In such a system, its key feature is that the ICIs are caused by multiple carrier frequency offsets corresponding to multiple transmit antennas. Meanwhile, the channel of HSR is fast time varying, which is another big challenge in the system design. In order to get a good performance, low complexity real-time ICI reduction is necessary. To this end, we first analyzed the property of the ICI matrix and then propose a low complexity ICI reduction method based on location information. For evaluating the effectiveness of the proposed method, the maximum and minimum remaining interference after ICI reduction is analyzed and the service quantity is also discussed. Numerical results are presented to verify our theoretical analysis and the effectiveness of the proposed ICI reduction method. One important observation is that our proposed ICI mitigation method can achieve almost the same service quantity with that obtained on the case without ICI when the velocity of the train is 300km/h.

Abstract Multiple‐Input, Multiple‐Output (MIMO)‐orthogonal frequency division multiplexing (OFDM) is a promising technique in 5G wireless communications. In high‐mobility scenarios, the transmission environments are time‐varying and/or the relative moving velocity between the transmitter and receiver is also time‐varying. In the literature, most of previous works mainly focused on fixed subcarrier group size and precoded the MIMO signals with unitary channel state information. In this way, the subcarrier grouping may naturally lead to big loss of channel capacity in high‐mobility scenarios because of the channel state information difference on the subcarriers in each group. To employ the MIMO‐OFDM technique, adaptive subcarrier grouping scheme may be an efficient way. In this paper, we first consider MIMO‐OFDM systems over double‐selective i.i.d. Rayleigh channels and investigate the quantitative relation between subcarrier group size and capacity loss theoretically. With developed theoretical results, we also propose an adaptive subcarrier grouping scheme to satisfy the preset capacity loss threshold by adjusting grouping size with the sensed environmental information and mobile velocity. Theoretical analysis and simulation results show that to achieve a better system capacity, a sparse scattering, lower signal‐to‐noise ratio, and lower velocity as well as properly large antenna number are matched with larger subcarrier group size. One important observation is that if the antenna number is too large and higher than a threshold, which will not bring any additional gain to the subcarrier grouping. That is, the system capacity loss will converge to a lower bound expeditiously with respect to antenna number, which is given in theory also. Copyright © 2016 John Wiley & Sons, Ltd.