Recent efforts have been made to integrate self-supervised learning (SSL) with the framework of federated learning (FL). One unique challenge of federated self-supervised learning (FedSSL) is that the global objective of FedSSL usually does not equal the weighted sum of local SSL objectives. Consequently, conventional approaches, such as federated averaging (FedAvg), fail to precisely minimize the FedSSL global objective, often resulting in suboptimal performance, especially when data is non-i.i.d.. To fill this gap, we propose a provable FedSSL algorithm, named FedSC, based on the spectral contrastive objective. In FedSC, clients share correlation matrices of data representations in addition to model weights periodically, which enables inter-client contrast of data samples in addition to intra-client contrast and contraction, resulting in improved quality of data representations. Differential privacy (DP) protection is deployed to control the additional privacy leakage on local datasets when correlation matrices are shared. We provide theoretical analysis on convergence and extra privacy leakage, and conduct numerical experiments to justify the effectiveness of our proposed algorithm.

Error correcting output code (ECOC) is a classic method that encodes binary classifiers to tackle the multi-class classification problem in decision trees and neural networks. Among ECOCs, the one-hot code has become the default choice in modern deep neural networks (DNNs) due to its simplicity in decision making. However, it suffers from a significant limitation in its ability to achieve high robust accuracy, particularly in the presence of weight errors. While recent studies have experimentally demonstrated that the non-one-hot ECOCs with multi-bits error correction ability, could be a better solution, there is a notable absence of theoretical foundations that can elucidate the relationship between codeword design, weight-error magnitude, and network characteristics, so as to provide robustness guarantees. This work is positioned to bridge this gap through the lens of neural tangent kernel (NTK). We have two important theoretical findings: 1) In clean models (without weight errors), utilizing one-hot code and non-one-hot ECOC is akin to altering decoding metrics from $l_2$ distance to Mahalanobis distance. 2) In non-clean models (with weight errors), if the normalized distance exceeds a threshold, then non-clean DNNs can reach the clean model's accuracy as long as the code length approaches infinity. This threshold is determined by DNN architecture (e.g. layer number, activation), weight error magnitude, and the distance between the output and the nearest codeword. Based on these findings, we further demonstrate how to practically use them to identify optimal ECOCs for simple tasks (short-code ECOCs) and complex tasks (long-code ECOCs), by balancing the code orthogonality (as per finding 1) and code distance (as per finding 2). Extensive experimental results across four datasets and four DNN models validate the superior performance of constructed codes, guided by our findings, compared to existing ECOCs. To our best knowledge, this is the first work that provides theoretical explanations for the effectiveness of ECOCS and offers associated design guidance for optimal ECOCs specifically tailored to DNNs.

This paper investigates the transmitter and receiver beamforming (TB and RB) for over-the-air computation (AirComp) in massive multiple-inputs and multiple-outputs (MIMO) systems. First, we propose a two-phase hybrid beamforming algorithm to design TB and hybrid RB. In the first phase, we adopt a projected gradient descent with momentum (PGDM) algorithm to search for the optimal fully-digital TB matrices. Compared with the benchmarks on the mean square error (MSE) performances, PGDM can achieve up to 5 dB gain in signal-to-noise ratio (SNR) with less algorithm execution time when fully-digital RB is assumed. In the second phase, we plug the TB matrices obtained in PGDM as well as the optimal baseband RB (BBRB) matrix into the MSE objective, and adopt gradient descent to search for the optimal radio-frequency RB (RFRB) matrix. Compared with the state-of-the-arts, the proposed two-phase algorithm reduces up to 30% of the algorithm execution time and 18% of the MSE. Second, we propose a statistical TB algorithm to reduce the communication overheads, in which TB completely depends on statistical channel state information (CSI) and thus does not rely on the feedback from the base station (BS). We prove that orthonormal matrices are asymptotically optimal for statistical TB when uncorrelated Rayleigh channels are assumed and the number of receiving antennas approaches to infinity. For correlated channels, experimental results show that the proposed statistical TB can achieve about 5 dB gain in SNR compared with orthonormal matrices in terms of the MSE performance. Third, a large scale system analysis is made in this paper. As the number of the receiving antennas approaches to infinity, asymptotically optimal choices for TB and RB are provided, and upper bounds of MSE are derived in terms of the number of clients and receiving antennas. For hybrid RB, an upper bound of the squared distance between the optimal hybrid RB and the optimal fully-digital RB is also derived.

Federated learning (FL) is a popular distributed learning paradigm, in which a global model at a server learns private data of clients without data shared among clients or the server. In this paper, we consider FL over a noisy fading multiple access channel (MAC) via over-the-air computation (AirComp). Benefiting from waveform-superposition propriety of wireless signals, AirComp is able to achieve fast aggregations in FL and improve spectral efficiency. However, most of the schemes exploiting AirComp require intensive channel estimations as the demands from precoders, which results in considerable communication overheads. For this reason, we propose two novel FL schemes with statistical channel state information (FL-SCSI-A and FL-SCSI-B) to reduce the efforts required by channel estimations. In FL-SCSI-A, precoders adjust phases of transmitted signals with phases of instant channel state information (CSI), and scale transmitted signal powers based on statistical CSI. The precoder design has the following two advantages. First, phases of instant CSI can be easier to estimate than complete instant CSI. Second, since clients only need to estimate phases of their own instant CSI (instead of CSI of all clients), with channel reciprocity, this can be easily achieved by letting the server broadcast pilots to all clients. The server in FL-SCSI-A is also efficient. It only needs to estimate the sum of channel gains of all clients, which can be easily achieved by letting clients transmit pilots simultaneously. To further reduce the communication overhead, FL-SCSI-B is proposed. The precoders in FL-SCSI-B are similar to FL-SCSI-A, while the server does not require any knowledge of instant CSI, which reduces the demands of channel estimations. For both schemes, we prove that the distortion caused by the noisy fading MAC is bounded, and the convergences of the learning processes are guaranteed for strongly smooth losses with heterogeneous data assumptions. Experimental results show that the proposed schemes perform better than benchmark schemes while reducing efforts required by channel estimation.

Internet of Things (IoT) devices often transmit very short message blocks without strong error correction coding or pilots for channel equalization. When the transmitted signals encounter a severe multipath channel, the receiver is often unable to equalize the resulting inter-symbol interference (ISI) via traditional methods, leading to a high retransmission rate. This paper proposes a joint channel equalization and symbol detection scheme for this scenario by utilizing the preamble sequence as a training pilot for equalization and the relatively weak cyclic redundancy check (CRC) 8-Dallas code for error correction. By assuming a sparse channel impulse response, the proposed joint channel equalization and symbol detection scheme formulates an l1norm constrained optimization problem and solves it by a saddle-point algorithm. The proposed algorithm is applied to a set of real-world underwater animal tracking IoT data, and the results show improvement of correct message detection rate from 21.5% (direct symbol decisions) to 41.6% (equalization and decoding).

Underwater acoustic communication often suffers from extended channel impulse response (CIR) and large Doppler spread such that signals become difficult to detect and decode. For the underwater Internet-of-Things (IoT) applications, the challenges are even bigger, because the IoT devices usually transmit very short messages due to severe power constraints, and the length of pilots are often shorter than that of CIR. Therefore, conventional pilot-assisted channel estimation and equalization approaches are incapable of detecting the information data. Moreover, the existing blind channel equalization algorithms, which do not require pilots, are not able to detect information data either, because the number of transmitted symbols is too small to approximate the expected loss with empirical loss, where the loss refers to as the error of estimated signal envelopes. In this article, a new equalization and decoding algorithm is proposed for the underwater IoT devices under harsh communication environments. Inspired by the recent blind deconvolution and compressive sensing techniques, we construct an optimization problem with the objective function rewarding sparsity of the estimated CIR using l4-norm and convexify the feasible set of the problem while guarantee the same solution. Then, we develop a pruned tree search initialization method and use gradient descent to find an optimal solution efficiently. The new algorithm is first verified by simulations, which show that the proposed algorithm outperforms conventional methods, such as the linear minimum mean square error (LMMSE) equalizer and a constant modulus algorithm (CMA). The proposed algorithm, along with a practical procedure for compensating large Doppler spread and carrier frequency offset, is further employed to process the real-world underwater IoT data collected in a fish-tag project. It shows that the proposed algorithm can equalize and detect the IoT data which were corrupted by channels whose lengths are longer than that of pilots, but the existing algorithms, such as LMMSE equalizer and CMA-based blind equalizer are not able to accomplish. It also shows that the proposed algorithm can provide a very impressive improvement compared to raw detection (no equalization or decoding) and decoding-only approaches.

For massive multiple-input multiple-output (MIMO) detection, belief propagation (BP) based on graphical models has become a popular detection algorithm since it provides a good tradeoff between performance and complexity. To further lower the complexity of BP detection, an efficient BP detection based on channel hardening (BP-CH) is proposed. In this paper, the comparison in terms of both performance and complexity between proposed BP-CH and general BP is firstly investigated exhaustively. Simulation results have shown that the proposed BP-CH achieves similar performance behavior as general BP while keeping lower computational complexity. Additionally, an folded hardware architecture for proposed BPCH detector is designed to improve the implementation efficiency. Meanwhile, VLSI implementation results have verified the great advantage of BP-CH regarding hardware overhead, especially for scenarios with large system loading factor.

Polar code is increasing its popularity recently for its capacity-achieving property for B-DMCs. However, when designing decoders for polar code, it has always been an inevitable concern for us to balance the decoding performance and the hardware consumption. In this paper, we propose an approximate belief propagation (BP) decoder for polar code for the first time. By introducing the approximate computation schemes, we reduced the critical path delay (CPD) and the hardware consumption of the conventional BP decoders. Simulation results show that the proposed approximate BP decoder achieves nearly the same decoding performance as the conventional one. Advantages of the proposed decoder has been verified by FPGA implementation.

With low-density parity-check (LDPC) code and polar code selected as the standard codes for 5G eMBB scenario, one challenge is how to improve the hardware efficiency when both decoders are required by one system. Since LDPC and polar codes can be decoded with belief propagation (BP) algorithms, this similarity allows us to design a reconfigurable decoder, which can decode both codes at the cost of only one decoder. Numerical and implementation results are also given in this paper to show that the proposed decoder achieves higher hardware efficiency than stand-alone LDPC or polar decoder, without harming the error performance.

With better spectral efficiency, multiple-input and multiple-output (MIMO) systems have drawn increasing attentions. Due to its near-optimal performance, K-best algorithm has been widely adopted for MIMO detection. To the best knowledge of the authors, this paper first proposes a joint detection and decoding (JDD) method for MIMO with low-density parity-check (LDPC) codes. By pruning the searching tree of K-best detection with LDPC coding constraint, the proposed JDD scheme benefits from both reduced tree-search complexity and improved performance compared to its uncoded MIMO counterpart. Numerical results of 16-QAM MIMO with (8, 2) LDPC code and 64-QAM MIMO with (18, 6) LDPC code have shown that, the proposed JDD scheme's performance is evidently superior over separated detection and decoding (SDD) scheme. More specifically, for the latter case with 12 antennas, JDD shows nearly 10 dB performance improvement than SDD when BER = 10-3. Hardware architecture and complexity analysis are also given in this paper to demonstrate JDD's advantages.