Decentralized learning (DL) has recently employed local updates to reduce the communication cost for general non-convex optimization problems. Specifically, local updates require each node to perform multiple update steps on the parameters of the local model before communicating with others. However, most existing methods could be highly sensitive to data heterogeneity (i.e., non-iid data distribution) and adversely affected by the stochastic gradient noise. In this paper, we propose DSE-MVR to address these problems. Specifically, DSE-MVR introduces a dual-slow estimation strategy that utilizes the gradient tracking technique to estimate the global accumulated update direction for handling the data heterogeneity problem; also for stochastic noise, the method uses the mini-batch momentum-based variance-reduction technique. We theoretically prove that DSE-MVR can achieve optimal convergence results for general non-convex optimization in both iid and non-iid data distribution settings. In particular, the leading terms in the convergence rates derived by DSE-MVR are independent of the stochastic noise for large-batches or large partial average intervals (i.e., the number of local update steps). Further, we put forward DSE-SGD and theoretically justify the importance of the dual-slow estimation strategy in the data heterogeneity setting. Finally, we conduct extensive experiments to show the superiority of DSE-MVR against other state-of-the-art approaches. We provide our code here: https://anonymous.4open.science/r/DSE-MVR-32B8/.

In this paper, the problem of semi-distributed resource allocation is considered in an orthogonal frequency division multiple access (OFDMA) downlink transmission of wireless relay networks, where subcarriers are allocated to the relay stations at the base station (BS) only with a small part of channel state information (CSI), and afterwards subcarrier and power are allocated to the users at the relay stations with perfect CSI. A high-efficiency semi-distributed algorithm of linear complexity is proposed to address the joint resource allocation, which observably reduces the computational complexity of the subcarrier allocation at the BS and alleviates the uplink transmission of CSI feedback from the relay stations. It is shown through numerical evaluation that the performance of the proposed semi-distributed algorithm is close to the optimal joint subcarrier and power allocation, which is of exponential complexity and requires entire CSI at the BS.

In this paper, the error rate performance and diversity gain of amplify-and-forward (AF) wireless relay networks are studied. The distributed space-time block code (DSTBC) in a synchronous relay network with multiple relay nodes is considered. The distributed linear dispersion code is used at the relays. The symbol (or codeword) error rate is studied in terms of upper and lower bounds through rigorous derivations, and the diversity gain of the DSTBC is derived, for any number of relay nodes. As a corollary, a necessary and sufficient condition is obtained for achieving full cooperative diversity. Arbitrary coding matrices at the relays, path loss and general linear power allocation are considered.

In this paper, the error rate performance and diversity gain of amplify-and-forward (AF) wireless relay networks are studied. The distributed space-time block code (DSTBC) in a synchronous relay network with multiple relay nodes is considered. The distributed linear dispersion code is used at the relays. The symbol (or codeword) error rate is studied in terms of upper and lower bounds through rigorous derivations, and the diversity gain of the DSTBC is derived, for any number of relay nodes. As a corollary, a necessary and sufficient condition is obtained for achieving full cooperative diversity. Arbitrary coding matrices at the relays, path loss and general linear power allocation are considered. Finally, the orthogonal frequency division multiplexing (OFDM)-based space-time transmission scheme in asynchronous cooperative networks is also discussed.

In this paper, the error performances and diversity gains of two wireless relay networks are studied. The distributed space-time block coding in a synchronous relay network with multiple relay nodes is considered. The distributed linear dispersion code is used at the relays. The symbol (or codeword) error rate is studied in terms of upper and lower bounds through rigorous derivations, and sufficient conditions the system achieves full cooperative in two cases without and with a direct path from the source to the destination are also given. Different from the existing works, here the strict proof of the diversity gain through analyzing upper and lower bounds of SER performance is given. Finally, the orthogonal frequency division multiplexing (OFDM)-based space-time transmission scheme in asynchronous cooperative networks is also discussed.