Beam codebooks are integral components of future millimeter wave MIMO systems. Therefore, it is critical to optimize these codebooks for efficient and reliable communications. Prior work has focused on single-cell codebook learning problems and under stationary interference. In this work, we generalize the interference-aware codebook learning problem to networks with multiple cells/basestations. One of the key differences is that the underlying environment becomes non-stationary, as the behavior of one basestation may influence the learning of the others. Further, we avoid information exchange between different learning nodes which leads to a fully decentralized system with increased learning difficulties. To tackle the non-stationarity, the averaging of measurements is used to estimate the interference nulling performance of a particular beam, based on which a decision rule is provided. Furthermore, we theoretically justify the adoption of such estimator, and prove that it is a sufficient statistic for the underlying quantity of interest in an asymptotic sense. Finally, a novel reward function is proposed to decouple the learning of the multiple agents running at different nodes. Results show that the developed solution is capable of learning well-shaped codebook patterns for different networks and significantly suppress the interference without requiring any information exchange between basestations.

Noteworthy strides continue to be made in the development of full-duplex millimeter wave (mmWave) communication systems, but most of this progress has been built on theoretical models and validated through simulation. In this work, we conduct a long overdue real-world evaluation of full-duplex mmWave systems using off-the-shelf 60 GHz phased arrays. We collect over 200,000 measurements of self-interference by electronically sweeping the transmit and receive beams of an experimental base station across a dense spatial profile, shedding light on the effects of the environment, array positioning, and beam steering direction. Then, we call attention to five key challenges faced by practical full-duplex mmWave systems and, with these in mind, propose a general framework for beamforming-based full-duplex solutions. Guided by this framework, we introduce a novel solution called STEER+, a more robust version of recent work called STEER, and experimentally evaluate both in a real-world setting with actual downlink and uplink users. Rather than purely minimize self-interference as with STEER, STEER+ makes use of additional measurements to maximize spectral efficiency, which proves to make it much less sensitive to one’s choice of design parameters. Experimentally, we demonstrate that STEER+ can reliably reduce self-interference to near or below the noise floor while maintaining high SNR on the downlink and uplink, thus enabling full-duplex operation purely via beamforming.

Employing large antenna arrays is a key characteristic of millimeter wave (mmWave) and terahertz communication systems. Due to the hardware constraints and the lack of channel knowledge, codebook based beamforming/combining is normally adopted to achieve the desired array gain. However, most of the existing codebooks focus only on improving the gain of their target user, without taking interference into account. This can incur critical performance degradation in dense networks. In this paper, we propose a sample-efficient online reinforcement learning based beam pattern design algorithm that learns how to shape the beam pattern to null the interfering directions. The proposed approach does not require any explicit channel knowledge or any coordination with the interferers. Simulation results show that the developed solution is capable of learning well-shaped beam patterns that significantly suppress the interference while sacrificing tolerable beamforming/combing gain from the desired user. Furthermore, a hardware proof-of-concept prototype based on mmWave phased arrays is built and used to implement and evaluate the developed online beam learning solutions in realistic scenarios. The learned beam patterns, measured in an anechoic chamber, show the performance gains of the developed framework and highlight a promising machine learning based beam/codebook optimization direction for mmWave and terahertz systems.

Millimeter wave (mmWave) and massive MIMO systems are intrinsic components of 5G and beyond. These systems rely on using beamforming codebooks for both initial access and data transmission. Current beam codebooks, however, generally consist of a large number of narrow beams that scan all possible directions, leading to large training overhead. Further, these codebooks do not normally account for hardware impairments or possible non-uniform array geometries, and their calibration process is expensive. To overcome these limitations, this paper develops an efficient online machine learning framework that learns how to adapt the codebook beam patterns to the specific deployment, surrounding environment, user distribution, and hardware characteristics. This is done by designing a novelcomplex-valued neural networkarchitecture in which the neuron weights directly model the beamforming weights of the analog phase shifters, accounting for the key hardware constraints. This model learns the codebook beams through online and self-supervised training avoiding the need for explicit channel state information. This respects the practical situations where the channel is imperfect or hard to obtain. Simulation results highlight the capability of the proposed solution in learning environment and hardware aware beam codebooks, which reduce the training overhead and improve the robustness against possible hardware impairments.

Millimeter wave (mmWave) and terahertz MIMO systems rely on pre-defined beamforming codebooks for both initial access and data transmission. These pre-defined codebooks, however, are commonly not optimized for specific environments, user distributions, and/or possible hardware impairments. This leads to large codebook sizes with high beam training overhead which makes it hard for these systems to support highly mobile applications. To overcome these limitations, this paper develops a deep reinforcement learning framework that learns how to optimize the codebook beam patterns relying only on the receive power measurements. The developed model learns how to adapt the beam patterns based on the surrounding environment, user distribution, hardware impairments, and array geometry. Further, this approach does not require any knowledge about the channel, RF hardware, or user positions. To reduce the learning time, the proposed model designs a novelWolpertinger-variant architecture that is capable of efficiently searching the large discrete action space. The proposed learning framework respects the RF hardware constraints such as the constant-modulus and quantized phase shifter constraints. Simulation results confirm the ability of the developed framework to learn near-optimal beam patterns for line-of-sight (LOS), non-LOS (NLOS), mixed LOS/NLOS scenarios and for arrays with hardware impairments without requiring any channel knowledge.