This study demonstrates the feasibility of proactive received power prediction by leveraging spatiotemporal visual sensing information towards reliable millimeter-wave (mmWave) networks. As the received power on a mmWave link can attenuate aperiodically owing to human blockages, a long-term series of the future received power cannot be predicted by analyzing the received signals prior to the blockage occurring. We propose a novel mechanism that predicts the time series of received power from the next moment to as many as several hundred milliseconds ahead. The key idea is to leverage camera imagery and machine learning (ML). Time-sequential images may involve the spatial geometry and mobility of obstacles representing mmWave signal propagation. ML is used to construct a prediction model from a dataset of sequential images labeled with received power in several hundred milliseconds ahead of the time at which each image is obtained. The simulation and experimental evaluations conducted using IEEE 802.11ad devices and a depth camera demonstrated that the proposed mechanism employing convolutional long short-term memory predicted a time series of received power up to 500 ms ahead, with an inference time of less than 3 ms and a root-mean-square error of 3.4 dB.

Camera-assisted millimeter-wave (mmWave) network is a new paradigm for mmWave communications where mobility of obstacles is captured by using RGB and depth cameras and conducts network operations by considering the captured information. For camera-assisted mmWave networks, this paper proposes a recurrent neural network (RNN)-based received signal strength (RSS) estimation scheme using depth camera images. This scheme enables us to estimate the RSS of any mmWave links, including links where nodes are not transmitting frames. An RNN enables us to model the relationship between current RSS and an image time series, which includes information regarding the mobility of nodes and obstacles. Simulation results demonstrate that the RNN-based estimation scheme achieves higher accuracy than that of a multi-layer perceptron.

This paper experimentally finds the optimum number of input images of a machine learning-based mmWave received signal strength (RSS) value prediction scheme from depth images. By modeling the relationships between time-sequential depth images and RSS values based on machine learning, it is possible to predict the future RSS values, and thereby, a predictive handover makes a moment of degradation of the RSS value avoidable. As prediction models of RSS value, three machine learning models are compared: the convolutional neural networks (CNN), the combination of CNN and convolutional long short-term memory (CNN+ConvLSTM), and random forest. As the number of input images increases, the prediction accuracy generally improves, however, too numerous input images may make the prediction accuracy worse because of over-fitting. Experimental results reveal that the number of input images that are input in order to predict the RSS value the most accurately is 16.

The human blockage problem is an open issue in next- generation wireless access networks using millimeter-wave (mmWave) communications. A proactive base station (BS) handover system leveraging RGB and depth (RGB-D) cameras is proposed for solving the human blockage problem. RGB-D cameras observe mmWave communication ranges, and BS handover is conducted proactively before a human blockage causes serious performance degradation. However, this system must rely on a scheme that provides a guideline for selecting a BS to which the transfer can be done. In this paper, we propose a mmWave throughput estimation scheme using an online machine learning algorithm and depth images obtained by the RGB-D camera. The algorithm learns the relationship between depth images and measured throughputs, and estimates throughput from depth images. The scheme enables the handover system to estimate throughput quickly and adaptively to the wireless environment without transmitting any control frames. We conducted a proof-of-concept experiment by using a testbed consisting of IEEE 802.11ad mmWave wireless local area network devices and an RGB-D camera. The experiment confirmed that the proposed scheme estimates throughput from a depth image with an RMS error of 114-178 Mbit/s in real time.