We consider the problem of autonomous channel access (AutoCA), where a group of terminals tries to discover a communication strategy with an access point (AP) via a common wireless channel in a distributed fashion. Due to the irregular topology and the limited communication range of terminals, a practical challenge for AutoCA is the hidden terminal problem, which is notorious in wireless networks for deteriorating throughput and delay performances. To meet the challenge, this paper presents a new multi-agent deep reinforcement learning paradigm, dubbed MADRL-HT, tailored for AutoCA in the presence of hidden terminals. MADRL-HT exploits topological insights and transforms the observation space of each terminal into a scalable form independent of the number of terminals. To compensate for the partial observability, we put forth a look-back mechanism such that the terminals can infer behaviors of their hidden terminals from the carrier-sensed channel states as well as feedback from the AP. A window-based global reward function is proposed, whereby the terminals are instructed to maximize the system throughput while balancing the terminals' transmission opportunities over the course of learning. Considering short-packet machine-type communications, extensive numerical experiments verified the superior performance of our solution benchmarked against the legacy carrier-sense multiple access with collision avoidance (CSMA/CA) protocol.

The existing medium access control (MAC) protocol of Wi-Fi networks (i.e., carrier-sense multiple access with collision avoidance (CSMA/CA)) suffers from poor performance in dense deployments due to the increasing number of collisions and long average backoff time in such scenarios. To tackle this issue, we propose an intelligent wireless MAC protocol based on deep learning (DL), referred to as DL-MAC, which significantly improves the spectrum efficiency of Wi-Fi networks. The goal of DL-MAC is to enable not only intelligent channel access but also intelligent rate adaptation. To achieve this goal, we design a deep neural network (DNN) that takes the historical received signal strength indications (RSSIs) as inputs and outputs joint channel access and rate adaptation decision. Notably, the proposed DLMAC takes the constraints of practical applications into account and the DL-MAC is evaluated using the experimental wireless data sampled from the actual environments on the 2. 4GHz frequency band. The experimental results show that our DLMAC can achieve around 86% performance of the global optimal MAC, and about twice the performance of the traditional Wi-Fi MAC in the environments of our lab and the Shenzhen Baoan International Airport departure hall.

In the next generation wireless networks, more applications will emerge, covering virtual reality movies, augmented reality, holographic three-dimensional telepresence, haptic telemedicine and so on, which require the provisioning of high bandwidth efficiency and low latency services. In order to better support the aforementioned applications and services, novel distributed channel access (DCA) schemes are necessary. Therefore, we propose a new MAC protocol, QMIX-advanced Listen-Before-Talk (QLBT), based on the cutting-edge multi-agent reinforcement learning (MARL) algorithm. It employs a centralized training with decentralized execution (CTDE) framework to exploit the overall information of all agents during training, and ensure that each agent can independently infer the optimal channel access behavior based on its local observation. We enhance QMIX, a well-known MARL algorithm, by introducing an extra individual Q-value for each agent in the mixing network apart from the original total Q-value, which makes QLBT more stable. Moreover, delay to last successful transmission (D2LT) is first introduced in this work as a part of the observations of each QLBT agent, which facilitates agents to reach a cooperative policy that prioritizes the agent with the longest delay. Finally, extensive simulation experiments are provided to show that the proposed QLBT algorithm: 1) outperforms CSMA/CA and even its theoretical performance bound in various scenarios including saturated traffic, unsaturated traffic and delay-sensitive traffic; 2) is robust in dynamic environment; and 3) is able to friendly coexist with “legacy” CSMA/CA stations.

In this work, we propose an artificial intelligence (AI) based channel quality assessment algorithm for Bluetooth adaptive frequency hopping (AFH) to avoid interference between heterogeneous systems coexist in 2.4GHz Industrial, Scientific, and Medical (ISM) band. The proposed network takes the received signal strength indicator (RSSI) of all Bluetooth channels as input and outputs the estimated channel quality, which is further used to update AFH channel map. A gated recurrent unit (GRU) is adopted to extract the temporal information of interference on each channel. A novel loss function combining classification loss and ranking loss is designed to improve the performance of the neural network. The complexity analysis shows that the network considered is lightweight and resource-friendly. Moreover, simulation experiments under different interference models show that the proposed method outperforms several existing channel selection schemes.