Acoustic communications are the most exploited technology in the so-called Internet of Underwater Things (IoUT). UnderWater (UW) environments are often characterized by harsh propagation features, limited bandwidth, fast-varying channel conditions, and long propagation delay. On the other hand, IoUT nodes are usually battery-powered devices with limited processing capabilities. Accordingly, it is necessary to design optimization algorithms to address the challenging propagation features while balancing them with the limited device capabilities. To address the constraints of the nodes in energy and processing resources, it is crucial to adjust the transmission parameters based on the channel conditions while also developing communication procedures that are both lightweight and energy-efficient. In this work, we introduce a novel Multi-Player Multi-Armed Bandit (MP-MAB) framework for modulation adaptation in Multi-Hop IoUT Acoustic Networks. As opposed to widely used, computation-demanding Deep Reinforcement Learning (DRL) techniques, MP-MAB algorithms are simple and lightweight and allow to iteratively make decisions by selecting one among multiple choices, or arms. The framework is fully-distributed and is able to dynamically select the best modulation technique at each IoUT node by leveraging on high-level statistics (e.g., network throughput), without the need to exploit hard-to-extract channel features (e.g., channel state). We evaluate the performance of the proposed framework using the DESERT UW simulator and compare it with state-of-the-art centralized solutions based on Deep Reinforcement Learning (DRL) for cognitive and heterogeneous networks, namely DRL-MCS, DRL-AM, PPO, SAC, as well as with a multiple-agent, distributed version of the PPO. The results highlight that, despite its simplicity and fully-distributed nature, the proposed framework achieves superior performance in UW networks in terms of throughput, convergence speed, and energy efficiency. Compared to DRL-MCS and DRL-AM, our approach improves network throughput by up to 33% and 20%, respectively, and reduces energy consumption by up to 18% and 16%. When compared to PPO, SAC, and Multi-PPO, the proposed solution achieves up to 11%, 34%, and 38% higher throughput, and up to 7%, 17%, and 33% lower energy consumption, respectively.

Underwater Acoustic (UWA) networks are vital for remote sensing and ocean exploration but face inherent challenges such as limited bandwidth, long propagation delays, and highly dynamic channels. These constraints hinder real-time communication and degrade overall system performance. To address these challenges, this paper proposes a bilevel Multi-Armed Bandit (MAB) framework. At the fast inner level, a Contextual Delayed MAB (CD-MAB) jointly optimizes adaptive modulation and transmission power based on both channel state feedback and its Age of Information (AoI), thereby maximizing throughput. At the slower outer level, a Feedback Scheduling MAB dynamically adjusts the channel-state feedback interval according to throughput dynamics: stable throughput allows longer update intervals, while throughput drops trigger more frequent updates. This adaptive mechanism reduces feedback overhead and enhances responsiveness to varying network conditions. The proposed bilevel framework is computationally efficient and well-suited to resource-constrained UWA networks. Simulation results using the DESERT Underwater Network Simulator demonstrate throughput gains of up to 20.61% and energy savings of up to 36.60% compared with Deep Reinforcement Learning (DRL) baselines reported in the existing literature.

Reliable communication in UnderWater (UW) environments is challenged by limited bandwidth, high temporal variability, and significant propagation delays. Orthogonal Frequency Division Multiplexing (OFDM) has emerged as a promising solution for UW communication; however, its performance heavily depends on dynamic subcarrier selection to adapt to fluctuating channel conditions. This paper presents MERMAID, a novel framework based on a Combinatorial Multi-Armed Bandit (CMAB) approach for adaptive subcarrier selection in Underwater Acoustic Networks (UANs). MERMAID dynamically activates subcarriers based on real-time Packet Delivery Ratio (PDR) feedback to maximize reliability while minimizing energy consumption. Unlike Deep Reinforcement Learning (DRL) techniques, which require extensive training and high computational overhead, MERMAID is specifically designed for resource-constrained UW nodes, enabling fast convergence and lightweight operation. We implement MERMAID in the DESERT simulator and benchmark its performance against state-of-the-art DRL baselines. Simulation results show that MERMAID improves PDR by up to 21.15% and reduces energy consumption by up to 45.40%. These findings highlight MERMAID as a scalable, efficient, and high-performance solution for real-world Internet of Underwater Things (IoUT) applications. The framework aligns with the 6G vision by advancing intelligent and energy-efficient connectivity in harsh environments, highlighting its potential to extend communication paradigms to UW networks.

UnderWater (UW) communication channels pose unique challenges due to their limited bandwidth, significant temporal variability, and long transmission delays. Addressing these challenges calls for the implementation of robust protocols capable of dynamically adjusting transmission parameters in response to channel conditions. In such a perspective, the development of intelligent algorithms capable of quickly adapting to current channel conditions based on measurements is of crucial importance. Indeed, this allows to adapt the signal transmission characteristics to the channel conditions, and to accordingly ensure optimal performance. In this perspective, this paper introduces a Multi-Player Multi-Armed Bandit (MP-MAB) framework for smart modulation adaptation in UnderWater Acoustic (UWA) Networks. Our solution is specifically tailored to run on resource-constrained UW nodes, thanks to the simplicity and low-complexity of MAB. Our framework can leverage real-time throughput statistics to dynamically select the optimal modulation technique for multi-hop signal transmission in UW scenarios. Notably, this happens in a fully-distributed way on a per-node basis, as each UW node runs a local MAB agent to autonomously select the best modulation to use according to its own channel conditions. Using the DESERT UW simulator, we evaluate the performance of our proposed framework and compare it with alternative state-of-the-art learning approaches. Results demonstrate the higher efficiency and responsiveness of our algorithm compared to the alternatives, despite its simplicity and fully-decentralized nature.

Long Range (LoRa) technology, with its low-power and long-range communication capabilities, has emerged as a popular choice for Internet of Things (IoT) applications. In spite of its long communication range, single-hop LoRa networks may be extremely inefficient in high path-loss scenarios, such as urban, indoor and underground environments. One solution to this issue is to instead resort to multi-hop LoRa networks. However, the routing of the generated packets towards the LoRa network gateway poses several challenges, such as balancing the network energy consumption, minimizing the number of hops, and selecting high-quality paths towards the gateway. One way to solve this issue is to implement distributed smart algorithms that can efficiently address all this aspects in a reliable and fully-scalable way. In this regard, this paper introduces MAGELLAN, a novel routing algorithm for multi-hop LoRa networks based on Multi-Armed Bandit (MAB) learning. MAGELLAN aims to achieve efficient packet delivery while minimizing the number of hops and fairly distributing the energy consumption across the network. We have conducted an extensive numerical evaluation by using the LoRaEnergySim simulator, and compared MAGELLAN against various routing approaches, including LMHP, a state-of-the-art routing algorithm. The results show that MAGELLAN achieves a superior performance in terms of energy-fairness and Packet Delivery Ratio (PDR), therefore improving the network lifetime and performance. More in detail, MAGELLAN achieves a reduction of up to 10% in the energy consumption as compared to a the random approach, and of to 14% in the PDR as compared to LMHP approach.

Underwater (UW) communication channels experience limited bandwidth, high time variability and much longer delays as compared to traditional terrestrial channels. This makes communication in UW scenarios particularly challenging. One way to cope with this issue is to employ reliable smart protocols that can dynamically adjust transmission parameters based on channel conditions. The effectiveness of these solutions also relies on the design of intelligent algorithms that can forecast channel conditions based on measurements and adapt the transmission characteristics of signals according to the current state of the UW channel, in order to always guarantee the best performance. In this work we propose AMUSE, the first Multi-Armed Bandit-based algorithm for smart modulation adaptation in Underwater Acoustic Networks. Note that AMUSE is specifically designed to suit resource-constrained UW nodes thanks to its simplicity and low-complexity. In particular, AMUSE relies on the current Packet Delivery Ratio (PDR) statistics to select in real time the best modulation technique to use for multihop signal transmission in the aforementioned UW scenarios. By employing the DESERT simulator we compare the performance achieved using AMUSE to those obtained using alternative state-of-the-art learning approaches. The results show that, in spite of its simplicity, our algorithm is more efficient and responsive than the other considered approaches.

Unlike traditional terrestrial scenarios, communication channels in underwater environments face severe limitations in bandwidth and experience long propagation delay. In order to address these issues, reliable techniques capable to dynamically adapt transmission parameters to time-varying channel conditions are necessary. Actually, their effectiveness primarily relies on an accurate characterization of the underwater communication channels, which is often obtained through predictive models. In this paper, we compare the performance of different types of SNR-based predictive models (i.e., Markov models, Hidden Markov models) in terms of balance between accuracy and complexity. We also provide a Kalman filter-based prediction of SNR values and compare this prediction with the performance achieved with the Markov models above mentioned. The models we have considered to carry out the comparison analysis have been developed based on real shallow water traces taken over the Tyrrhenian Sea, Italy.

Underwater communications suffer from numerous challenges typically associated with relevant signal attenuation, long propagation delay, limited available bandwidth, and high error rates that severely affect underwater transmission performance. Therefore, it is crucial to apply adaptive or predictive techniques to ensure the best possible performance and guarantee reliability in underwater communication, especially in rapidly changing environments. Using adaptive (i.e., reactive) or predictive (i.e., proactive) methods, it is possible to avoid data retransmission, improve the lifetime of underwater nodes, reduce maintenance frequency and the necessary equipment replacement and recharge, and consequently optimize performance in general. In this regard, many works in the literature propose various adaptive or predictive techniques for UnderWater Acoustic (UWA) networks, which we critically classify and discuss in this qualitative survey.