In this study, the task assignment problem for heterogeneous underwater vehicle collaborative system, which involves autonomous underwater vehicles (AUVs) and underwater gliders (UGs), is studied for high-efficiency data collection. UGs and AUVs show different motion modes. The advantages of different motion modes can be mutually complemented to achieve the preference-matched task assignment results, which show greatly promising prospect to enhance the data collection efficiency. Most of existing underwater task assignment algorithms focus on the single-type vehicles, which can not be applied to the heterogeneous system. To address this issue, a hierarchical learning algorithm is proposed. Firstly, based on the evaluated emergency degree of tasks, the preliminary-task-assignment hierarchy is proposed to assign the emergency tasks to AUVs and assign the non-emergency tasks to UGs, thereby achieving the preference-matched task assignment. Therefore, the collaborative efficiency of heterogeneous system can be enhanced. Then, in the UG-task-assignment hierarchy, the adaptive serial cluster mechanism is proposed to extract the high utility task-connectivity regions for UGs, thereby fully leveraging the UG advantages in region data collection. Furthermore, in the AUV-task-assignment hierarchy, the extended self-organizing mapping neural network is constructed to eliminate the disorganization of neuronal loops. As a result, the crossed paths of AUVs can be excluded to reduce the energy consumption. Finally, the superior performance is verified by numerical results.

This paper investigates a joint discrete-channel and continuous-power allocation problem for multi-user underwater acoustic communication networks. The unknown underwater acoustic Channel State Information (CSI) and the distributed optimization requirement make the proposed hybrid discrete-continuous optimization problem full of challenges. Firstly, an adversarial multi-player bandit game model is formulated, which enables each user to independently optimize its own strategy, thereby achieving the distributed decision. In the strategic game, the Multi-armed Bandit (MAB) learning theory is exploited to achieve the best response strategy of independent user without prior CSI. Secondly, an evolutive finite discrete strategy pool learning structure is proposed to achieve an efficient search for the hybrid discrete-continuous space. The constant evolvement of strategy pool endows the proposed MAB-based algorithm with the ability to search the whole continuous power space, thereby avoiding missing the superior strategy caused by the discretization of continuous space. Thirdly, a selection probability setting rule is proposed, which promotes the exploration-exploitation balance for the dynamic strategy pool, thereby improving the learning efficiency. Finally, simulation results demonstrate the superiority of the proposed algorithm.

This article deals with a joint multiple resources (relay, channel, and power) allocation problem for underwater acoustic (UWA) cooperative communication in time-varying Internet of Underwater Things scenarios. The strong coupling of multiple resources and the unknown time-varying characteristic of UWA communication scenes make the joint optimization problem full of challenges. To address this issue, the adversarial multiarmed bandit online learning model without any prior channel information and statistic assumptions is employed. Furthermore, a double closed-loop learning structure with multiple intelligent experts assistance is proposed. Multiple experts embedded in inner loop can intelligently learn the derived inferential information to provide more efficient advice for the player in outer loop, thereby enriching learning information and enhancing learning ability. In addition, the expert diversity learning mechanism is proposed to fully reflect the characteristics of seeking advantages and avoiding disadvantages in the double closed-loop learning structure. As a result, the learning speed and performance of the proposed algorithms are significantly improved. The superiorities of the proposed algorithms are demonstrated through numerical results.

This article deals with the task assignment problem for multiple autonomous underwater vehicles to efficiently collect underwater data from sensors. We formulate a unified framework to consistently address the heterogeneous task assignment problem (nonemergency and emergency cases) without strictly distinguishing the mixed cases. First, a unified problem, which bridges the gap between different constraints and optimization objectives of different cases, is constructed. Then, the proposed reinforced self-organizing mapping algorithm is reinforced in three aspects: the regional learning rate, the self-configuring neuron (SCN) strategy, and the workload balance mechanism. Specifically, the proposed regional learning rate comprehensively considers the individual worth of tasks and the topology to generate the regional learning rate of dynamic task regions, which consists of dynamic remaining tasks and the reconstructed topology. Based on this idea, the constructed unified problem can be solved consistently. Furthermore, the proposed SCN strategy optimizes the neuron population both in quality and quantity, and guides the update of neurons with enriched historical information to improve the mapping ability. This strategy greatly improves learning efficiency and applicability in a wide range of scenarios. Meanwhile, the proposed workload balance mechanism takes into consideration of both the work capability and consumed energy to extend the continuous working capability. The numerical results validate the effectiveness and adaptability of the proposed unified task assignment framework.

This study deals with a joint channel selection and power allocation problem for time-varying underwater acoustic communication system. Without any prior channel information, designing a highly adaptable resource allocation algorithm to cope with the fast time-varying environment is a very challenging issue. To address this issue, a hierarchical learning approach, which is combined with adversarial multiarmed bandit theory and outdated pilot-based feedback information, is proposed. The proposed learning approach can online optimize joint resource allocate strategy without any prior channel state information. Specifically, a hierarchical self-reflection learning structure is proposed to offer different learning manners and spaces for the actual played information and outdated feedback information, thereby balancing the exploitation and exploration to cope with the time-varying environment effectively. Further, an integration learning structure is proposed to alleviate the solving difficulty and policy explosion of joint multiple substrategies problem. The user can rapidly achieve a few superior strategies in low-dimension space, then efficiently search the expected optimal strategy in high-dimension space, as a result, the learning efficiency is significantly improved. The proposed algorithms show strong tolerance for delay and noncomplete information due to the elaborate learning structures. The superiority of the proposed algorithms is demonstrated through numerical results.

In Internet of Underwater Things, data collection is assisted by autonomous underwater vehicle (AUV) to enhance the reliable transmission. AUV acts as a mobile collector and transmits the collected data to the station via relay nodes. However, the highly mobile nature of AUV needs an adaptive and efficient relay selection scheme for achieving good capacity performance. In this article, we propose a new contextual multiarmed bandit with evolving relay set (CMAB-ERS) learning framework, which successfully addresses crucial issues, including dynamic environment conditions and evolving relay set. To deal with the evolving relay set, CMAB-ERS incorporates collaborative effects into inference as well as learning processes, the new relays will acquire prior knowledge by having experienced nodes sharing observations, reducing the learning time significantly. To overcome the uncertainty of environmental information, we exploit the contextual environment factors to assist relay reward estimation and execute time-sensitive parameter update after every transmit–receive cycle, aiming for minimizing potential loss due to the time-varying channel. Correspondingly, the collaboration-aware online contextual bandit learning (COCBL) algorithm is designed that enables AUV to switch optimal relay adaptively and promises high-capacity transmission. Further, we rigorously prove the convergence of the COCBL algorithm by considering the evolving relay set and give its upper bound on the cumulative regret. Finally, extensive simulation results elucidate the effectiveness of the proposed COCBL.

Relay selection solutions for underwater acoustic cooperative networks suffer significant performance degradation as they fail to adapt to incomplete information, noisy interference and overwhelming dynamics. To address this challenge, a hierarchical adversarial multi-armed bandit learning framework by proposing an online reward estimation layer is designed to improve adaptive relay decision control. In online reward estimation layer, adaptive Kalman filter estimator is developed to properly handle noisy observation to support accurate reward. Meanwhile, an online predict mechanism is projected for all relays to enrich learning information. Furthermore, based on estimate error variance, an adaptive exploration structure is developed to accelerate the balance between exploration and exploitation. All gathered information are exploited to learn relay quality for the decision-making. Accordingly, we present a Hierarchical Adversarial Bandit Learning (HABL) algorithm to fully exploit the heuristic interaction between the hierarchical framework. HABL integrates reward estimation, information prediction, adaptive exploration and decision making carefully in a holistic algorithm to maximize the learning efficiency. Thereby, the HABL-based relay selection algorithm has higher system throughput and lower communication cost. Further, we rigorously analyze the convergence of HABL algorithm and give its upper bound on the cumulative regret. Finally, extensive simulations elucidate the effectiveness of the HABL.

Underwater acoustic communications (UACs) are demanded in a wide range of marine applications. However, one of the main challenges for UACs is unexpected interferences from nearby acoustic activities, especially dynamic and short-period interferences (called shot interference), which are hard to detect, estimate, and mitigate over time- and frequency-selective acoustic channels. In this article, we propose a novel filter that mitigates the interference effects for both single-carrier and multi-carrier systems without any prior knowledge of the interference. An Adaptive Sliding Window Interference Detection (ASWID) algorithm is developed to detect the location and the number of interfered symbols. Meanwhile, a least trimmed squares (LTS) equalizer is presented for the shot interference estimation and mitigation based on robust regression. The proposed algorithms are evaluated through numerical simulations and real channel measurements. Our results show that the proposed designs can detect and mitigate shot interferences effectively for single-carrier and multi-carrier UAC systems.

Femtocell is a promising technology for wireless service networks to facilitate sustainable and efficient services for users. This paper deals with a downlink joint channel assignment and power allocation problem with multiple channels, users, constraints, and uncertainties in the orthogonal frequency division of a multiple-access hierarchical femtocell network system. Specifically, a hierarchical robust Stackelberg game, which aims to achieve robust equilibrium, is first proposed for resource allocation with uncertainties. Then, a low-complexity, low-interference, high-efficiency, and high-performance algorithm is presented to handle the complex robust joint allocation problem. Considering the demand capacity of macro-base stations, an efficient fitness function in conjunction with particle swarm optimization-constriction factor, is utilized to yield the best response in the upper game to satisfy multiple constraints. Meanwhile, an iterative waterfilling algorithm is exploited to achieve the best response of femto-base stations in the lower game. Lastly, a stop protocol is established for two-tier users to accomplish an efficient robust Stackelberg equilibrium. Comparative results demonstrated that the proposed algorithm is superior to the existing game-based algorithms.

Femtocell network is regarded as the potential and effective technique to improve the capacity and coverage of traditional cellular networks. One of the challenges is how to access the network and manage the interference among different users. Compared with other access strategies, hybrid access strategy allows femtocell base stations (FBSs) to provide preferential access to femtocell users (FUEs) while other users can access nearby FBS with specific restrictions. In this paper, a robust downlink power control scheme is studied in two‐tier femtocell networks, where femtocells share the same frequency with macrocell. A hierarchical game framework that takes the different users' requirements into consideration is constructed. In addition, as the link gains are actually uncertain in dynamic environment, probabilistic constraints are used to describe the uncertainty. Then two sub‐problems are obtained to maximise the sum rate of macrocell and femtocells, respectively and guarantee the quality of service (QoS) of different users. To tackle the nonlinear and nonconvex optimization problem, successive convex approximation is introduced. And the practical iterative power allocation algorithm is provided. Finally, numerical results show that the proposed scheme is effective in aspect of energy saving and QoS guarantee under dynamic environment.

We study the problem of joint relay selection and power allocation in a underwater cooperative system with multiple users assisted by multiple relays. Due to the harsh underwater environments, the channel state information (CSI) at the transmitter is imperfect, which leads to the performance degrading in the underwater cooperative acoustic system. Therefore, we analyze the cooperative underwater acoustic channel with limited feedback to increase the sum-rate of the system. Meanwhile, different from other researches, we do not only focus on the single system scenario, but also consider the presence of nearby acoustic activities and the problem of joint relay selection and power allocation is solved in a cognitive acoustic (CA) scenario. Thus the codebook of interference CSI and the codebook of quantized relay selection and power allocation strategy are designed, respectively. Simulation results show that a few bits feedback can significantly improve the performance of the CA cooperative acoustic system.

In this study, the authors study the power control for a two‐tier network system which is comprised of a central macrocell and several femtocells. In practice, the communication environment is fairly complex and dynamic which leads to the imperfect channel state information (CSI). To enhance the robustness of the two‐tier network system, the imperfect CSI in both signal links and interference links are considered and the uncertainties of the CSI are uniformly distributed in an ellipsoid uncertainty set. Then, a probability‐constrained optimisation problem is formulated to deal with the uncertainties and protect the quality‐of‐service of all users. A novel method is provided to convert probability constraints into deterministic ones. Based on them, they propose an iterative algorithm and an admission control algorithm to enhance the network efficiency. They also investigate the case when the distributions of the uncertainties of the CSI are unknown. Finally, numerical results are given to illustrate the effectiveness of the authors’ power control scheme.

Since a microgrid consists of various distributed energy resources and local loads, integration of a distributed grid and multiple microgrids leads to an integrated hybrid energy system. This paper focuses on improving profit, economy, security, and dynamic performance of the integrated hybrid energy system by means of developing different areas of management and control strategies upon four-level hierarchical multiagent system. The level 1 agent implements optimal price bidding strategies for high profit of the overall system. The level 2 agent optimizes the energy management strategies for economic operation of each microgrid. The level 3 agent executes coordinated switching control for maintaining security of each microgrid. The level 4 agent facilitates local hybrid control of each distributed energy resource for guaranteeing dynamic performance. Finally, validity of the proposed scheme is tested by means of simulation study.