Multiple autonomous underwater vehicles (AUVs) integrating multi-agent reinforcement learning (MARL) have made remarkable achievement and widely utilized for underwater search and rescue missions. However, to perform collaborative multi-AUV search efficiently in harsh and communication-constrained marine environments, challenging issues need to be addressed, such as cold-start problem and poor collaborative information fusion. To deal with these challenges, this paper proposes a MACS scheme which integrates the reasoning capabilities of large language models (LLMs) into the MARL framework to solve the cold-start problem and facilitate efficient collaborative information fusion. In MACS, to alleviate the cold-start problem of MARL caused by the lack of prior knowledge, we design a LEMACS algorithm, which leverages LLMs to infer the initial Target Probability Map (TPM) from search tasks and underwater terrain information to accelerate the search process. Furthermore, to address low efficient data exchange and fusion issue under unstable channel, we propose a LLM-enhanced link selection algorithm LESCL which integrates TPM information and AUV link metrics to optimize the link selection procedure to enhance multi-AUV cooperative search information fusion. To validate the effectiveness of the proposed algorithms, we conduct extensive numerical simulations using open-source regional underwater terrain data, such as coral reef map dataset of Arizona State University (ASU) and the terrain data of the Dongsha Islands, and the simulation results indicate that MACS achieves a search success rate of up to 95% in emergency multi-AUV cooperative search missions. The code is available at https://github.com/SDUST-smartocean/MACS.

Autonomous Underwater Vehicles (AUVs) have been becoming excellent platforms in marine environmental monitoring and data collection missions. However, it is challenging to obtain real-time data (e.g., video and high definition images) from AUV swarms, as traditional methods have limitations, such as low-data-rate communication and restricted mobility. To tackle these challenges, in this paper, we exploit complementary strengths of acoustic-optical technology, as well as the mobility of unmanned aerial vehicles (UAVs), and propose a cooperative cross-boundary multi-modal (CCM) communication scheme, in which multiple UAVs carrying acoustic-optical communication nodes serve as mobile sinks for swarms of AUVs. In this scheme, to deal with the imbalanced coverage problem, we design a cooperative movement strategy (CMS) algorithm for multiple UAVs based on the multi-agent reinforcement learning (MARL) approach, combined with particle filtering and task assignment to UAVs for achieving dynamic coverage of AUVs. Furthermore, to achieve high data-rate transmission dealing with optical beam link misalignment problem caused by interference from currents and movement of AUVs, we design an adaptive pointing adjustment (APA) algorithm to construct a stable optical link by controlling the pointing and divergence angles of optical beam for reliable high-speed data transmission. Through extensive simulations using open-source AUV trajectory datasets and ocean current datasets, we demonstrate the effectiveness of the proposed scheme with robust network performance under dynamic maritime conditions. The code is available at https://github.com/SDUST-smartocean/CCM.

The direct cross-boundary communication between Unmanned Aerial Vehicles (UAVs) and Autonomous Underwater Vehicles (AUVs) is a pivotal component in establishing the 6G integrated air-sea-space network, holding significant importance for applications such as marine data collection and maritime collaborative search and rescue. Nevertheless, existing solutions exhibit pronounced deficiencies in path planning efficiency, the coverage range of wireless optical communication, and edge computing load balancing, which result in a high Age of Information (AoI), severely compromising the performance of time-sensitive maritime missions. To address these challenges, this paper proposes a maritime data collection scheme called MECOS, which includes LMAR2P algorithm for UAVs path planning and MAPBal algorithm for UAVs to deal with the load balancing issue. In LMAR2P, a multi-agent deep reinforcement learning (MARL) architecture is adopted, in which we leverage the global understanding capability of large language models (LLMs) to provide state representation for MARL, in order to improve path planning efficiency. Furthermore, to solve the unbalanced computational load problem, we design a kullback-leibler (KL) divergence-based reward correction mechanism and propose a distributed adaptive offloading balancing algorithm MAPBal, which enables resource-aware task allocation to ensure load balancing and reduce data processing latency. The simulation results indicate that the MECOS scheme reduces the AoI by 23.6% and 26.4% during the data collection and data processing phases of maritime missions, respectively. This research provides a viable technical solution for practical applications such as maritime monitoring, demonstrating significant scientific value and promising application prospects. The code is available at https://github.com/SDUST-smartocean/MECOS.

In complex underwater environments, multiple autonomous underwater vehicles (AUVs) typically rely on under-water acoustic communication when performing collaborative target search tasks. However, traditional underwater acoustic technology has communication constraints (e.g., high latency and low bandwidth), which leads to poor information sharing and degrade the performance of multi-AUV collaboration target search missions. To address these challenges, this paper proposes a multi-AUV collaborative acoustic-optical communication based target search scheme (AOTSS), which consists of two main components: a particle filter-based path planning algorithm (PFPPA) and a multi-agent reinforcement learning-based multi-AUV Collaborative Search Algorithm (MASA). In PFPPA, to implement efficient multimodal communication among AUVs, we design a navigation algorithm based on particle filter method and deep reinforcement learning. This approach maximizes optical communication to enhance information sharing among AUVs. Furthermore, In MASA we leverage multimodal communication to enhance information sharing among AUVs to obtain precise probability maps, and incorporate pheromones into these maps to guide AUVs performing efficient cooperate search via the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) approach. Through extensive simulations, the results demonstrate that the proposed scheme significantly enhances the multi-AUV collaboration target search efficiency.

The cross-boundary communication between unmanned aerial vehicles (UAVs) and autonomous underwater vehicles (AUVs) constitutes a pivotal component in achieving full coverage of the space-air-ground-sea integrated network of 6G. In such hybrid networks, it is imperative to intelligently plan the trajectory of UAVs by exploiting the deep reinforcement learning (DRL) technique and direct optical wireless communication (OWC) technology to ensure timely data collection. However, the traditional DRL technique suffers from issues such as sparse rewards and low sampling efficiency, which leads to a decrease in the freshness of data. To address these issues, in this paper, we investigate the trajectory planning problem for swarms of UAVs conducting data collection with age of information awareness. Leveraging the prior knowledge of large language models (LLMs) and multi-agent reinforcement learning technique, we propose a novel multi-UAV maritime data collection scheme. Firstly, we extract the prior knowledge strategies of LLMs in the form of expert trajectories through structured prompts. Then, initial strategy models are rapidly generated for UAVs through multi-agent behavior cloning method, which reduce ineffective exploration and accelerates learning. Finally, the MATD3 algorithm is used to fine-tune these strategy models, enhancing their policy learning capability in sparse reward environments. We also conduct simulations to validate the effectiveness of the proposed algorithms.

In cooperative pursuit missions, multiple unmanned aerial vehicles (UAVs) can be leveraged to perform the pursuit task collaboratively. However, uneven energy distribution and low coordination efficiency among non-stationary agents lead to low pursuit success rate. To address these challenges, we propose a scheme which integrates energy load balancing with pursuit efficiency optimization leveraging multi-agent reinforcement learning approach. In this scheme, we first develop a CatBoost-based task allocation (CBTA) algorithm to balance multi-UAVs' residual energy and minimize overall pursuit time by designing an energy-efficiency scoring function which enables real-time online task allocation optimization. Then, to reduce policy fluctuations and improve optimal policy performance, we propose a FAA-MAAC algorithm performing multiple UAVs pursuit task. To deal with the low coordination efficiency problem, in this algorithm, we design a fluctuation-triggered asynchronous actor network update (FAAU) module and incorporate it into the original multi-actor attention critic (MAAC) framework to enhance policy stability and improve performance. Extensive simulation experiments are conducted to evaluate the scheme performance, and the simulation results show that the proposed scheme reduces pursuit time and improves success rates with three evasion strategies.

In maritime Internet of Things (IoT) systems, leveraging a swarm of Uncrewed Aerial Vehicles (UAVs) and optical communication can achieve a variety of potential maritime missions. However, due to the high directionality of the optical beam and interference from the marine environment, the optical link via UAVs as relays is prone to interruption. To address this challenge, we propose a Joint Link Optimization Scheme (JLOS) that includes Wind Disturbance Resistance (WDR) and Adaptive Beamwidth Adjustment (ABA). In WDR, we first model the problem as a Partially Observed Markov Decision Process (POMDP), and then design a collaborative Multi-Agent Reinforcement Learning (MARL) approach to control a swarm of UAVs in windy conditions, to maintain mechanical stability and prevent link interruption. Furthermore, in ABA, to reduce uncertainties from control activities and environmental factors like sunlight and fog, we design an adaptive algorithm using distributed MARL. It adjusts beamwidth based on historical UAV locations and link Bit Error Ratio (BER) to improve communication reliability. Numerical simulations confirm its effectiveness in enhancing robust data transmission.

In complex underwater environments, multiple autonomous underwater vehicles (AUVs) typically use underwater acoustic communication to collaboratively search for unknown targets. However, traditional underwater acoustic communication suffers from high latency and low bandwidth issues. In contrast, optical communication provides higher bandwidth and lower latency, but its propagation distance is limited, which seriously affects the efficiency of AUV search. To address this challenge, this paper proposes a multi-AUV acoustic-optical multimodal communication target search scheme based on deep reinforcement learning (DRL). Firstly, the AUV search problem under communication constraints is analyzed, aiming to maximize search success rate, minimize search time, and maximize effective search area per unit time. Secondly, this problem is modeled as a distributed partially observable Markov decision process (Dec-POMDP), and then a multi-AUV acoustic-optical multimodal search algorithm (AOSA) is designed based on the multi-agent deep deterministic policy gradient (MADDPG) method. The AOSA algorithm enhances information sharing among AUVs through acoustic-optical multimodal communication, promoting accurate updating of probability maps. Additionally, by incorporating pheromones to correction probability maps, it enables AUVs to effectively adjust their search paths, thereby improving the efficiency of collaborative search. The numerical simulation results verify the efficiency of this scheme.

For search and rescue operations in vast maritime areas, using multiple unmanned aerial vehicles (UAVs) for cooperative search while adjusting the observation granularity of UAVs during the search process is a promising and efficient approach. However, the limited sensing and communication range of UAVs makes it difficult to obtain global information, and pose a challenge for achieving a globally optimal cooperative search strategy. To address the communication constraints, such as limited communication range and the dynamic communication topology, this paper proposes a communication gating control scheme for multi-UAV cooperative multi-granularity search based on multi-agent reinforcement learning (MARL). Through multi-hop communication, we control the UAVs to maintain a dynamic communication formation using simple single-hop position signals during the search, and employ costly multi-hop communication to transmit critical observation information based on the weights of multi-granularity observations, thereby improving global cooperative search efficiency. We evaluate the proposed algorithm with simulations, and demonstrate the effectiveness of the scheme.

The collaboration between unmanned aerial vehicles (UAV s) and unmanned surface vehicles (USV s) is critical in mar-itime search scenarios, in which UAV s can leverage high altitude and wide field of view providing real-time target information, while USV s have longer endurance capability of performing precise operations for surface targets. However, UAV s are limited by their battery capacity, which reduces their search duration and range. Furthermore, the traditional fixed charging stations necessitate UAV s to return for recharging, resulting in mission interruptions and decreased search efficiency. To address this problem, we propose an optimized scheduling scheme for UAV-USV cooperative search, in which USV s act as mobile charging stations to provide wireless charging services for UAV s to increase cooperative search mission duration with uninterrupted search execution. Firstly, the USV trajectory optimization problem under the target search constraint is formulated to minimize its energy consumption and maximize the UAV energy utilization. Then, we model the problem as a partially observable Markov decision process (POMDP) and design a scheduling algorithm leveraging the multi-agent deep deterministic policy gradient (MADDPG) method for long endurance UAV-USV collaborative search mission under energy constraints. Numerical simulations confirm the effectiveness of the proposed scheduling scheme.

Maritime data transmission with unmanned aerial vehicles (UAVs) in maritime Internet of Things (MIoT) systems has received increasing attention due to its flexibility and low cost. To further improve the efficiency of maritime data transmission between the UAVs and the maritime buoys, optical communication is considered as a promising technique because of its low latency and high bandwidth. However, optical communication encounters the challenge of beam pointing alignment, particularly in maritime data transmission involving wave disturbance and drift of buoy, which deteriorates and even interrupts the line-of-sight (LOS) optical transmission. To tackle the challenge, this paper proposes DERLOC, a reliable data transmission solution based on deep reinforcement learning (DRL). We first provide the optimization analysis of reliable data transmission and formulate the data transmission procedure as a Markov decision process (MDP) aiming at maximizing the received signal intensity. Afterwards, we propose a beam pointing adjustment algorithm based on the soft actor-critic (SAC) approach to alleviate the performance deterioration caused by waves. Then, we analyze the drift characteristic of a buoy and develop a method which enables UAV to predict the position and determine an optimal movement control strategy for ensuring the effectiveness of beam pointing and maintaining stable LOS communication. Through extensive simulations and real-time data validation, the results demonstrate that DERLOC is effective and enables a reliable data transmission via optical links.

To implement high-speed wireless communication from the deep ocean to the sea surface leveraging multiple autonomous underwater vehicles (AUVs) with underwater wireless optical communication (UWOC) technique is an emerging and promising technology that enables real-time data collection for accurate underwater exploration and monitoring, e.g., coordinated moving targets tracking. However, multi-hop UWOC is more susceptible to beam misalignment and positional uncertainty caused by external interference in the harsh marine environments. To address these challenges, we design a cooperative movement scheme for multiple AUVs based on deep reinforcement learning (DRL) approach to realize robust and reliable optical communication under mobile target tracking scenarios. In this scheme, we first model the optical channel with optical noise and then analyze the link performance of multiple AUVs to meet the bit error rate (BER) requirements. Afterwards, we map the cooperative optical communication problem to a Markov decision process (MDP) by incorporating the extended Kalman filter (EKF) technique to enhance effective communication. Finally, we propose a cooperative control strategy for multiple mobile AUVs based on deep deterministic policy gradient (DDPG). Through extensive simulations, it is demonstrated that the proposed algorithm is effective in achieving reliable underwater optical communication under mobile scenarios.

In marine wireless sensor networks, swarms of unmanned aerial vehicles (UAVs) based optical communication system can be leveraged to transmit underwater real-time monitoring data which enables a variety of potential maritime missions, such as video streaming for underwater surveillance and target tracking. However, due to the high directionality of optical beams and the mechanical instability of UAVs caused by the wind, the optical link via UAVs as relays is very fragile. To deal with the challenge, in this study, we model the problem as a partially observed Markov decision process (POMDP), and propose a novel link maintenance scheme based on the cooperative multi-agent deep deterministic policy gradient (MADDPG) approach to control the swarms of UAVs, which integrates the UAV dynamics model and the optical communication model. In this scheme, multiple UAVs act as agents controlling their own states in real-time under the complex wind field, in order to keep mechanical stability through cooperation, and dynamically maintain the reliability of the optical links which maximize communication performance to achieve reliable end-to-end optical communication and reduce energy consumption. Through numerical simulations, it demonstrates that the proposed optimization scheme is effective and achieves robust performance in terms of communication quality and energy consumption.

Mini-batch gradient descent (MBGD) is an attractive choice for support vector machines (SVM), because processing part of examples at a time is advantageous when disposing large data. Similar to other SVM learning algorithms, MBGD is vulnerable to the curse of kernelization when equipped with kernel functions, which results in unbounded linear growth in model size and update time with data size. This paper presents a budgeted mini-batch parallel gradient descent algorithm (BMBPGD) for large-scale kernel SVM training which can run efficiently on Apache Spark. Spark is a fast and general engine for large-scale data processing which is originally intended to deal with iterative algorithms. BMBPGD algorithm has constant space and time complexity per update. It uses removal budget maintenance method to keep the number of support vectors (SVs). The experiment results show that BMBPGD achieves higher accuracy than SVMWithSGD algorithm in MLlib on Spark environment, and it takes much shorter time than LibSVM.