Parkinson's Disease (PD) treatment is challenging due to symptom heterogeneity and the lack of a definitive cure. Lifelong medication requires personalized treatment plans developed by physicians, but such approaches are constrained by high costs and limited physician capacity. Although deep learning (DL) methods have been explored, they lack interpretability and are restricted to numerical data inputs. In this study, we propose a novel framework that leverages large language models (LLMs) to design personalized PD treatment strategies, integrating both patient information in natural language form and external textual knowledge sources (e.g., medical guidelines). To enhance effectiveness, we use Monte Carlo Tree Search (MCTS) to refine strategies and establish a robust medication recommendation dataset. To enhance reliability and interpretability, we incorporate Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT) reasoning within the LLM system, ensuring that each proposed strategy is accompanied by step-by-step explanations and references to similar historical cases. Experimental evaluations using the Parkinson's Progression Marking Initiative (PPMI) dataset show that our method surpasses physician-prescribed treatments, achieving an average reduction of over 1.4 points in the revised unified Parkinson's disease rating scale part III (MDS-UPDRS-III) scores. Our method also outperforms the RL-method by 1.01 points on average. Furthermore, over 43% of patients achieve more than 2 point-reduction of MDS-UPDRS-III scores. A detailed case study highlights the flexibility of LLMs in dynamically adjusting medication plans for patients at different disease stages, highlighting its potential to advance personalized PD management in real-world settings.

It is significant to employ multiple autonomous underwater vehicles (AUVs) to execute the underwater target tracking task collaboratively. However, it's pretty challenging to meet various prerequisites utilizing traditional control methods. Therefore, we propose an effective two-stage learning from demonstrations training framework, FISHER, to highlight the adaptability of reinforcement learning (RL) methods in the multi-AUV underwater target tracking task, while addressing its limitations such as extensive requirements for environmental interactions and the challenges in designing reward functions. The first stage utilizes imitation learning (IL) to realize policy improvement and generate offline datasets. To be specific, we introduce multi-agent discriminator-actor-critic based on improvements of the generative adversarial IL algorithm and multi-agent IL optimization objective derived from the Nash equilibrium condition. Then in the second stage, we develop multi-agent independent generalized decision transformer, which analyzes the latent representation to match the future states of high-quality samples rather than reward function, attaining further enhanced policies capable of handling various scenarios. Besides, we propose a simulation to simulation demonstration generation procedure to facilitate the generation of expert demonstrations in underwater environments, which capitalizes on traditional control methods and can easily accomplish the domain transfer to obtain demonstrations. Extensive simulation experiments from multiple scenarios showcase that FISHER possesses strong stability, multi-task performance and capability of generalization.

This paper develops a novel Unmanned Surface Vehicle (USV)–Autonomous Underwater Vehicle (AUV) collaborative system designed to enhance underwater task performance in extreme sea conditions. The system integrates a dual strategy: (1) high-precision multi-AUV localization enabled by Fisher Information Matrix (FIM)-optimized USV path planning, and (2) a Reinforcement Learning (RL)-based cooperative planning and control framework for multi-AUV task execution. Extensive experimental evaluations in the underwater data collection task demonstrate the system's operational feasibility, with quantitative results showing significant performance improvements over baseline methods. The proposed system exhibits robust coordination capabilities between USV and AUVs while maintaining stability in extreme sea conditions. To facilitate reproducibility and community advancement,

Ocean exploration places high demands on autonomous underwater vehicles, especially when there's observation delay. We propose age of information optimized Markov decision process (AoI-MDP) to enhance underwater tasks by modeling observation delay as signal delay and including it in the state space. AoI-MDP also introduces wait time in the action space and integrates AoI with reward functions, optimizing information freshness and decision-making using reinforcement learning. Simulations show AoI-MDP outperforms the standard MDP, demonstrating superior performance, feasibility, and generalization in underwater tasks. To accelerate relevant research, we have made the codes available as open-source at https://github.com/Xiboxtg/AoI-MDP.

We propose ERFSL, an efficient reward function searcher using large language models (LLMs) for custom-environment, multi-objective reinforcement learning (RL). ERFSL generates reward components based on explicit user requirements and rectifies them, and iteratively optimizes the weights of these components based on textual context. Applied to an underwater data collection RL task, ERFSL corrects reward codes with only one feedback iteration per requirement, and acquires diverse reward functions within the Pareto set. ERFSL also presents robust capability for deviated weights and small-size LLMs such as GPT-4o mini. The full-text prompts, examples of LLM-generated answers, and source code are available at https://360zmem.github.io/LLMRsearcher/ .

Ocean exploration requires effective collaboration between the unmanned surface vehicle (USV) and autonomous underwater vehicles (AUVs). We propose UACOF, a USV-AUV collaboration framework that enhances multi-AUV performance under extreme sea conditions. The framework includes high-precision multi-AUV location via USV path planning with Fisher information matrix optimization and reinforcement learning training for cooperative tasks. Experimental results show UACOF's superior feasibility, performance, coordination and robustness in extreme conditions.

Autonomous underwater vehicles (AUVs) are valuable for ocean exploration due to their flexibility and ability to carry communication and detection units. Nevertheless, AUVs alone often face challenges in harsh and extreme sea conditions. This study introduces a unmanned surface vehicle (USV)–AUV collaboration framework, which includes high-precision multi-AUV positioning using USV path planning via Fisher information matrix optimization and reinforcement learning for multi-AUV cooperative tasks. Applied to a multi-AUV underwater data collection task scenario, extensive simulations validate the framework’s feasibility and superior performance, highlighting exceptional coordination and robustness under extreme sea conditions. To accelerate relevant research in this field, we have made simulation code (demo version) available as open-source1.

This study presents a novel environment-aware reinforcement learning (RL) framework designed to augment the operational capabilities of autonomous underwater vehicles (AUVs) in underwater environments. Departing from traditional RL architectures, the proposed framework integrates an environment-aware network module that dynamically captures flow field data, effectively embedding this critical environmental information into the state space. This integration facilitates real-time environmental adaptation, significantly enhancing the AUV’s situational awareness and decision-making capabilities. Furthermore, the framework incorporates AUV structure characteristics into the optimization process, employing a large language model (LLM)-based iterative refinement mechanism that leverages both environmental conditions and training outcomes to optimize task performance. Comprehensive experimental evaluations demonstrate the framework’s superior performance, robustness and adaptability.

The adaptivity and maneuvering capabilities of Autonomous Underwater Vehicles (AUVs) have drawn significant attention in oceanic research, due to the unpredictable disturbances and strong coupling among the AUV’s degrees of freedom. In this paper, we developed large language model (LLM)-enhanced reinforcement learning (RL)-based adaptive S-surface controller for AUVs. Specifically, LLMs are introduced for the joint optimization of controller parameters and reward functions in RL training. Using multi-modal and structured explicit task feedback, LLMs enable joint adjustments, balance multiple objectives, and enhance task-oriented performance and adaptability. In the proposed controller, the RL policy focuses on upper-level tasks, outputting task-oriented high-level commands that the S-surface controller then converts into control signals, ensuring cancellation of nonlinear effects and unpredictable external disturbances in extreme sea conditions. Under extreme sea conditions involving complex terrain, waves, and currents, the proposed controller demonstrates superior performance and adaptability in high-level tasks such as underwater target tracking and data collection, outperforming traditional PID and SMC controllers.3

The Internet of Underwater Things (IoUT) offers significant potential for ocean exploration but encounters challenges due to dynamic underwater environments and severe signal attenuation. Current methods relying on Autonomous Underwater Vehicles (AUVs) based on online reinforcement learning (RL) lead to high computational costs and low data utilization. To address these issues and the constraints of turbulent ocean environments, we propose a multi-AUV assisted data collection framework for IoUT based on multi-agent offline RL. This framework maximizes data rate and the value of information (VoI), minimizes energy consumption, and ensures collision avoidance by utilizing environmental and equipment status data. We introduce a semi-communication decentralized training with decentralized execution (SC-DTDE) paradigm and a multi-agent independent conservative Q -learning algorithm (MAICQL) to effectively tackle the problem. Extensive simulations demonstrate the high applicability, robustness, and data collection efficiency of the proposed framework.

Accurate underwater target localization is essential for underwater exploration. To improve accuracy and efficiency in complex underwater environments, we propose the Electric Field Inversion-Localization Network (EFILN), a deep feedforward neural network that reconstructs position coordinates from underwater electric field signals. By assessing whether the neural network's input-output values satisfy the Coulomb law, the error between the network's inversion solution and the equation's exact solution can be determined. The Adam optimizer was employed first, followed by the L-BFGS optimizer, to progressively improve the output precision of EFILN. A series of noise experiments demonstrated the robustness and practical utility of the proposed method, while small sample data experiments validated its strong small-sample learning (SSL) capabilities. To accelerate relevant research, we have made the codes available as open-source11Codes are available at https://github.com/Xiboxtg/EFILN.

Unmanned underwater vehicles (UUVs) have been widely used in various ocean applications, such as underwater exploration and data collection. And the underwater pursuit-evasion game (UPEG) is the key to efficient implementation of other tasks, holding significant research value. However, testing the UPEG task in real ocean environment is both costly and risky, and currently, UUV control algorithms that rely on specific environmental models struggle to complete the complicated UPEG task. To address above challenge, we propose UPEGSim, an UUV simulator specifically designed for the UPEG task. Built through Gazebo and robot operating system, UPEGSim provides a reinforcement learning (RL) environment to train UUVs for improving the intelligent performance in the UPEG task. Furthermore, we propose an efficient UPEG training framework (ETFDU), which includes multiagent decentralized training and execution techniques, scene transfer training methods, and offline RL techniques based on decision transformer, to facilitate efficient UUV training. Through training on the UPEG task in UPEGSim, we validate the effectiveness and feasibility of the proposed UPEGSim simulator and the ETFDU training framework.

Since seamless tracking of the underwater target is crucial for various underwater applications, we propose a fusion algorithm combining deep learning and reinforcement learning for multi-autonomous underwater vehicles (AUVs) to seamlessly track the underwater target. The framework of our proposed fusion algorithm consists of two stages. In the first stage, we propose an underwater target localization method based on convolutional neural network (CNN) that relies on shaft-rate electric fields, in which the data collected by underwater sensors is utilized to train CNN to achieve accurate target localization. In the second stage, we innovatively propose a multi-agent soft actor-critic (MASAC) reinforcement learning algorithm based on centralized training with decentralized execution, in which appropriate reward functions are designed to encourage multiple AUVs to cooperate in seamlessly tracking the target in unknown environments while avoiding obstacles. Simulation results show that the proposed fusion algorithm has excellent performance, while the real-time target localization accuracy is 97.8%, and AUVs can carry out seamlessly cooperative tracking of the target in unknown environment.

Considering the wide-area distribution and limited transmission power of sensing devices in the Internet of Underwater Things (IoUT), employing autonomous underwater vehicles (AUVs) to collect data is considered a promising solution. While most existing AUV-assisted data collection schemes primarily focus on enhancing data collection throughput and identifying the shortest path, they often overlook the influence of the underwater environment on AUV and the timeliness of data collection. In this article, we design a multi-AUV-assisted data collection system, in which AUVs select their own target devices to collect data according to the data upload urgencies of IoUT devices. Considering the disturbance of turbulent ocean environment and the limited energy of AUV, we propose an environment- and energy-aware AUV-assisted data collection scheme. This scheme aims to conduct path planning for multiple AUVs based on perceived environmental information, including turbulent fields and device statuses. The primary goals are to maximize the sum data collection rate and total data throughput, minimize AUV energy consumption, reduce the average data overflow times. To solve this high-dimensional NP-hard problem, we first model the problem as a Markov decision process, and propose a multiagent independent soft actor–critic to solve it. Extensive simulations validate the effectiveness and adaptability of our approach.