Reinforcement learning (RL) has gained attention for complex decision-making in uncertain environments. However, high costs and risks of real-world experimentation limit its direct application to marine vehicles. This motivates the use of simulation-based training and sim-to-real transfer techniques. Despite growing interest, a systematic understanding of how to design effective transfer strategies for marine contexts remains lacking. This paper presents a sim-to-real transfer framework tailored for marine vehicles, integrating high-fidelity, data-driven dynamics modeling with multi-factor domain randomization to address marine environmental uncertainties. Maneuvering data is utilized to extract nonlinear hydrodynamic characteristics of marine vehicles to enhance model realism. Additionally, domain randomization is explored across multiple environmental factors, including wind, wave, and current. To evaluate transferability, we construct a sim-to-sim platform with a pseudo-real environment that emulates the reality gap and adopt a path-following task using Soft Actor-Critic. We comprehensively assess the impacts of model fidelity and environmental randomization strategies on sim-to-real transfer performance. Results indicate that model accuracy positively impacts transfer performance, while aggressive domain randomization may reduce adaptability in calm conditions. Finally, a data-driven modeling and multi-factor randomization recipe is proposed for RL policy transfer in marine applications.

Abstract The Industrial Internet has emerged as a key technology in the field of industrial automation, revolutionizing traditional manufacturing processes and enabling advanced control systems by integrating machines, sensors, and software systems through network connectivity, allowing for real‐time data exchange, analysis, and decision‐making in industrial environments. Programmable Logic Controllers (PLCs) play a critical role in industrial automation systems, which are specialized digital computers designed to control various manufacturing processes and machinery. As a promising language, Structured Text (ST) is of paramount importance for industrial Internet, and it is essential to translate ST into C, which is widely used in various industries for implementing control systems. However, due to differences in syntax, data types, and programming paradigms between the two languages, the translation process is facing several challenges. In this paper, we present a domain‐specific translation tool that automates the process of converting ST code to C code. This tool incorporates advanced AI techniques, particularly leveraging the capabilities of NL‐PL LLMs, to facilitate accurate and efficient translation while generating comprehensive documentation to assist developers. Finally, the evaluation and case studies conducted validate the effectiveness of the tool, demonstrating its practical applicability and benefits in real‐world industrial settings.

Highly accurate multi-person pose estimation at a high framerate is a fundamental problem in autonomous driving. Solving the problem could aid in preventing pedestrian-car accidents. The present study tackles this problem by proposing a new model composed of a feature pyramid and an original head to a general backbone. The original head is built using lightweight CNNs and directly estimates multi-person pose coordinates. This configuration avoids the complex post-processing and two-stage estimation adopted by other models and allows for a lightweight model. Our model can be trained end-to-end and performed in real-time on a resource-limited platform (low-cost edge device) during inference. Experimental results using the COCO and CrowdPose datasets showed that our model can achieve a higher framerate (approx. 20 frames/sec with NVIDIA Jetson AGX Xavier) than other state-of-the-art models while maintaining sufficient accuracy for practical use.

Multiagent algorithms often aim to accurately predict the behaviors of other agents and find a best response accordingly. Previous works usually assume an opponent uses a stationary strategy or randomly switches among several stationary ones. However, an opponent may exhibit more sophisticated behaviors by adopting more advanced reasoning strategies, e.g., using a Bayesian reasoning strategy. This paper proposes a novel approach called Bayes-ToMoP which can efficiently detect the strategy of opponents using either stationary or higher-level reasoning strategies. Bayes-ToMoP also supports the detection of previously unseen policies and learning a best-response policy accordingly. We provide a theoretical guarantee of the optimality on detecting the opponent's strategies. We also propose a deep version of Bayes-ToMoP by extending Bayes-ToMoP with DRL techniques. Experimental results show both Bayes-ToMoP and deep Bayes-ToMoP outperform the state-of-the-art approaches when faced with different types of opponents in two-agent competitive games.

This paper presents an efficient method to construct white space database for devices to communicate in TV white space (TVWS). The goal is to build a TVWS database which senses the spectrum signal strength from white space devices (WSDs). Considering the incompleteness of measurement data, we formulate the problem of spatial inference as a matrix completion problem and propose a data recovery method by combining a fixed point continuation algorithm (FPCA) with a popular k-nearest neighbor (KNN) algorithm. Simulation results show that the proposed approach has a better performance in the TVWS database recovery than the traditional FPCA.