Abstract Existing knowledge distillation methods indiscriminately transfer knowledge from teacher networks, including output-level decisional biases, i.e., incorrect final predictions that can mislead student learning and limit student performance. We challenge this paradigm by proposing BTKD++, a framework that systematically filters and rectifies teacher’s output-level biased knowledge into corrective signals. Our approach partitions training data into Easy Tasks (correct teacher predictions) and Hard Tasks (incorrect predictions), then applies bias elimination and rectification modules orchestrated by dynamic learning curriculum. We provide an interpretive information-theoretic abstraction to explain the observed competence-threshold phenomenon, under which bias rectification becomes more effective when teacher errors contain sufficiently structured corrective information. BTKD++ demonstrates broad applicability across classification, detection, and segmentation tasks when task outputs are equipped with suitable probabilistic interfaces, and shows consistent effectiveness across CNNs, Transformers, and State-Space Models. Extensive experiments show consistent student-teacher transcendence, establishing new state-of-the-art results. This work redefines knowledge distillation from blind mimicry to critical learning, proving that students can surpass teachers through principled bias correction. The source code is available at https://github.com/smartyige/BTKD .

Current semi-supervised semantic segmentation (SSS) methods improve generalization via weak-to-strong pseudo-supervision with image perturbations. However, many methods are limited by employing a single perturbation mode and a specific weak-to-strong learning strategy, restricting exploration of the perturbation space and hindering performance in fine-grained segmentation. While diverse perturbations are intuitively beneficial, simply combining them can lead to inefficient optimization and instability. In this paper, we propose a multi-branch strong perturbation constraint learning framework for SSS. Our framework introduces a novel multi-branch perturbation learning (MSPL) strategy, employing multiple parallel branches with diverse strong augmentations to expand the perturbation space and capture complex semantic variations. We further design a novel constraint simulation loss (CSSL), based on a hierarchical consistency learning structure (weak-to-strong and strong-to-strong), which enforces strong-to-strong consistency between different perturbation branches. CSSL mitigates instability and enhances robustness to perturbation-induced noise, enabling the network to better generalize and achieve more accurate segmentation, especially for fine object boundaries. Extensive evaluations on benchmark datasets (PASCAL VOC 2012, Cityscapes, COCO) demonstrate that our method achieves state-of-the-art performance. Ablation studies further validate the effectiveness of our proposed MSPL and CSSL components.

Autonomous vessel docking presents significant challenges due to complex vessel dynamics, confined waterways, and critical safety requirements, demanding robust trajectory prediction models. This study proposes an interactive multiple-model (IMM) approach that seamlessly integrates physics-based and data-driven approaches to enhance predictive accuracy, interpretability, and reliability in real-world docking operations. The proposed approach adapts dynamically to changing operational conditions throughout the docking process, ensuring robust performance. The IMM approach incorporates a temporal belief decay mechanism to manage prediction uncertainties, assigning higher confidence to near-term estimates while accommodating long-term variability. This approach is complemented by a locally linear weighting mechanism, which provides interpretable insights into the influence of each model component, thereby enhancing transparency and trust in autonomous docking systems. The approach is validated in 28.9 m-length research vessel Gunnerus’ real-world docking tests, demonstrating superior trajectory prediction accuracy, confidence-aware outputs, and explainable decision-making compared to conventional single-model approaches. Results highlight that the hybrid methodology improves prediction precision and reliability, effectively bridging the gap between physics-based knowledge and data-driven adaptability. These findings suggest that the proposed IMM approach can enhance the safety, reliability, and operational efficiency of autonomous maritime systems.

Understanding human motion is fundamental to the development of intelligent systems capable of seamless interaction with people. Trajectory prediction is a critical component in domains, such as intelligent transportation, surveillance, and human–robot collaboration. However, accurately forecasting human movement remains a significant challenge due to its inherently uncertain and multimodal nature. In this work, we propose a deep neural network that models agent dynamics and predicts future trajectories by representing them as a probabilistic multimodal distribution. To effectively capture the stochasticity of human behavior, our method employs a score-based diffusion model that learns to generate realistic trajectory samples by denoising latent representations. In addition, we introduce a novel social attention mechanism designed to model complex interagent interactions, further improving predictive performance. We validate our approach in both pedestrian and marine vessel trajectory datasets, demonstrating its superior ability to capture social dynamics and forecast diverse plausible future outcomes. Extensive experiments and ablation studies confirm the robustness, generalizability, and accuracy of our framework in varied real-world environments.

This research provides a comprehensive examination of hybrid modeling approaches that integrate physics-based and data-driven methods across three transportation sectors: automotive, aviation, and maritime. Motivated by the need for improved generalization, interpretability, and robustness, hybrid models combine the expressiveness of machine learning with the reliability of physical laws. Existing hybridization techniques are categorized according to the fusion point, including input features, loss functions, model architecture, and output correction, and their respective advantages and challenges are assessed. Domain-specific trends are identified: automotive and aviation benefit from high-fidelity physical models and abundant labeled datasets, while maritime applications often contend with sparse, noisy data and low-fidelity models, thus placing greater emphasis on hybrid methods to ensure safety and reliability. We further analyze strategies such as physics-informed neural networks, residual learning, transfer learning with synthetic data, and multimodel architectures, and evaluate their suitability under different data availability and physical modeling constraints. This review highlights the critical role of domain knowledge, the impact of physical model fidelity, and the importance of adaptive integration strategies in achieving robust and trustworthy dynamic models in real-world transport systems.

Some advanced methods have leveraged the zero-shot recognition capability of the contrastive language–image pretraining (CLIP) model and adapted it to weakly supervised semantic segmentation (WSSS), achieving promising performance. However, they primarily use CLIP as an auxiliary feature extractor, leaving the fundamental limitations of class activation mapping unresolved, particularly in preserving fine-grained object boundaries and achieving precise pixelwise localization under sparse supervision. To address these challenges, this article proposes a novel end-to-end WSSS framework WTCLIP, which aims to fully exploit the potential of CLIP for weakly supervised segmentation tasks. Different from traditional methods that use CLIP only as a static feature extractor, we innovatively introduce a learnable wavelet transform decoder to enhance the information extraction capability and significantly improve the model's perception of object boundaries. We dynamically adjust the weight distribution ratio of the CLIP feature layer, capture multiscale edge information, and make full use of the time–frequency localization characteristics of the wavelet transform to significantly improve the quality of pseudolabels and achieve more accurate semantic segmentation. Experimental results show that our method significantly improves the performance of the WSSS task on two public benchmark datasets, notably by4.0%over the state-of-the-art methods, especially in capturing weakly annotated object boundary details.

Knowledge distillation (KD) is a model compression technique that transfers knowledge from a large teacher model to a smaller student model to enhance its performance. Existing methods often assume that the student model is inherently inferior to the teacher model. However, we identify that the fundamental issue affecting student performance is the bias transferred by the teacher. Current KD frameworks transmit both right and wrong knowledge, introducing bias that misleads the student model. To address this issue, we propose a novel strategy to rectify bias and greatly improve the student model's performance. Our strategy involves three steps: First, we differentiate knowledge and design a bias elimination method to filter out biases, retaining only the right knowledge for the student model to learn. Next, we propose a bias rectification method to rectify the teacher model's wrong predictions, fundamentally addressing bias interference. The student model learns from both the right knowledge and the rectified biases, greatly improving its prediction accuracy. Additionally, we introduce a dynamic learning approach with a loss function that updates weights dynamically, allowing the student model to quickly learn right knowledge-based easy tasks initially and tackle hard tasks corresponding to biases later, greatly enhancing the student model's learning efficiency. To the best of our knowledge, this is the first strategy enabling the student model to surpass the teacher model. Experiments demonstrate that our strategy, as a plug-and-play module, is versatile across various mainstream KD frameworks.

Battery systems are increasingly utilized in oceangoing ships, with a growing number of fully electric or hybrid vessels relying on battery power for propulsion. Ensuring the safety of these ships necessitates continuous monitoring of the available energy storage within the batteries. Classification societies typically mandate that the state of health (SOH) be verified through independent tests. Effective monitoring and maintenance of maritime battery systems are crucial for the sustainability and safety of marine operations. This study proposes a novel approach to predict the SOH using dual-perspective 1D fully convolutional neural networks (1D-FCNs) and snapshot data processing techniques. This paper aims to enhance predictive accuracy and model robustness by integrating snapshot curves extracted from operational data into the proposed dual-perspective FCN model. The snapshot method is notable for its ability to capture the instantaneous state of the battery through various operational parameters. Unlike traditional cumulative methods, which require extensive historical operational data, the proposed approach only necessitates specific operational data with the required features. This reduces processing time, data transfer demands, and infrastructure costs, which are particularly critical for maritime applications.

Accurately predicting the remaining useful life (RUL) of lithium-ion batteries plays a crucial role in the sustainable development and efficient operation of fields such as new energy vehicles, numerous electronic products, and energy storage power stations. This study presents a dual-exponential model integrated with an Extended Kalman Filter (EKF) for RUL prediction, enhanced by Bayesian optimization to automatically adjust the model parameters for improved accuracy. A dynamic weight loss function is employed to adapt to battery characteristics at various degradation stages, enabling optimal model parameterization. The model is evaluated using four datasets under different starting prediction cycles (300, 400, and 500 cycles). Experimental results demonstrate that the model effectively tracks capacity degradation and predicts RUL with high accuracy, fitting well with actual degradation curves and showing strong generalization ability across various datasets. Based on the conducted experiment, the proposed model demonstrates improved accuracy in tracking capacity degradation and predicting RUL.

Self-adaptive robots (SARs) in complex, uncertain environments must proactively detect and address abnormal behaviors, including out-of-distribution (OOD) cases. To this end, digital twins offer a valuable solution for OOD detection. Thus, we present a digital twin-based approach for OOD detection (ODiSAR) in SARs. ODiSAR uses a Transformer-based digital twin to forecast SAR states and employs reconstruction error and Monte Carlo dropout for uncertainty quantification. By combining reconstruction error with predictive variance, the digital twin effectively detects OOD behaviors, even in previously unseen conditions. The digital twin also includes an explainability layer that links potential OOD to specific SAR states, offering insights for self-adaptation. We evaluated ODiSAR by creating digital twins of two industrial robots: one navigating an office environment, and another performing maritime ship navigation. In both cases, ODiSAR forecasts SAR behaviors (i.e., robot trajectories and vessel motion) and proactively detects OOD events. Our results showed that ODiSAR achieved high detection performance—up to 98% AUROC, 96% TNR@TPR95, and 95% F1-score—while providing interpretable insights to support self-adaptation.

Multi-agent collaborative simultaneous localization and mapping (Mac-SLAM) facilitates mutual localization among multi-agent and mapping in unknown environments. However, Mac-SLAM faces two main practical challenges in resource-constrained situations: heavy communication load and conflicts among multi-source maps. To address these issues, we propose Alice-SLAM: an accurate and lite-communication client-server collaborative SLAM system, reducing communication load while accuracy-guaranteed. Specifically, regarding high communication demand, we optimize communication load by compressing keyframe data and sharing only key map information instead of full map information. For inconsistency among multi-maps, we combine specific bundle adjustments (BA) and an adaptive strategy for active map optimization to enhance the consistency of the global map. A set of experiments demonstrates the superior accuracy and reduced communication load of the proposed Alice-SLAM on the EuRoC dataset and in multi-user augmented reality (AR) experiments conducted in our lab, highlighting its effectiveness in resource-constrained cases. We plan to open-source our code1to encourage further research and collaboration in this area.

Parameter identification in nonlinear offshore ship dynamics is crucial yet challenging, especially when measurement noise complicates the process. Differential models are particularly susceptible to large errors due to the discrete numerical differentiation of noisy data. To enhance noise resilience and achieve accurate parameter estimates, this article proposes the use of physics-informed neural networks (PINN) to identify ship roll dynamics. Constrained by physical principles, the PINN learns ship parameters with physical interpretability, employing automatic differentiation to circumvent the noise issues inherent in discrete differentiation. Leveraging the periodic nature of roll motion, a novel activation function is introduced to improve training efficiency. Robustness is validated through simulations of ship roll dynamics under regular and random wave excitations at various noise levels. Full-scale experiments conducted in open sea conditions confirm the practical effectiveness of the proposed approach in real-world scenarios.

Current cross-domain few-shot semantic segmentation (CD-FSS) faces multiple challenges, including inconsistent feature mapping among domains and insufficient utilization of low-level information and background information from the source domain. To address these issues, this article proposes a novel cascade feature enhancement and contrastive learning framework to improve the generalization capability of CD-FSS. Within this framework, we first introduce a cascade feature enhancement module to construct distinctive feature representations, enhancing the model’s transferability across domains. By effectively integrating multilevel feature information from support images, this module strengthens the representation capability of query images. Second, we employ contrastive learning to form positive and negative sample pairs for the foreground and background, capturing rich correlations between them. Finally, the iterative prototype enhancement module we propose gradually refines the correspondence between the support image and the query image through iteration, making full use of the embedded supervisory information in the limited support samples. Experimental results demonstrate that the proposed method outperforms existing approaches on multiple benchmark datasets, achieving up to a 9.7% improvement over state-of-the-art methods.

Digital twin (DT) technology, which creates virtual representations of physical systems to optimize their life-cycle, has drawn significant attention across various industries. The automotive and aviation industries have been pioneers in adopting DTs for enhanced efficiency, predictive maintenance, and real-time decision-making. However, the maritime industry, crucial to global trade and logistics, has lagged in DT implementation. This paper aims to bridge this gap by systematically surveying DT applications in the automotive and aviation industries and exploring how this knowledge can be transferred to the maritime industry. By analyzing existing literature, identifying key trends, and summarizing best practices, a comprehensive roadmap is provided for maritime industry adoption of DT technology. The surveyed papers are selected systematically following the PRISMA statement and categorized based on characteristics such as single vs. multiple systems, modeling methods (model-driven, data-driven, and hybrid), and life-cycle phases. We introduce DT models using a five-dimensional framework and analyze their characteristics in terms of research object, subsystem application, and modeling method. Additionally, DT applications from a product life-cycle perspective, covering design, manufacturing, operation, and maintenance phases are examined. Knowledge transfer from the automotive and aviation industries to the maritime industry is summarized. In the automotive industry, DTs enhance vehicle efficiency and safety, particularly for autonomous and electric vehicles. Aviation DT research focuses on predictive maintenance, pilot training, and real-time monitoring to improve operational efficiency and safety. The maritime industry faces data challenges and operational complexity but has significant potential for DTs to enhance ship performance, safety, and predictive maintenance.

Sea state information is significant for the guide of maritime activities of autonomous vessels. The sea state estimation (SSE) model, powered by artificial intelligence (AI), has shown great effectiveness but is susceptible to malicious data attacks. These attacks can lead to significant declines in the system’s performance and result in incorrect predictions about the sea state. This study introduces SecureSSE, a strategy for protecting SSE models in autonomous marine vessels from adversarial attacks. This approach incorporates three main components: 1) the multiscale feature extraction learning (MFEL) module; 2) the feature convolution aggregation learning (FCAL) module; and 3) the perturbation examples training (PET) module. The PET module is specifically crafted to create perturbation examples that are in line with unaltered data, leveraging the capabilities of both the MFEL and FCAL modules to efficiently extract and integrate detailed features from ship motion data. Our proposed SecureSSE approach is shown to significantly improve the resilience of deep learning models against potential attacks. Through experimental testing, we have validated the effectiveness of this method in enhancing SSE. Additional ablation studies highlight the critical role of each module within the SecureSSE framework. To our knowledge, this is the first study to address adversarial attacks in this context and to propose a comprehensive defense mechanism for SSE systems in autonomous marine vessels.

The safety of navigation is critical in areas with heavy and complex traffic. Accurate prediction of the future trajectory of ships is crucial, especially in encounter situations where conventional navigational devices are prone to high uncertainty and risk due to their limitations. Unfortunately, kinematic models and data-driven methods suffer from numerous issues, such as poor performance, intricate architecture, and reduced interpretability. In response, this article introduces a novel approach, driven by automatic identification system (AIS) data, to enhance the efficacy of ship encounter trajectory prediction. It revolves around encounter classification, wherein the invaluable insights of seamanship play a pivotal role in identifying and categorizing legitimate encounters into three distinct types. This categorization forms the foundation for developing a probability-based classification model, in conjunction with a hybrid predictor that amalgamates a kinematics-based model and a neural network-based model. Historical AIS data collected in Oslofjord, Norway, are utilized in this study, and experiments have been conducted to assess the performance of the proposed method through real cases. The results substantiate the promise of the classification model and underscore the exceptional predictive capabilities of the hybrid approach. This superiority is attributed to the synergistic effects arising from the integration of its constituent models.

Navigation safety is of paramount importance in areas with heavy and complex maritime traffic. Any ship navigating such a scenario should be able to foresee the future positions of other ships and adjust its path accordingly to avoid collisions. However, predicting future trajectories is a very challenging problem due to many possible future trajectories from the inherent uncertainty and the complex interaction dynamics between different ships. In this article, we propose a deep generative model based on the conditional variational autoencoder framework to learn marine vessel movement and predict future trajectories. The model is able to produce a multimodal probability distribution over future trajectories and model the complex interactions between vessels. Experiments are performed in two-vessel encounter scenarios from real-world automatic identification system data. The proposed model outperforms the baseline methods, including both kinematics-based and data-driven methods. The trajectories predicted by the proposed model are also analyzed to demonstrate the effectiveness of the model.

Toward autonomy in marine vehicles, a high-quality dynamic model is the first step before robust controls and should be properly addressed. However, vessels operating on the ocean surface are subjected to continuous disturbances, and the maneuvering dynamics exhibit a high degree of complexity and nonlinearity. In this study, a learning-based model is proposed to capture the ship maneuvering dynamics along with temporal variations. By taking the time series of ship maneuver commands as input, the model directly predicts the corresponding motions sequence, considering the presence of disturbances. The proposed neural network model is formulated as time-discretized ODEs, and model training is facilitated by posing multistep constraints. Extensive full-scale ship maneuvering experiments are conducted in the open sea to validate the effectiveness. Comparative experiments against existing learning-based models demonstrate improved accuracy in estimating ship velocities and positions over multistep intervals.

Autonomous vessels have been identified as a promising innovation in advancing marine transportation, providing an effective means to mitigate the risk of accidents, pollution incidents, and carbon dioxide emissions. Accurate sea state estimation (SSE) plays a critical role in facilitating onboard decision-making and optimizing operational efficiency for autonomous ships. Traditional SSE approaches relying on external sensors, such as wave buoys and wave radars, are limited by cost considerations. Model-based methods are highly relying on the understanding of human knowledge to ships. Data-driven models also provide promising solutions, but their generalization is low. To address this challenge, a semi-supervised transfer learning approach for SSE (SAFENESS) is proposed. The model is trained using sufficient data in the source ship and limited data from the target ship and finally applied to the target ship. A data alignment algorithm is utilized to use the limited data of the target ship. To enhance the learning capability of the framework, two attention mechanisms are proposed, and a multi-class adversarial discriminator is introduced that can align the distributions of different domains. The effectiveness of our approach is validated through comprehensive comparisons with eleven established transfer learning methods, demonstrating the superiority of our model. The competitiveness of the proposed attention modules is verified by comparing them with state-of-the-art attention modules. The significance of each component and the influence of key parameters have been thoroughly explored in the ablation and sensitivity analysis. Our method has potential applications in maritime safety, navigation, and operation optimization.

To decrease the risk of collisions and ensure safe navigation, an Automatic Identification System (AIS) was developed to broadcast real-time ship states information, such as ship position and sailing speed. Due to the real-time characteristic of AIS data, it, hence, has been widely used to construct a data-driven model for ship trajectory prediction, which can provide onboard decision support for navigators. However, data provided by AIS is sometimes partially missing or simply wrong. Poor quality of AIS data can degrade the performance of data-driven models and lead to large uncertainty in the predicted ship positions. Therefore, this paper proposes a hybrid model to reduce uncertainty by integrating the multi-output Gaussian process (MOGP) model predictions and historical trajectory information. Historical trajectories provide useful prior knowledge to calibrate the performance of MOGP model predictions. This model is built through three steps: 1) extracting historical ship trajectory information from the route that a ship is following; 2) predicting ship positions with a data-driven predictor built by an MOGP; and 3) obtaining a trajectory by knowledge fusion of historical information and predicted results. The experiment shows the proposed hybrid model outperforms other data-driven models and has small errors and uncertainty quantification of the ship position prediction. Especially for the 6-minute position prediction, its RMSE is over 100 meters smaller than other methods.

As the development of emerging technologies is applied in the maritime industry and nautical operations, an interdisciplinary supervision approach is expected to be designed in this subject to enable students the ability to handle the issues brought by the fusion of industrial conventions and technological evolutions. This paper provides the profile of the participants and the subject in which the co-supervision is to be engaged. We investigate how interdisciplinary supervision and co-supervision can be implemented in the nautical science undergraduate program. The resources of both undergraduate education and maritime-related research at the Department of Ocean Operations and Civil Engineering at NTNU are taken as the principal for the fundamental subject. As expected, not only do the undergraduates are benefitted from the co-supervision, the researchers taking co-supervision responsibilities also pro-mote their insights by absorbing the human-dominant expertise knowledge generated by the students. Through the paper, we propose a route map to explore the mechanism of co-supervision in this subject and the expected outcomes.

Ship roll is a crucial metric in assessing the vessel's safety in offshore operations. This paper investigates input selection for predicting short-term ship roll motion using the Bidirectional Long Short-Term Memory Network (Bi-LSTM) and the Sobol sensitivity analysis of ship roll based on the predicted models. Considering the complexity of the impact of forces, velocities, and positions with six degrees of freedom on ship roll, a data-driven model is established to represent the relationship adequately. Firstly, one-step prediction models with different time intervals are established based on Bi-LSTM to express the relationship between all input features and output. Afterward, the Sobol sensitivity analysis is carried out to evaluate the impact of input features on the output based on the predicted models. Finally, mathematical statistics are utilized to optimize input selection for multi-step prediction models by analyzing the sensitivity results. The experimental results demonstrate that optimizing the input feature dimensions can improve the accuracy of one-step, five-step, and ten-step prediction models.

In this work, a survey of DT-related research in the maritime industry is presented. A five-dimensional DT model for the maritime industry is presented and explained. Moreover, research object and characteristics of DT in maritime industry are categorized and discussed. The research objects of DT in the maritime industry are classified into ships, marine structures, underwater vehicles and marine engines. The characteristics of the maritime industry DT models are discussed in terms of target system, research contribution and simulation method. In addition, DT in the context of product life-cycle perspective in the maritime industry is analyzed and discussed in terms of design, manufacturing, operation, and maintenance phases. Based on our analysis and discussion of the research, we found that the current DT research in the maritime sector is mainly focused on relatively small modules, or small systems, or even only on individual components. In addition, only a small fraction of the reviewed DT-related papers focuses on the whole life-cycle in the maritime industry. The reason may come from that the models and sub-models are not yet flexible and adaptive enough at different life-cycle stages. Therefore, we believe that the development of DT technology is still in the developmental stage in the maritime industry.

Thrust allocation (TA) is an important component in dynamic positioning (DP) system of marine vessels. It plays a crucial role in offering power optimisation while meeting physical constraints of actuators in the vessel. Non-linear objective functions in TA formulation poses challenge to deriving the solution of TA schemes. While numerical optimisation techniques have tried to offer solution, the ability of neural network to model non-linearity offers a good technique of solving it. In this paper, design of constraints for TA scheme based on neural network is discussed. The allocator is based on a multi-layered autoencoder network. The allocator thus formulated is tested in various environmental and operational profiles against a numerical optimisation scheme to test its effectiveness in meeting the constraints.

Estimating oceanic wave fields from sparse observations has been a long-standing challenge in oceanography and an important environmental metric desired for maritime operations. The requirement for frequent real-time updates of the wave field within the local area poses difficulties for data assimilation approaches, as they can be computationally complex and rely on external atmospheric forcing. The relationship between the wave field and local sparse observations is embedded in reanalysis or hindcast data. We propose a data-driven deep-learning model capable of estimating the local wave field using sparsely distributed floating wave buoys. This novel model simultaneously produces wave height, period, and direction, along with their respective uncertainties. In a year-long test period within a local fjord region characterized by complex wave patterns influenced by intricate geography, the proposed model demonstrates remarkable accuracy and efficiency in estimating wave fields. This study demonstrates the promising potential of data-driven deep-learning models as an alternative to rapidly estimating the wave field.

The visual attention of navigators is imperative to understand the logic of navigation as well as the surveillance of navigators' status and operation. Current studies are implemented with the help of wearable eye-tracker glasses; yet, the high expenditure demanded by such equipment and service and its limitations on usability have impeded related research from further development. In this letter, the authors propose a framework, which is the first attempt in the maritime domain, to provide a camera-based deep-learning (CaBDeeL) visual attention recognition solution that outperforms the intrusive eye tracker regarding its shortcomings. A wide-angle camera is configured in front of the navigator in the advanced ship-bridge simulator in a way that visual attention reflected by their facial and head movements is captured in the front view. A pair of eye-tracker glasses is used to classify the captured visual attention images, which then form the primary database. During the process of classifying camera-captured images, a convolutional neural network (CNN) is built as an automatic classifier. The CNN is applied to two scenarios, and it shows an overall 95 % accuracy.

The development of maritime autonomous surface ships (MASS) has triggered interest from both academia and industry in recent years. Nevertheless, human operators will continue to perform dominant roles onboard for the next decades. There are several critical phases where human operators are in the navigation loop (MASS-I, II, and III) before full ship autonomy (MASS-IV) is achieved. The authors conceive a cyber-physical human framework for the experiment design and implementation in the maritime domain: based on multiple experimental platforms with data exchange ports, apply monitoring on and learning from navigators’ behaviors, and adapt the ship-bridge system to provide decision support in terms of guidance, navigation, and control. The platforms include a compact simulator from Kongsberg®and an immersive simulator for preliminary research design, a group of standard maritime training simulators, and a research vessel. These platforms are utilized for data collection, scenario design, testbeds to demonstrate and verify new techniques and algorithms, etc. The authors illustrate how the framework aids the MASS research and benefits the development process.

In recent years, maritime operations have become more technologically demanding due to the more complex working condition and stricter safety requirements. The need to improve the performance of human-machine cooperation in navigation through a more intelligent system in path planning, while taking into account the human factors, is more and more urgent. In this paper, a model predictive control (MPC) optimization scheme is proposed for collision-free path planning taking into account the ship dynamics and International Regulations for Preventing Collisions at Sea (COLREGS) explicitly. It utilizes three potential fields which are designed based on COLREGS and the experience of navigators. Different tuning parameter sets are tested in a single encounter scenario including give-way, head-on and overtaking, and multiple ship encounter with 5 target ships is evaluated. The simulation shows promising results where the own ship can perform evasive action according to different encounter types following COLREGS Rule 13-15 and achieve the target positions while maintaining a safe distance during the collision avoidance period. The shortest distances between the own ship and target ships in multiple encounter scenario are all larger than 250m, which further proves the effectiveness of the algorithm.

Understanding current environmental conditions is essential for autonomous ships, among which real-time estimation of sea conditions is a key aspect. Considering the ship as a large wave buoy, the sea state can be estimated from motion responses without extra sensors installed. This task is challenging since the relationship between the wave and the ship motion is hard to model. Existing methods include a wave buoy analogy (WBA) method, which assumes linearity between wave and ship motion, and a machine learning (ML) approach. Since the data collected from a vessel in the real world are typically limited to a small range of sea states, the ML method might fail when the encountered sea state is not in the training dataset. This article proposes a hybrid approach that combines the above two methods. The ML method is compensated by the WBA method based on the uncertainty of estimation results, and thus, the failure can be avoided. Real-world historical data from the Research Vessel Gunnerus are applied to validate the approach. Results indicate that the hybrid approach improves the estimation accuracy.

High-fidelity models capable of accurately predicting ship motion are critical for promoting innovation and efficiency in the maritime industry. However, creating an advanced model that comprehensively represents the system and its interaction with dynamic environments has always been challenging. Many models provide partial knowledge about a system. To handle the deficiency and improve model fidelity, in this artile, we propose a hybrid modeling methodology, in which prior knowledge describing the ship dynamic effects is incorporated into a data-driven calibrator, yielding a representative model with high predictive capability. Enabled by the integration of model estimated ship states into the calibrator, the informative information could be interpreted and carried forward. Simulation and full-scale experiments are conducted on the research vessel Gunnerus to exemplify the concept. A best available numerical model and a neural network are prepared to be the foundation and calibrator, respectively. Experiment results show that the cooperative model greatly improves the predictive capability of the research vessel. From the ship modeling perspective, this study provides new insights by bridging the gap between two separate domains: 1) model-based and 2) data-driven.

Sea state estimation is beneficial for marine systems to enhance on-board decision-making and improve work efficiency. In the era of ship intelligence, artificial intelligence has greatly promoted the technology of sensing environment, such as by using the deep learning. However, it is difficult to collect enough motion data from a marine system to train a deep learning model. In addition, the model for sea state estimation is trained using the data from a specific marine system; applying the model directly to another marine system may result in performance degradation. In this paper, a supervised transfer learning based framework for sea state estimation (STLSSE) is proposed. The STLSSE focuses on knowledge transfer when the collected data for the source marine system is sufficient but the collected data of the target marine system is scarce. In STLSSE, a data pairing algorithm is proposed to determine the relationship of the source and the target marine system. Based on these paired data, a Siamese convolutional neural network, including a new proposed residual fully convolutional network and two novel attention modules, is designed for the semantic alignment. Moreover, the conventional contrastive loss is improved to characterize the distributions when there are only few samples in the target marine system. The extensive comparisons between STLSSE and state-of-the-art transfer learning approaches show its superior performance. The comparisons with state-of-the-art attention modules has verified the competitiveness of the proposed attention modules. The key parameters and each component of STLSSE are emphasized in the ablation and sensitivity studies.

The advent of the COVID-19 pandemic disrupted global commercial activities and the tourism industry heavily. Impacts on maritime transportation were huge, as seaborne trade represents over 80% of global merchandise trade. Investigating how COVID-19 has affected ship behaviours is significant for economic condition evaluation, port management. This paper develops an analysis method to mine knowledge from raw Automation Identification System (AIS) data. First, berths are identified by improved density-based spatial clustering of applications with noise by Pythagoras distance (PD-DBSCAN). Data features, such as ship deadweight, arrival time, dwelling time, ship types, etc., can then be extracted using information matching and statistical analysis. Next, the dynamic time warping method is employed to analyse abnormal ship behaviour patterns and quantify the impacts of COVID-19. After that, a significance test is employed to determine an impact threshold through year-on-year analysis on ship flow, daily throughout and berthing time of quays. Finally, statistical analysis is used for the short-term impact analysis. This research examines a case study based on four-year AIS data in the Oslo port area. The results show that the proposed method can identify abnormal patterns caused by COVID-19 and estimate its impacts. Passenger ships are influenced heavily compared with cargo ships. The variation of passenger ships’ flow is over 90% during 2020, larger than the average variation before 2020. The discovered knowledge could be used for future decision-making and preplanning in the next health crisis.

The COVID-19 pandemic has posed significant challenges to transportation systems in various aspects, such as transferring patients and medical resources, enforcing physical distancing in public transportation, and controlling virus transmission through transportation networks. To address these challenges, a variety of artificial intelligence technologies, such as autonomous driving, big data analytics, intelligent vehicle routing and scheduling, and intelligent traffic control, have been employed in the design of intelligent transportation systems. This Special Issue provides a forum for researchers and practitioners to present the most recent advances in presenting and applying intelligent technologies to promote transportation systems in large-scale epidemics.

Real-time digital twins of ships in operation find many applications such as predictive maintenance, climbing the ladders of ship autonomy, and offshore operational excellence. The literature describes a focus on digital twinning of individual equipment such as navigation, propulsion, engine and power system, or crane. Yet, digital twinning and virtual prototyping for offshore operations are in their infancy and the onboard digitisation hardware and the telecommunication infrastructure are becoming accessible and affordable. Previous work has failed to address the need for building a holistic model and thus contextualising the equipment with the state of the whole vessel. A prototype of an online digital twin of a research vessel is proposed, its architecture described and its suitability for virtual prototyping demonstrated in a remote control centre. The study shows a viable proof of concept for remote monitoring and crew assistance in nominal and contingency response for offshore crane operations.

Situation awareness is of great importance for autonomous ships. One key aspect is to estimate the sea state in a real-time manner. Considering the ship as a large wave buoy, the sea state can be estimated from motion responses without extra sensors installed. However, it is difficult to associate waves with ship motion through an explicit model since the hydrodynamic effect is hard to model. In this paper, a data-driven model is developed to estimate the sea state based on ship motion data. The ship motion response is analyzed through statistical, temporal, spectral, and wavelet analysis. Features from multi-domain are constructed and an ensemble machine learning model is established. Real-world data is collected from a research vessel operating on the west coast of Norway. Through the validation with the real-world data, the model shows promising performance in terms of significant wave height and peak period.

In the last decade researchers have increasingly considered eye tracking of the operators of cars and airplanes as a means to address human error and evaluate operational effectiveness. This article presents a systematic survey of recently published papers about this approach in service to the question as to whether eye tracking can be used to address operational safety in marine operations. The surveyed papers are selected systematically and were categorized according to several defined characteristics. Eye tracking depends on defining operators' areas of interest (AOIs) and measuring operators focus on them over time. We identified the method of defining AOIs as a key distinction between studies; the papers fell into four categories, depending on whether researchers relied on an expert, based it on the stimulus itself, or used an attention map or a clustering algorithm to define the AOIs they used. The article also summarizes and analyzes the design and procedure of the eye-tracking experiments in the papers. Based on the features of marine operation, instruction on AOI definition in different scenarios is extracted; guidelines on experimental design and procedure selection are provided. In the article's conclusion we apply the results to a case study of a heavy-lifting operation to demonstrate the effectiveness of eye-tracking in marine operations.

This paper introduces PythonFMU, an easy to use framework for exporting Python 3.x code as cosimulation compatible models compliant with version 2.0 of the Functional Mock-up Interface (FMI). The framework consists of a set of helper classes and a command line utility for transforming compliant python source into ready to use cross-platform FMUs. PythonFMU seamlessly takes care of a number of lowlevel FMI functions such as getting and setting variable values, and state handling, including serialization and deserialization. Furthermore it provides pre-built binaries for Windows and Linux 64-bits, generates the required modelDescription.xml containing meta-data about the model and packages all related files into a Functional Mock-up Unit (FMU) - ready to be imported into any FMI compatible simulation tool. The framework can be effortlessly installed using de-facto standard Python package managers pip and conda. While PythonFMU is more geared towards ease of use and enabling Python driven co-simulation models, it is shown to have adequate performance compared to much more low-level alternatives targeting other programming languages.

Sea State is significant to the operations on the sea. The traditional model-based approaches need lots of knowledge of vessels, which limit the real-world use. This paper proposes a spectrogram-based deep learning model for sea state estimation (SpectralNet). In this model, the ship motion data is converted to spectrogram using short time Fourier transform (STFT). Unlike other methods, the spectrogram of each sensor will be combined to a new image. And then, a 2D convolutional neural network (CNN) is built as the classifier and the sea state can be identified. The experimental results show the proposed approach can achieve higher classification accuracy compared these methods applied directly in raw time series data. Through the comparison results of the proposed approach and the combination of spectrogram of different number of sensors, the proposed approach can achieve highest classification accuracy, and the classification accuracy is growing with the number of combined sensors. The sensitivity analysis finds the classification accuracy is easily influenced by the scale factor of images.

Multi-ship collision avoidance is challenging in busy waters like the Dover Strait. Usually, ships follow the rules for avoiding collisions which are given by the Convention on the International Regulations for Preventing Collisions at Sea (COLREGS), by the International Maritime Organization (IMO). However, COLREGS can be ambiguous to follow in the close-quarters situations due to the complex crossing scenarios. In that situations, multiple ship avoid collisions with each other is highly dependent upon seamanship and crew's experience. In this study, we propose a COLREGS-compliant decision making strategy that integrates the human expertise with the artificial intelligent method. A simulation scenario was utilized to validate the effectiveness and feasibility of the proposed method.

Improving the training programs for maritime operations is beneficial to enhance the maritime safety in practice. In this article, we propose a novel approach to the assessment of visual attention in a maritime operation so as to support an expert-in-the-loop training program. Experts' knowledge of maritime operation and experiences in the simulator are incorporated into the training program in three ways. First, through a questionnaire, information about task division, identification of critical operation, and definitions of areas of interest (AOIs) are incorporated as prior knowledge for modeling visual attention. Second, a weight scale factor that emphasizes the high importance of visual focus in critical operations is utilized to generate an operations-dependent attention map. Third, based on an expert's attention map and visual switch between AOIs, a similarity metric is designed as a comprehensive evaluation between saliency and visual transition. A case study of heavy lifting operation is performed by two groups of trainees who have received different briefings about “critical operation.” The assessment result shows that the group with more detailed briefing obtains a 6% similarity score higher than the other group, which is consistent with the debriefing result of a superior performance of that group. The proposed approach is thus verified effective in assessing visual attention for the expert-in-the-loop training program.

Developing a reliable model to identify the sea state is significant for the autonomous ship. This paper introduces a novel deep neural network model (SeaStateNet) to estimate the sea state based on the ship motion data from dynamically positioned vessels. The SeaStateNet mainly consists of three components: an Long-Short-Term Memory (LSTM) recurrent neural network to capture the long dependency in the ship motion data; a convolutional neural network (CNN) to extract time-invariant features; and a Fast Fourier Transform (FFT) block to extract frequency features. A feature fusion layer is designed to learn the degree affected by each component. The proposed model is applied directly to the raw time series data, without needing of any hand-engineered features. A sensitivity analysis (SA) method is applied to assess the influence of data preprocessing. Through benchmark test and experiment on ship motion dataset, SeaStateNet is verified effective for sea state estimation. The investigation on real-time test further shows the practicality of the proposed model.

This paper provides a brief survey of recent developments on the use of electroencephalogram (EEG) sensors for detecting mental fatigue (MF) in human operators during tasks involving human-machine interaction. This research topic has received much attention since there is a consensus among experts on the increasing relation between human failure and accidents in safety-critical tasks. MF is one of the most influential aspects leading to human failure and the most reliable way to assess it is using operator's physiological data, especially EEG. In the past few decades, hundreds of publications have explored the use of EEG alone or together with other objective and subjective measures for assessing MF, drowsiness, and tiredness in human operators. With recent improvements in data preprocessing, feature extraction, and classification algorithms, the monitoring and mitigation of MF in real time has become a reality. This trend is mainly due to the increasing use of machine learning techniques. This paper provides a comprehensive look at the current state of the art in the field of MF detection using EEG, identifying the currently used technique, algorithms, and methods and possible trends and promising areas for further research. The paper is concluded by suggesting a kernel partial least squares discrete-output linear regression based model as an all-around good option for an MF assessment system.

The maritime industry widely expects to have autonomous and semiautonomous ships (autoships) in the near future. In order to operate and maintain complex and integrated systems in a safe, efficient, and cost-beneficial manner, autoships will require intelligent Prognostics and Health Management (PHM) systems. Deep learning (DL) is a potential area for this development, as it is rapidly finding applications in a variety of domains, including self-driving cars, smartphones, vision systems, and more recently in PHM applications. This paper introduces and reviews four well-established DL techniques recently applied to various practical PHM problems. The purpose is to support creativity and provide inspiration toward the PHM based on DL in autoships and the maritime industry. This paper discusses benefits, challenges, suggestions, existing problems, and future research opportunities with respect to this significant new technology.

This paper presents a modeling approach based on Bond Graph (BG) method for offshore hydraulic crane focusing on its hydraulic system characteristics. A hydraulic library is built in the modeling software tool 20-sim using BG elements. The hydraulic submodels are designed according to one specific type of offshore crane, however, they can be easily modified and reused for other similar systems. BG method is a modelling technique for modeling of complex system by describing the energy flow inside the physical system. One of the main benefits of modeling using BG for the hydraulic system is the model provide interfaces to systems of other domains, for example, cooling system, mechanical model, control unit, etc. In this paper it is shown how an integrated BG model of the hydraulic system for a knuckle boom crane is derived and used for simulation. The simulation results proved the validation and effectiveness of the presented modeling approach for simulation of multi-domain systems. INTRODUCTION Cranes are found onboard almost all kinds of vessels and platforms for handling personnel and cargo. Cranes onboard vessels and platforms handling goods between the quayside and vessel or between vessels are normally referred to as offshore cranes. Cranes that are used for handling submerged loads as well e.g. launch and recovery of submersibles or installation of subsea hardware, are normally referred to as subsea cranes. Compare to land based cranes with a solid fixed base, offshore and subsea cranes are subject to significant dynamic forces from the resulting payload sway directly or indirectly caused by the vessel motion. As field testing in offshore industry is expensive and time consuming to carry out and constrained by many factors such as weather condition and vessel availability, modeling and simulation become a crucial part for product design, testing and analysis. On one hand, offshore cranes are mostly hydraulic actuated due to the consideration for stable performance and safety redundancy. On the other hand, it is rather delicate to model and control hydraulic systems because of the complex dynamic behavior and nonlinear aspect of fluid energy transfer. Many studies on hydraulic system modeling dedicated to one or several specific components. There are many software tools available for modelling and simulation of hydraulic systems. Modelling tools used in former researches include SimHydraulic from MathWorks (Vĕchet and Krejsa 2009), Easy5 from MSC (Li et al. 2011), SimulationX from ITI (En et al. 2013), 20-sim from Controllab (Aridhi et al. 2013), etc. These programs provide standard libraries for hydraulic components which can be parameterized and modified to certain levels. The generalized models are not designed for a specific system which means they might be over-complicated thus compromise the simulation efficiency. It is possible, to a certain level, to create new specific models for components that are not included in these libraries from these software tools, but that’s not always the best way. Take 20-sim as example, a hydraulic library is developed according to the Modelica hydraulic library. The library doesn’t include all the valves in a crane system. Instead of using BG elements, the models are written in a way which is difficult for the users to understand and edit. In this paper we present a modeling approach for offshore hydraulic crane system based on BG method. The submodels are created from scratch using basic BG elements and are completely open for editing as detailed as necessary depending on the simulation purpose. Another reason of choosing 20-sim as the modelling tool is using BG method complex systems, e.g. an offshore hydraulic crane, involving multiple energy domains can be modelled and integrated. The rest of the paper starts with introducing the basics of the BG method and the hydraulic system of the Proceedings 28th European Conference on Modelling and Simulation ©ECMS Flaminio Squazzoni, Fabio Baronio, Claudia Archetti, Marco Castellani (Editors) ISBN: 978-0-9564944-8-1 / ISBN: 978-0-9564944-9-8 (CD) kunckle boom crane. Then, the modelling of the main components using BG is described and the results from the simulation of the model are presented. Finally, the conclusion and future work is discussed. BOND GRAPH METHOD BG method as a general approach for modeling interacting systems is based on identifying the energetic structure in a system. A system can be decomposed into a few basic physical properties depending on what is going to be studied, and then the system can be described by interrelated idealized elements. The energy or power interaction between two elements is called a “power bond” represented by a half arrow. Another type of bond called “signal bond” represented by a full arrow indicates a signal flow at negligible power. A power bond is defined by two variables with generalized names of “effort” and “flow”, of which the product is power. Table 1 lists a number of energy domains and their corresponding power variables. Table 1 Common used BG energy domains Energy Domain Effort (e) Flow (f) Name Unit Name Unit Mechanical translation Force N Linear velocity m/s Mechanical rotation Torque Nm Angular velocity rad/s Electrical Voltage V Current A Hydraulic Pressure Pa Volume flow m/s Thermal Temperature K Entropy flow W/°C Magnetic Magnetomotive force A Flux rate Wb/s Chemical Chemical potential J/mol Reaction rate Mol/s Roughly speaking, the basic elements account for energy supply based on supply of effort and flow (Se-element and Sf-element), potential and kinetic energy storage (Celement, I-element), energy dissipation (R-element) and energy transform (TF-element) or conversion (GYelement). In addition to the basic elements describing the boundary components, the interconnection in between two elements is described using an ideal 1junction or 0-junction element, which neither store nor dissipate the energy. In brief, a 1-junction has equal flow on all bonds adjoining and the sum of efforts equals to zero, while a 0-junction is just the opposite: the effort is the same and the sum of flow is zero. The essence of defining an element is to establish the relation of the energy variables. Below Figure 1 so-called tetrahedron of state, illustrates the basic 1-port elements relating the energy variables (Pedersen and Engja 2008). Figure 1: Tetrahedron of state for basic 1-port elements OFFSHORE CRANE HYDRAULIC SYSTEM The hydraulic system of a common offshore knuckle boom crane is studied in this paper. The crane consists of three joints actuated by a hydraulic motor and two hydraulic cylinders (Figure 2). Figure 2: Offshore hydraulic knuckle boom crane When considering the complexity of the model, it is vital that the simulation can be done in real time. Thus the hydraulic system schematic is simplified to include only the main components at a level corresponding to the characteristics that shall be studied (Figure 2). The main components of the crane hydraulic system include a Hydraulic Power Unit (HPU), pipelines, valves (compensator, 4/3proportional direction valve, load control valve), cylinders, and motors. Figure 3: Hydraulic system schematic BOND GRAPH MODELING OF CRANE HYDRAULIC SYSTEM After identified the main components of the hydraulic system, in this chapter modeling of these components using BG elements is described. The hydraulic submodels are created based on the basic principles of fluid dynamics (ASSOFLUID 2007). To reduce the complexity of the overall model, the model of each component is also simplified. Fluid inertia and flexibility are dominant in the pipeline and cylinder chambers, thus neglected in the other components. As mentioned, BG method is modelling approach by describing the energy flow of the system. In the hydraulic domain, the key principle is to establish the connection of pressure and flow through the system. HPU (pump) The HPU of the crane mainly consist of a pressure compensated pump, which maintains a preset pressure at its outlet by adjusting its delivery flow in accordance with the pressure at any given time. If the system pressure is less than the pressure set point, the pump outputs its flow proportional to the pressure deviation. In the BG method a pump is modelled as a flow source element (Sf-element). The Sf-element has one output power port associated with the pump outlet. The effort and flow relationship is given by the following equations:

This paper presents an open source, object-oriented machine learning framework, formally named Java Intelligent Optimisation (JIOP). While JIOP is still in the early stages of development, it already provides a wide variety of general learning algorithms that can be used. Initially designed as a collection of existing learning methods, JIOP aims to emphasise commonalities and dissimilarities of algorithms in order to identify their strengths and weaknesses, providing a simple, coherent and unified view. For this reason, JIOP is suitable for pedagogical purposes, such as for introducing bachelor and master degree students to the concepts of intelligent algorithms. The problems that JIOP aims to solve are initially discussed to demonstrate the need for such a framework. Later on, the design architecture and the current functions of the framework are outlined. As a validating case study, a real application where JIOP is used to minimise the cost function for solving the inverse kinematics (IK) of a KUKA industrial robotic arm with six degrees of freedom (DOF) is also presented. Related simulations are carried out to prove the effectiveness of the proposed framework.

In this paper, an FPGA based UAV INSAR raw signal simulator is designed to address high computational complexity. It is based on a time domain raw signal generation algorithm and can compute in real time. This signal simulator is designed for Hardware-in-the-loop (HWIL) UAV INSAR simulation which can be used for UAV operator training and system verification. MultiFPGAs are used in this simulator with optimisation methods to improve FPGA resource costs, including a modified non-restoring algorithm for slant range computing, as well as pipelined FFT and IFFT processors for a fast convolution method.

This paper addresses the development of a novel antisway control approach for marine shipboard cranes, offering stability, safety, and efficiency during lifting, handling, transportation, and other manipulation. The proposed idea consists of the development of an integrated system with control strategies to both reduce the effect of three-dimensional payload pendulation and to minimize wave impact during shipboard crane manipulation. We propose to use a control mechanism based on energy dissipation. The simulation results confirm the principle and effectiveness of the proposed methods for damping out pendulation. In future work, we aim to minimize wave impact on the payload by reducing the dynamic forces through controlling the length of the hoist cable while adapting to the lateral wave velocity. During the final phase of this project, the proposed control strategy will be implemented as a real physical prototype for controlling different kinds of shipboard cranes.

Swimming stability is essential to a bionic robotic fish which is aiming to be applied to practical application. The stability can be controlled through two kinds of methods, the passive control method and the active control method. The latter one performs more flexibility. Forces of disturbance caused by movements of the bionic pectoral foils and the horizontal tail of a bionic fish imitating cownose ray propelled by oscillating paired pectoral foils are analyzed. Simulation model based on both PID method and fuzzy control method for the stability performance of the bionic fish in condition of compensating work by horizontal tail or not are built. Results show that the stability of the bionic fish can be obviously improved by actively control of the horizontal tail.

The Searazor is a novel underwater vehicle designed for underwater inspection and maintenance of ships and underwater structures. In this paper, the vehicle’s dynamic characteristics and control scheme are studied. A series of sub-models, representing the major components of the vehicle, are developed in a bond graph environment. A servo control system that governs the attitude and steering is also implemented. These models are then assembled and operated as a rigid body under the force input from thruster, wheels and environmental factors. The resulting motion of the vehicle under human manoeuvring input is simulated in 20sim software. Analyses and simulations are also implemented for dynamic stabilisation under disturbance from waves and current. The simulation currently provides support for Searazor prototype design. The entire model will be the simulation framework for developing and testing control algorithms to maneuvre the vehicle in complex marine environments.

This paper introduces a modular prototyping system architecture that allows for the modeling, simulation and control of different maritime cranes or robotic arms with different kinematic structures and degrees of freedom using the Bond Graph Method. The resulting models are simulated in a virtual environment and controlled using the same input haptic device, which also provides the user with a valuable force feedback. The arm joint angles can be calculated at runtime according to the specific model of the robot to be controlled. The idea is to develop a library of crane beams, joints and actuator models that can be used as modules for simulating different cranes. The base module of this architecture is the crane beam model. Using different joint modules to connect several such models, different crane prototypes can be easily built. The library also includes a simplified model of a vessel to which the crane models can be connected in order to get a complete model. Related simulations were carried out using the so-called 20-sim simulator to validate efficiency and flexibility of the proposed architecture. In particular, a two-beam crane model connected to a simplified vessel model was implemented. To control the arm, an omega.7 from Force Dimension was used as an input haptic device.

In this paper a novel GUI for a modular robots simulation environment is introduced. The GUI is intended to be used by unexperienced users that take part in an educational workshop as well as by experienced researchers who want to work on the topic of control algorithms of modular robots with the help of a framework. It offers two modes for the two kinds of users. Each mode makes it possible to configure everything needed with a graphical interface and stores configurations in XML files. Furthermore, the GUI not only supports importing the user’s control algorithms, but also provides online modulation for these algorithms. Some learning techniques such as genetic algorithms and reinforcement learning are also integrated into the GUI for locomotion optimization. Thus, its easy to use, and its scalability makes it suitable for research and education.

In this paper, the configuration of a five-limbed modular robot is introduced. A specialised locomotion gait is designed to allow for omni-directional mobility. Due to the large diversity resulting from various gait sequences, a criteria for selecting the best gaits based on their stability characteristics is proposed. A series of simulations is then performed to evaluate the various gaits in different walking directions. A gait arrangement scheme toward omni-directional locomotion is finally derived. Lastly, Experiments are also carried out on our pentapedal robot prototype in order to validate the results of simulation. The experiments confirm the gait analysis and selection is highly accurate in the evaluation of gait stability.

To detect and localize objects is an indispensable step for many computer vision tasks. Most of the state-of-the-art methods of object detection and localization are category-dependent. These methods can achieve a significant performance. However, they are useless for detecting and localizing objects belonging to an unknown category when applying them to an unknown environment. In this paper, a method is proposed for detecting and localizing generic objects without specifying their categories. The proposed method combines diverse cues, including multi-scale saliency, superpixels straddling, intensity, depth and global information, into a uniform Bayesian framework to obtain accurate detection and localization. By comparison to state-of-the-art methods, our experiments show the promising performance of the proposed method based on the PASCAL VOC 08 dataset and our indoor scene dataset.

The redundant driving problem is an inherent phenomenon existing in a modular caterpillar robot. To limit the internal forces arising from the redundant driving, this paper proposes a joint torque control method, which is based on an assumption that there is only one active joint in the four-link mechanism driving the climbing gait. Except the active joint, the other three joints are all considered as passive joints, whose torques tend to be zero, although they are driven by motors in reality. According to the analyses of static forces in the closed chain state, the ideal torque of named active joint is calculated, and will be followed by the joint in real climbing locomotion. The experiments reveal the reasonability and feasibility of the proposed joint control method, as well as the limitations of current prototype and control algorithm.

This paper presents an efficient kinematical solution to a multi-robot system with serial and parallel mechanisms. JL-I is a reconfigurable robot featuring active spherical joints formed by serial and parallel mechanisms endowing the robotic system with the ability of changing shapes in three dimensions. The active joint here can combine the advantages of the high rigidity of a parallel mechanism and the extended workspace of a serial mechanism. However, the kinematic analysis of the serial and parallel mechanism is always the bottleneck in designing a robot and control realization. In order to deal with this problem, the whole kinematical analysis is organized in the sequence from the direct mechanical analysis related to the serial and parallel mechanism over the numerical solutions to the simplified kinematics expression. The latest results obtained demonstrate that the deduced closure-form solution is time efficient and easy to implement while offering a satisfactory motion performance in on-site experiments.

JL-2, as a new version of the JL reconfigurable mobile robot system, features not only a docking and 3D posture adjusting capability between its robots, but also a multi-functional docking gripper. The basic concept of JL is that the robots in the system can simultaneously perform basic tasks in flat terrains, and in the case of rugged terrains, the robots can interconnect to enhance their locomotion capabilities. This paper introduces new designs for JL-2 by which the docking mechanism can be used as a simple gripper with 3 DOFs. Then the technologies of the docking mechanism are discussed in detail, including the workspace of the docking gripper, the docking procedure and analyses of the self-aligning ability. Then the workspaces of the posture adjusting mechanisms between two docked robots are analyzed to clarify the reconfiguration ability of JL-2. At last, a series of real experiments are proposed to test the designs and analyses and the basic performance of JL-2.

Based on the modular concept, this paper presents two caterpillar robot prototypes which are inspired by two typical caterpillars: Inchworm and Pine Caterpillar. The inchworm robot prototype features simplest kinematics and open chain architecture. Due to the fact that there is only one attachment module supporting the inchworm robot during crawling, we apply an Unsymmetrical Phase Method (UPM) to realize a stable crawling gait for it. A pine caterpillar robot is derived from combining two inchworm robots together. The crawling gait of it features a repetitive changing chain: Open-Closed-Open. Besides the UPM in open chain states, a four-links kinematic model is applied to control the corresponding joints to transfer the crawling wave along the robot body in the closed chain state. These two prototypes are all constructed and, and their crawling locomotion abilities have been tested on vertical glasses respectively.

A vision sensor is the robot's eye to perceive its environment, but the perception performance can be significantly affected by illumination conditions. This paper presents strategies of adaptive illumination control for robot vision to achieve the best scene interpretation. It investigates how to obtain the most comfortable illumination conditions for a vision sensor. In a "comfort" condition the image reflects the natural properties of the concerned object. "Discomfort" may occur if some scene information is lost. Strategies are proposed to optimize the pose and optical parameters of the luminaire and the sensor, with emphasis on controlling the intensity and avoiding glare.

To overcome the overshoot and oscillation in the normal Bang-Bang controller applied in the pneumatic driving glass-wall cleaning robot, this paper presents a F&A compensating variable Bang-Bang control algorithm for the pneumatic position servo. Besides a new cylinder driving plan is designed, the control algorithm is also improved by adding a fiction force compensating item in the chamber force evaluation equation. To adjust the controlling parameters dynamically, certain increasing functions, such as step function, linear function and arc-tangent function, are used to define the piston's expectative acceleration along with the reduction of the piston's position error. To testify the feasibility of the control algorithms with the above three expectative acceleration setting functions respectively, a pneumatic glass-wall cleaning robot "Sky Cleaner 4" is used as a testing platform to implement a series of on-load position servo experiments. And the different results of them are compared.

This paper presents a novel field robot JL-I based on a reconfigurable concept for urban search and rescue applications. The robot consists of three identical modules; each module is an entire robotic system that can perform distributed activities. It features three-degrees-of-freedom (DOF) active joints actuated by serial and parallel mechanisms for changing shape and flexible docking mechanism. The docking mechanism enables adjacent modules to connect or disconnect flexibly and automatically. DOF analysis, working space analysis and the kinematics of the 3D active joint between connected modules are studied thoroughly. In the end a series of successful tests confirm the principles and the robot's capabilities.

Reconfigurable robots consist of many modules which are able to change the way they are connected. As a result, the robots have the capability of adopting different configurations to match various tasks and suit complex environments. This paper presents a novel field robot named JL-I which consists of three uniform modules. With the docking mechanisms, the modules can connect or disconnect flexibly and automatically. Furthermore the active joints formed by serial and parallel mechanisms endow the robot with the ability of changing shape in three dimensions. Consequently useful locomotion capabilities, such as crossing high vertical obstacles, getting self-recovery when the robot is upside-down are achieved. After describing the structural principle of the robot, the related kinematics analysis follows. In the end, the successful on-site tests confirm the principles described above and the robot's ability.

Taking Shanghai Science and Technology Museum as the operation target, the robot named skycleaner which is totally actuated by pneumatic cylinders and is sucked to the glass walls with vacuum suckers is presented. In order to solve the problems of lower stiffness and the nonlinear movement characteristic of the pneumatic system, a method of segment and variable bang-bang controller is proposed to implement the accurate control of the position servo system using the principle of pneumatic pulse width-modulation (PWM). Testing results show that the controller can effectively improve the control quality. This implies that the method can meet the requirements of realization.