Safety is a critical concern in human-robot collaboration (HRC). As collaborative robots take on increasingly complex tasks in human environments, their systems have become more sophisticated through the integration of multimodal sensors, including force-torque sensors, cameras, LiDARs, and IMUs. However, existing studies on HRC safety primarily focus on ensuring safety under normal operating conditions, overlooking scenarios where internal sensor faults occur.While anomaly detection modules can help identify sensor errors and mitigate hazards, two key challenges remain: (1) no anomaly detector is flawless, and (2) not all sensor malfunctions directly threaten human safety. Relying solely on anomaly detection can lead to missed errors or excessive false alarms.To enhance safety in real-world HRC applications, this paper introduces a deep learning-based method that proactively predicts hazards following the detection of sensory anomalies. We simulate two common types of faults—bias and noise—affecting joint sensors and monitor abnormal manipulator behaviors that could pose risks in fenceless HRC environments. A dataset of 2,400 real-world samples is collected to train the proposed hazard prediction model.The approach leverages multimodal inputs, including RGB-D images, human pose, joint states, and planned robot paths, to assess whether sensor malfunctions could lead to hazardous events. Experimental results show that the proposed method outperforms state-of-the-art models, while offering faster inference speed. Additionally, cross-scenario testing confirms its strong generalization capabilities.The code and datasets are available at: DL-based-Hazard-Prediction.

We present LiHRA, a novel dataset designed to facilitate the development of automated, learning-based, or classical risk monitoring (RM) methods for Human-Robot Interaction (HRI) scenarios. The growing prevalence of collaborative robots in industrial environments has increased the need for reliable safety systems. However, the lack of high-quality datasets that capture realistic human-robot interactions, including potentially dangerous events, slows development. LiHRA addresses this challenge by providing a comprehensive, multi-modal dataset combining 3D LiDAR point clouds, human body keypoints, and robot joint states, capturing the complete spatial and dynamic context of human-robot collaboration. This combination of modalities allows for precise tracking of human movement, robot actions, and environmental conditions, enabling accurate RM during collaborative tasks. The LiHRA dataset covers six representative HRI scenarios involving collaborative and coexistent tasks, object handovers, and surface polishing, with safe and hazardous versions of each scenario. In total, the data set includes 4,431 labeled point clouds recorded at 10 Hz, providing a rich resource for training and benchmarking classical and AI-driven RM algorithms. Finally, to demonstrate LiHRA’s utility, we introduce an RM method that quantifies the risk level in each scenario over time. This method leverages contextual information, including robot states and the dynamic model of the robot. With its combination of high-resolution LiDAR data, precise human tracking, robot state data, and realistic collision events, LiHRA offers an essential foundation for future research into real-time RM and adaptive safety strategies in human-robot workspaces.

This paper presents ETA-IK, a novel Execution-Time-Aware Inverse Kinematics method tailored for dual-arm robotic systems. The primary goal is to optimize motion execution time by leveraging the redundancy of the entire system, specifically in tasks where only the relative pose of the robots is constrained, such as dual-arm scanning of unknown objects. Unlike traditional IK methods using surrogate metrics, our approach directly optimizes execution time while implicitly considering collisions. A neural network based execution time approximator is employed to predict time-efficient joint configurations while accounting for potential collisions. Through experimental evaluation on a system composed of a UR5 and a KUKA iiwa robot, we demonstrate significant reductions in execution time. The proposed method outperforms conventional approaches, showing improved motion efficiency without sacrificing positioning accuracy.

Due to the increasing behavioral and structural complexity of robots, it is challenging to predict the execution outcome after error detection. Anomaly detection methods can help detect errors and prevent potential failures. However, not every fault leads to a failure due to the system’s fault tolerance or unintended error masking. In practical applications, a robotic system should have a potential failure evaluation module to estimate the probability of failures when receiving an error alert. Subsequently, a decision-making mechanism should help to take the next action, e.g., terminate, degrade performance, or continue the execution of the task. This paper proposes a multimodal method for failure prediction for vision-based manipulation systems that suffer from potential camera faults. We inject faults into images (e.g., noise and blur) and observe manipulation failure scenarios (e.g., pick failure, place failure, and collision) that can occur during the task. Through extensive fault injection experiments, we created a FAULT-to-FAILURE dataset containing 4000 real-world manipulation samples. The dataset is subsequently used to train the failure predictor. Our approach processes the combination of RGB images, masked images, and planned paths to effectively evaluate whether a certain faulty image could potentially lead to a manipulation failure. Results demonstrate that the proposed method outperforms state-of-the-art models in terms of overall performance, requires fewer sensors, and achieves faster inference speeds. The analytical software prototype and dataset are available at Github: MultimodalFailurePrediction.

The cyber-physical systems (CPS) are characterized by a high degree of complexity due to the presence of networked heterogeneous components. This complexity makes it crucial to prevent error propagation in the system. Therefore, error detection and mitigation are necessary requirements in CPS. Recently, DL-based techniques have emerged as popular solutions for error detection in CPS. However, the main concern of DL-based error detection models in power constrained CPS is the trade-off between accuracy and speed. This leads to the necessity of designing optimized, accurate, and lightweight models.This paper proposes an optimized lightweight error detection model based on prediction approach. The paper addresses the limitations of conventional DL-based approaches in error detection for hardware-constrained CPS, particularly an exoskeleton system. The model adopted state-of-the-art efficient architecture that comprises in parallel CNN and LSTM layers, which is then transformed into a lightweight network through data quantization and network pruning techniques. The effectiveness of the proposed method is demonstrated through its application in error detection of the exoskeleton system’s data.

This paper proposes a reachability-aware model predictive control with a discrete control barrier function for backward obstacle avoidance for a tractor-trailer system. The framework incorporates the state-variant reachable set obtained through sampling-based reachability analysis and symbolic regression into the objective function of model predictive control. By optimizing the intersection of the reachable set and iterative non-safe region generated by the control barrier function, the system demonstrates better performance in terms of safety with a constant decay rate, while enhancing the feasibility of the optimization problem. The proposed algorithm improves real-time performance due to a shorter horizon and outperforms the state-of-the-art algorithms in the simulation environment and on a real robot.

Since the introduction of the term Cyber-Physical Systems (CPS) in 2006, they came to a long way. CPS are now autonomous and networked systems of systems with state-space exceeding the capabilities of conventional risk analysis methods. Model-based fault injection methods allow assessment of a system’s fault tolerance not only during its design phase but also in the course of operation. This allows the evaluation of updates and new modules before deploying such changes to a real system. Such operational model-based fault injection on a system’s digital twin can ensure continuous safety throughout all system life cycles.Modern risk analysis tools and Machine Learning-based safety methods require vast amounts of representative input and training data. Such methods not only will require mountains of erroneous time-series data from a myriad of operational cycles, but also corresponding fault parameter labels. As the state space of the system component explodes in complexity, it becomes problematic to cover all possible component fault combinations. As such, only those faults that could lead to potential failures or increased risk scenarios are of interest for automated safety assessment methodologies. It is clear that an intelligent and effective model-based fault injection method is required for the operational safety assessment of industrial CPS.Recently we introduced a new model-based fault injection method implemented as a highly customizable Simulink block called FIBlock. It supports the model-based injection of typical faults of CPS components such as sensors, software, computing, and network hardware. In this paper, we proposed a Deep Learning-based approach for model-based fault injection called FIDGET. It extends the FIBlock with Deep Reinforcement Learning capabilities. We employed a Deep Deterministic Policy Gradient algorithm with Long Short-Term Memory (LSTM) architecture to train the Reinforcement Learning agent to per-form the automated search of fault parameters that yield the biggest system response. It allows automatic generation of labeled training data for further use in risk analysis tools or to train fault classifiers. The generated training data consists of errors that lead to the biggest response (i.e., disturbance) of the system.

As it happens, despite yet unmatched by robots perception and motor skills humans drop objects during handover because of false grasp detection and early release. Accordingly, the fluent robot-human handover is still an open challenge. This paper presents an approach to a natural robot to human handover using Capacitive Proximity Sensor (CPS) for robust grasp detection and release trigger. We propose an experimental setup for the evaluation using a collaborative robot, an eye-in-hand depth camera, and CPS integrated into the gripper. Three grasp detection methods were implemented and an object release was triggered based on torque-sensing, capacitive sensing, and the combination of both. Finally, a user study was designed and conducted, indicating that the capacitive method is the most preferred type with the shortest human idle time and the highest fluency ratings.