Hand-eye calibration is a common problem in the field of collaborative robotics, involving the determination of the transformation matrix between the visual sensor and the robot flange to enable vision-based robotic tasks. However, this process typically requires multiple movements of the robot arm and an external calibration object, making it both time-consuming and inconvenient, especially in scenarios where frequent recalibration is necessary. In this work, we extend our previous method which eliminates the need for external calibration objects such as a chessboard. We propose a generic dataset generation approach for point cloud registration, focusing on aligning the robot base point cloud with the scanned data. Furthermore, a more detailed simulation study is conducted involving several different collaborative robot arms, followed by real-world experiments in an industrial setting. Our improved method is simulated and evaluated using a total of 14 robotic arms from 9 different brands, including KUKA, Universal Robots, UFACTORY, and Franka Emika, all of which are widely used in the field of collaborative robotics. Physical experiments demonstrate that our extended approach achieves performance comparable to existing commercial hand-eye calibration solutions, while completing the entire calibration procedure in just a few seconds.

Accurate ship detection and tracking has become increasingly vital due to the growth of global maritime trade and complex oceanic activities. Passive acoustic or seismic-based tracking methods, though proven effective, require extensive deployment of sensors, posing challenges in terms of range and accuracy. The recently developed Distributed Acoustic Sensing (DAS) technology offers a dense sampling, cost-effective and real-time solution by using optical fiber cables for wide-area vibration monitoring. This study investigates DAS technology for ship tracking by analyzing the Doppler shift characteristics of ship-generated wavefields. First, ships were detected by the seismic energy maps in spatial and spectral domains. Utilizing the Fourier Synchrosqueezing Transform (FSST), the Doppler shift characteristics of ship signals are examined, and ship trajectories are determined. The inverted trajectory of a ship aligns closely with the actual GPS-based trajectory, thereby validating the effectiveness and accuracy of our approach. These results demonstrate the potential of DAS for reliable ship detection and tracking, providing a robust and reliable tool for maritime surveillance and monitoring in future.

This paper presents a novel robot arm that is capable of switching between a rigid robot arm and a continuum robot arm. Therefore, the novel robot arm can perform adaptive physical interaction and manipulation against complex working environments and tasks. The switch-ability of the robot arm is achieved with two types of joints: knee-like flexible joints and continuum flexible joints, with which the continuum segment of the robot arm is capable of locking and losing, hence the degree of freedom of the robot arm is capable to be switched. In this work, kinematics is established for specifying the relationship between joints space and global coordinates in both rigid and continuum configurations. Then, the posture and workspace in rigid and continuum configurations are analyzed and illustrated with numerical simulations, and compared based on the established kinematic model. Finally, a series of preliminary experimental testing toward the joint motion and stiffness has been carried out to validate the design, the kinematic model, and the motion performance of the proposed robot arm. Both the numerical and experimental results show that the knee-like joints can guarantee favorable motion accuracy, and the motion of continuum segment from the testing is well aligned with the motion calculated from the theoretical model. Moreover, the stiffness of rigid configuration is larger than the continuum configuration based on the stiffness experiment results. Therefore, the proposed novel robot arm is capable to handle adaptive interaction and manipulation in a diverse environment through the switching between the rigid and continuum configurations.

Collaborative robots are gradually taking over the leading position in automating the production and manufacturing of the SMEs, where the human-robot collaboration is highly emphasized. Therefore, estimating the force and simulating the performance of robots are of great importance. As a newly introduced technology, digital twin, has gained more attentions for simulation, process evaluation, real-time monitoring, etc. However, the current state-of-the-art of digital twin for robots still remains on the kinematic level, and the integrated robot system dynamics is too complex to be incorporated into the digital twin. Therefore, this research starts with the perspective of harmonic drive based robot joint, and proposes a dynamic model of robot joint by analyzing the composition, transmission principle, and internal interactions. Then the experimental parameter identification is performed to obtain the inherent parameters, which can reflect the system performance characteristics. Finally, a preliminary digital twin of robot joint integrated with dynamic model is established with Gazebo and MATLAB. The proposed approach could be used to simulate the dynamic behavior of robot joint in real time and make contributions to the state of the art for digital twin.

To achieve real-time dynamic simulation analysis and optimization design, a dynamic digital twin of a nonholonomic mobile manipulator (one UR5e mounted on an industrial mobile robot MIR 200) has been developed in this paper. First, the digital twin integrated with dynamics of a mobile manipulator is established. The framework of the dynamic digital twin is presented in detail. Then, the dynamic model of the system has been established with the consideration of the physical interaction between the robot and humans/environments using Lagrange formulation. Finally, the experimental testing has been conducted to validate the dynamic model and evaluate the performances (such as real-time property, accuracy, etc.) of the dynamic digital twin that is integrated with the physical human/environment-robot interaction.

This study presents a novel dynamic modelling method for analyzing the modal properties of a curved extensible continuum manipulator (ECM). Considering the variable bending angle and the length, the kinematic and static models are firstly established for descripting the geometric posture deformation and deflection generated by gravity. Then, the modal dynamic model is developed along the centerline direction of the ECM in the polar coordinates, by which the modal properties, include modal frequency and mode shape of curved continuum manipulator with different posture parameters are analyzed. The modal response is further analyzed based on the generalized coordinate method. Finally, the modal properties of ECM with variable posture are experimentally tested and compared with the theoretical value using the in-house developed ECM physical prototype. In the experiments, the bending, length varying and circular tracking motion are performed for the comprehensive validation of the proposed dynamics modelling method.

Increasing research attention has been attracted to automatic plug-in charging in an unmanned and dangerous environment. In this work, we develop an object detection solution based on deep learning on 3D point clouds using a mobile robot manipulator to provide mobility and manipulation. In this solution, the 3D point cloud technology is adopted to measure the shapes and depth information for plugin charging. Then the deep learning is employed to deal with the uncertainty in 3D detection, such as inconsistent light conditions, irregular distribution, and structural ambiguity of point clouds. We utilize a mobile robot manipulator carrying a 3D camera and a gripper to detect the targeted objects and automate plug-in charging operations. The proposed 3D object detection principle and procedure for the automatic plug-in charging are presented in detail. The automatic plug-in charging testing is conducted to validate the developed 3D object detection algorithm using a mobile robot manipulator.

Lightweight design leads to the unwanted vibration of industrial robot manipulators. Input Shaping (IS) has been proven to be an effective vibration suppression method. However, applying IS to suppress the vibration of industrial robots faces a challenging problem: time-varying dynamics. To address the time-varying dynamics of robot manipulators, this paper presents a novel and practical solution to vibration suppression based on Time-Varying Input Shaping Technology (TVIST). Our focus in this paper is to develop a practical implementation strategy that can be applied in discrete time. A Fractional Delay Finite Impulse Response filter is employed to design and implement TVIST. This solution makes TVIST more useful in practice because it can be combined with online and discrete-time trajectory generation. It can also be implemented in combination with position control using feed-forward velocity and torque. The performance of the new approach is validated through experimental implementation on a lightweight robot from Universal Robots A/S. Experimental results are analyzed to demonstrate significant vibration suppression and increased productivity of the robot with the proposed solution. The proposed method can be extended to the vibration suppression of other types of industrial robotic manipulators with serial links as well as other time-varying dynamic systems.

Flexibility commonly exists in the joints of many industrial robots due to the elasticity of the lightweight strain-wave type transmissions being used. This leads to a dynamic time-varying displacement between the position of the drive actuator and that of the driven link. Furthermore, the joint flexibility changes with time due to the material slowly being worn off at the gear meshing. Knowing the stiffness of the robot joints is of great value, e.g. in the design of new model-based feedforward and feedback controllers, and for predictive maintenance in the case of gearing unit failure. In this paper, we address on-line estimation of robot joint stiffness using a recursive least squares strategy based on a parametric model taking into account the elastic torques' nonlinear dependency on transmission deformation. Robustness is achieved in the presence of measurement noise and in poor excitation conditions. The method can be easily extended to general classes of serial-link multi-degree-of-freedom robots. The estimation technique uses only feedback signals that are readily available on Universal Robots' e-Series manipulators. Experiments on the new UR5e manipulator demonstrate the effectiveness of the proposed method.

Efficient and reliable dynamic modelling and analysis is critical to the shape control and estimate of a backbone continuum manipulator. This paper presents a novel dynamic modelling method to investigate the deformation modal properties of a vertical stretch-retractable continuum manipulator (VSRCM) under the tip horizontal load. Based on the equivalence of the elastic bending potential energy, this method simplifies the backbone continuum manipulator to a single beam, an equivalent-guided beam (EGB). One end of the EGB is fixed, and the other end is guided. The modal dynamics model of the EGB is then established by employing the vibration theory of a continuous beam, and the modal properties of the continuum manipulator are analyzed. Static and Dynamic experiments of the VSRCM are performed to validate the dynamic modeling method through the comparison and analysis of the free dynamic response and frequency properties of the continuum manipulator. To analyze the free dynamic response, the large deflection static model with tip horizontal load is also established based on the elliptic integral approach. Experimental results and comparison demonstrate that the static model is efficient and precise to predict the large deformation, and the modal dynamic models well reflect the actual vibration modal characteristics of the VSRCM.

Extensive research efforts have been made toward automating the microinjection of biological cells by leveraging micro-robotic technologies. However, to best knowledge of the authors, there is no report on the automation of the time-consuming process: moving the injection tools (a micropipette, a grippers, etc.) and cells into the field of view (FOV) of microscope from the macro FOV(outside the microscopic FOV). This paper presents a novel macro-micro conversion strategy, and a grid detection and positioning algorithm to automate the time-consuming step of moving the injection tools and cells to the microscopic FOV. The proposed solution can free the technician from the laborious hand-eye coordination operations for moving the injection tools and cells to the target position within the microscopic FOV. Furthermore, this paper proposes an auto-focusing algorithm to automate the operation step: moving down the gripper from the air outside the culture media and then precisely clamping a cell in the liquid environment for injection. In the proposed solution, the active window-based auto-focusing algorithm is developed to solve the challenging problem: the image information is lost due to the “viscous effect” taking place when the gripper jaw touches the water surface. The proposed solutions are tested and validated by the microinjection experiments of zebrafish embryos using the in-house develop micro-robotic system. The technologies and strategies proposed in this paper significantly improve the automation level of the cell microinjections, and can be easily extended to any other micromanipulation of biological cells.

This paper presents a new method to generate vibration free rest-to-rest (RTR) trajectories for configuration dependent dynamic systems, such as robots, cranes or machine tools. The new method named 3-Segmented Input Shaping is based on a combination of the widely known Input Shaping method and a new trajectory segmentation strategy for piece wise shaping of the trajectory. The new segmentation strategy facilitates the capability of accounting for variations in system dynamics during motion by shaping acceleration and deceleration profiles with individual frequencies. In this paper the new segmentation strategy is used in combination with the bang-coast-bang (BCB) trajectory. The generated trajectories are described in closed form, hence requires no optimization and thereby provides strong computational performance. The new method is verified by numerical simulations and detailed analysis and shows great potential in vibration-free RTR trajectory generation for systems with configuration dependent dynamics.

With the increasing demand on the data rate of high-definition (HD) remote detection using unmanned aerial vehicles (UAVs), scientists have moved on from the technique of traditional microwave communication to the technique of optical communication. Through transmitting data to a geosynchronous orbit relay satellite at a high data rate, a UAV can release the storage capacity of its hard drive so as to immediately conduct a new round of HD photo shooting. In this case, we construct a theoretical model of a UAV-to-satellite optical communication system. To optimize the system performance, this paper analyzes the Doppler effect, the pointing error effect, and the atmospheric turbulence effect on the communication performance based on a theoretical study and numerical simulations. In terms of the Doppler effect, the extra background noise caused by slightly increasing the optical filter bandwidth at the receiver has a far weaker effect on the communication performance compared with the effects of atmospheric turbulence and pointing error. In terms of the effects of atmospheric turbulence and pointing error, we further analyze the key parameters of both uplink and downlink systems to better understand these effects. The numerical results can provide data references for practical system designs.

Abstract In this paper, we propose a new security protocol for a wireless sensor network, which is designed for monitoring long range power transmission lines in smart grid. Part of the monitoring network is composed of optical fiber composite over head ground wire (OPGW), thus it can be secured with conventional security protocol. However, the wireless sensor network between two neighboring OPGW gateways remains vulnerable. Our proposed security protocol focuses on the wireless sensor network part, it provides mutual authentication, data integrity, and data confidentiality for both uplink and downlink transmissions between the sensor nodes and the OPGW gateway. Besides, our proposed protocol is adaptive to the dynamic node changes of the monitoring sensor network; for example, new sensors are added to the network, or some of the sensors are malfunctioning. We further propose a self‐healing process using an “i‐neighboring nodes” public key structure and an asymmetric algorithm. We also conduct energy consumption analysis for both general and extreme conditions to show that our security protocol improves the availability of the monitoring sensor network. Copyright © 2015 John Wiley & Sons, Ltd.

Abstract In order to timely and precisely locate a problem over the power lines, the control center needs to monitor the status of the transmissionlines and the towers. In this paper, we design such a monitoring network by taking advantage of the existing optical fiber composite overhead ground wire (OPGW) alongside the transmission lines. Because it is not cost‐effective to have gateway access to the OPGW for every transmission tower, we propose to deploy a multi‐hop wireless sensor network in between two sparsely deployed neighboring OPGW gateways. We mainly study the power allocation for the data transmission of the wireless sensors because of the assumption that such sensors are powered by green energy and a battery with limited capacity for easy deployment and maintenance. Specifically, we propose several centralized schemes with different objectives, for example, the minimum power usage and fast computation. We also propose a distributed scheme so that the sensors can be even more energy efficient dealing with dynamic traffic in field operations. Moreover, we analyze the centralized schemes to study their pros and cons. We also conduct a case study for the distributed scheme to demonstrate its feasibility in field operations. Copyright © 2015 John Wiley & Sons, Ltd.

In this paper, we introduce a security protocol for wireless sensor network which is designed for monitoring long range power transmission lines in smart grid. The proposed security protocol provides authentications to the sensor nodes and the information data, and the encryption for uplink and downlink information of the power line monitoring sensor network. Different from the existing protocol, our proposed one has an auto-correction process to keep the network operating even with malfunctioning nodes. We also conduct energy consumption analysis and select more energy efficient authentication and encryption methods. In addition, the results of energy consumption analysis help to determine the transmitting power for each node so that the sensor network can meet the delay requirement at the same time.

Smart grid is a modernized power grid that uses information and communication technology to gather and act on information, to use the information to provide automatic control to improve the efficiency, reliability and sustainability of the grid. In this paper, we study a wireless sensor network for monitoring long range power transmission line in smart grid. The energy efficiency is the major concern in this paper since the monitoring network is powered by renewable energy. Taking delay in consideration, we first get the optimal energy efficiency. To provide better quality-of-service (QoS) of the sensor network, we propose schemes for weighted average energy efficiency and average delay. Then, we propose a sequential control scheme, which achieves higher weighted average energy efficiency by increasing signal-to-interference-plus-noise ratio. Moreover, with the combination of transmitting power, sequential control and delay, we conduct numerical study to demonstrate that the proposed sequential control is a practical method in improving weighted average energy efficiency. We also study the practical application of the sequential control when delay is taken into consideration since monitoring data in smart grid is delay-sensitive.

Manipulating single cells with a micropipette is the oldest, yet still a widely used technique. This paper discusses the positioning of a single cell to a target position inside the micropipette after the cell is aspirated into the micropipette. Due to the small volume of a single cell (pico-liter) and nonlinear dynamics involved, this task has high skill requirements and is labor intensive in manual operation that is solely based on trial and error and has high failure rates. We present automated techniques in this paper for achieving this task. Computer vision algorithm was developed to track a single cell inside a micropipette for automated single-cell positioning. A closed-loop robust controller integrating the dynamics of cell motion was designed to accurately and efficiently position the cell to a target position inside the micropipette. The system achieved high success rates of 97% for cell tracking (n=100) and demonstrated its capability of accurately positioning a cell inside the micropipette within 8 seconds (vs. 25 seconds by highly skilled operators).

This paper is the first report of robotic ICSI (intracytoplasmic sperm injection). ICSI is a clinical procedure performed worldwide in fertility clinics, requiring pick-up of a single sperm and insert it into oocyte (i.e., an egg cell). Since its invention 20 years ago, ICSI has been conducted manually by a handful of highly skilled embryologists; however, success rates vary significantly among clinics due to poor reproducibility and inconsistency across operators. We leverage our work in robotic cell injection to realize robotic ICSI and aim ultimately, to standardize how clinical ICSI is performed. This paper presents some of the technical aspects of our robotic ICSI system, including a cell holding device and motion control and computer vision algorithms. The system performs visual tracking of single sperm, robotic immobilization of sperm, aspiration of sperm with pico-liter volume, and insertion of sperm into an oocyte with a high degree of reproducibility. The system requires minimal human involvement (requiring only a few computer mouse clicking), and is human operator skill independent. Using the hamster oocyte-human sperm model in preliminary trials, the robotic system demonstrated a high success rate of 90.0% and survival rate of 90.7% (n=120).

Biological experiments and drug screen require the transfer of individual zebrafish embryos into standard multi-well microplates. Manually pipetting embryos into wells is tedious and time consuming. This paper reports a prototype cooperative robotic system capable of transferring zebrafish embryos in parallel and depositing a single embryo per well in a 96-well microplate. A cell holding device was developed to trap multiple embryos in a regular pattern. The cell holding device and a microplate are positioned and aligned along multiple axes by the system. Embryo release strategies were systematically studied and compared. Experiments demonstrated that out of the 1,056 zebrafish embryos used in experiments (i.e., 44 times parallel transfer into 11 96-well plates), 996 wells were successfully filled with one and only one zebrafish embryo, representing a success rate of 94.3%. Further experiments confirmed that the transferred embryos were able to develop into zebrafish with 100% survival rate.

This paper reports a prototype cooperative robotic system capable of transferring zebrafish embryos in parallel and depositing a single embryo per well in a 96-well microplate. A cell holding device was developed to trap multiple embryos in a regular pattern. The cell holding device and a microplate were positioned and aligned along multiple axes by the system. Embryo release strategies were systematically studied and compared. Experiments demonstrated that out of the 1056 zebrafish embryos used in experiments (i.e., 44 times parallel transfer into 11 96-well plates), 996 wells were successfully filled with one and only one zebrafish embryo, representing a success rate of 94.3%. Further experiments confirmed that the transferred embryos were able to develop into zebrafish with 100% survival rate.

Entropy terms are analyzed for use in a heterogeneous network involving ordered weighted averaging (OWA) operators and feed-forward neural networks, referred to as FOWA networks. The OWA operators are used to aggregate feature information and were previously used successfully in a landmine application. Weight vectors of OWA operators are special cases of Choquet capacities for which the Shannon entropy has been defined. One can also consider the quadratic Renyi entropy for OWA weight vectors. A term is added to the objective function of the FOWA network involving these entropy terms. An analysis of the derivatives used in gradient descent is conducted.