In this paper, a vision-based method is proposed for cable installation tasks in constrained environments. The main challenge of such tasks lies in the potential interference between the cable and surrounding obstacles. Model-based approaches are not well-suited for these industrial scenarios due to variations in the physical properties of workpieces. To address this, the proposed method integrates a potential field-based tip trajectory regulation with a shape deformation servo. In the shape deformation servo, a planner is employed to determine a feasible shape curve that avoids obstacles. This step is crucial, as an infeasible reference for shape control may result in unstable behavior. The effectiveness of the proposed method is validated through experiments. Notably, this approach does not rely on prior model information, making it highly adaptable for industrial deployment.

Industrial part reorientation remains a critical challenge in automated manufacturing workflows, particularly with parallel-jaw grippers lacking the dexterity for complex manipulations. This paper presents a systematic flipping strategy for structured environments. A quasi-static force equilibrium model is developed to characterize multi-contact manipulation systems, and stability criteria are derived through wrench space analysis, enabling A*-based optimal trajectory generation within the derived stable configuration space. To ensure persistent fingertip-object contact, adaptive impedance control dynamically adjusts gripper stiffness based on real-time force thresholds, preventing unintended detachment. Experimental validation demonstrates robust performance in two representative scenarios:1) cube flipping on a compliant surface(84.3g, 90% success over 50 trials), 2) vision-free continuous pivoting of an irregular part on a rigid substrate(56g, 88% success over 50 trials). The methodology requires neither environmental modification nor expensive tactile sensing, showing promise for practical deployment in structured manufacturing systems.

In recent years, tactile sensors have become essential for robotic systems, particularly in tasks requiring high-precision interaction and manipulation. The Vision-Based Tactile Sensor (VBTS) represents a significant advancement in tactile sensing, utilizing cameras to monitor the deformation of soft materials at the sensor tip. Pressure applied to the sensor alters the light propagation path, thereby changing the image captured by the camera. By combining image processing and deep learning, VBTS provides highly accurate estimates of contact position and force, achieving micrometer-level resolution. This paper presents a novel VBTS design that leverages a light-conductive plate and a silicone membrane to enhance the sensor’s sensitivity to force perception. The soft, thin nature of the silicone membrane allows for precise detection of minimal forces, making it suitable for tasks involving highly deformable objects. Experimental results demonstrate the sensor’s capability in detecting contact areas and force distributions, which can be applied in diverse domains such as soft object assembly, medical assistance, and food processing. Moreover, the proposed VBTS outperforms traditional sensors by utilizing computationally efficient algorithms that maintain real-time performance without compromising resolution.

In this work, we propose a novel strategy to ensure infants, who inadvertently displace their quilts during sleep, are promptly and accurately re-covered. Our approach is formulated into two subsequent steps: interference resolution and quilt spreading. By leveraging the DWPose human skeletal detection and the Segment Anything instance segmentation models, the proposed method can accurately recognize the states of the infant and the quilt over her, which involves addressing the interferences resulted from an infant’s limbs laid on part of the quilt. Building upon prior research, the EM*D deep learning model is employed to forecast quilt state transitions before and after quilt spreading actions. To improve the sensitivity of the network in distinguishing state variation of the handled quilt, we introduce an enhanced loss function that translates the voxelized quilt state into a more representative one. Both simulation and real-world experiments validate the efficacy of our method, in spreading and recover a quilt over an infant.

The manipulation of deformable objects, especially deformable linear objects (DLOs), represents an open challenge in robotics. The keys to solving the problem are accurately recognizing the topological model describing the entwined state of DLO(s) and a manipulation strategy based on it. The situation becomes more challenging in practical applications since the DLO(s) may be placed under complex and unencountered environments, making distinguishing between the targets and the background difficult. In addition, with the information obtained from a sensor system with a limited field of sense, a robot has to treat the encountered DLO(s) as multiple ones of an unknown quantity. This paper proposes a solution based on deep learning techniques for these complicated scenarios. The approach can derive a topological model describing the entangled structure of single or multiple DLOs. Based on the model, we proposed a strategy for untangling the DLO(s), considering both the possible self-tangling in one DLO and the tangling between multiple DLOs. The strategy ensures that the entangled DLO(s) can be arranged to be the neatest state given a limited field of view. The feasibility and effectiveness of the proposed solution were verified by untangling experiments utilizing a dual-arm robot system. Even if the exact quantity of the DLO(s) is unknown, the robots can still untangle the DLO(s). Moreover, the proposed approach performed robustness to unfamiliar background textures, which is preferable in practical applications. Dataset used in this paper can be found at https://github.com/lancexz/dlos-dataset.Note to Practitioners—This article proposes an automatic solution for untangling an unknown quantity of DLO(s) using robots. Issues concerned with complex backgrounds, an unknown quantity of DLOs, and limits of field of view, which are easily encountered in practical applications, are solved. Experiments demonstrated the feasibility and robustness of the approach in different scenarios. Potential applications for the proposed method include pipeline bin-picking tasks, rope-like object manipulation in household robots, and cable assembly in industrial areas. Moreover, the topological state recognition process can also be utilized in knot-tying studies and producing datasets. As the current system can not deal with scenes where DLO segments or crossings severely overlap, practitioners should integrate real-time tracking, deformation control method, and unqualified cases detection to avoid the appearance of such cases during manipulation. Future works should also consider the rebound caused by the elastic potential of DLOs.

Accurately and robustly estimating the state of deformable linear objects (DLOs), such as ropes and wires, is crucial for DLO manipulation and other applications. However, it remains a challenging open issue due to the high dimensionality of the state space, frequent occlusions, and noises. This paper focuses on learning to robustly estimate the states of DLOs from single-frame point clouds in the presence of occlusions using a data-driven method. We propose a novel two-branch network architecture to exploit global and local information of input point cloud respectively and design a fusion module to effectively leverage the advantages of both methods. Simulation and real-world experimental results demonstrate that our method can generate globally smooth and locally precise DLO state estimation results even with heavily occluded point clouds, which can be directly applied to real-world robotic manipulation of DLOs in 3-D space.

Achieving in-hand manipulation of a grasped object is challenging in robotic communities. This article proposes a planar intrinsic in-hand manipulation approach based on isometric transformation. We demonstrate that the isometric transformation (excluding reflections) can be reduced to rotations with two fixed centers, if more than two rotations around two different centers are performed. Furthermore, this article applies this theory to reposition and reorient a grasped object by performing three or more primitive rotations sequentially. We further analyze the feasible manipulation space, where the desired position and orientation of a grasped object can be attained. The theoretical results have been verified through simulations and experiments. In addition, we performed several real-world tasks using a novel robotic hand with two rotational degrees of freedom at each finger, demonstrating that the proposed in-hand manipulation method can reposition and reorient different grasped objects in$\text{SE}(2)$.

In this paper, we presented a new method for deformation control of deformable objects, which utilizes both visual and tactile feedback. At present, manipulation of deformable objects is basically formulated by assuming positional constraints. But in fact, in many situations manipulation has to be performed under actively applied force constraints. This scenario is considered in this research. In the proposed scheme a tactile feedback is integrated to ensure a stable contact between the robot end-effector and the soft object to be manipulated. The controlled contact force is also utilized to regulate the deformation of the soft object with its shape measured by a vision sensor. The effectiveness of the proposed method is demonstrated by a book page turning and shaping experiment.

Embryos/oocytes vitrification is an essential cryopreservation technique in IVF (in vitro fertilization) clinics. The reliable and effective transferring of embryos/oocytes is crucial to the subsequent steps in the whole procedure of vitrification. After each transferring, the straw needs to be replaced with a new one. Due to the uncertainties in the fabrication and installation, the exact knowledge of the kinematic model of the straw is usually unknown, and the relationship between the microscope and the straw is also unknown without calibration beforehand. In such situation, automatically transferring the oocytes from micropipette to the narrow tip of straw (0.7mm) is very challenging. In this paper, a new vision-guided robotic system is developed to automate the transferring of the oocyte without calibration. To this end, the unknown depth information is estimated then compensated by constructing a deep vision network through microscope image, and an approximate Jacobian control algorithm is also proposed to servo control the end tip of the uncalibrated straw to contact the micropipette with the vision feedback. After that, the oocyte is automatically transferred from the micropipette to the straw to finalize the task. The stability of the closed-loop control system is rigorously proved with Lyapunov methods, and the effectiveness of the developed robot is validated in experiments.

In industry assembly lines, parts feeding machines are widely employed as the prologue of the whole procedure. They play the role of sorting the parts randomly placed in bins to the state with specified pose. With the help of the parts feeding machines, the subsequent assembly processes by robot arm can always start from the same condition. Thus it is expected that function of parting feeding machine and the robotic assembly can be integrated with one robot arm. This scheme can provide great flexibility and can also contribute to reduce the cost. The difficulties involved in this scheme lie in the fact that in the part feeding phase, the pose of the part after grasping may be not proper for the subsequent assembly. Sometimes it can not even guarantee a stable grasp. In this paper, we proposed a method to integrate parts feeding and assembly within one robot arm. This proposal utilizes a specially designed gripper tip mounted on the jaws of a two-fingered gripper. With the modified gripper, in-hand manipulation of the grasped object is realized, which can ensure the control of the orientation and offset position of the grasped object. The proposal in this paper is verified by a simulated assembly in which a robot arm completed the assembly process including parts picking from bin and a subsequent peg-in-hole assembly.

Wall polishing is a typical and essential procedure in the interior renovation. However, such works are mainly carried out by humans, which have the disadvantages of low efficiency, inconsistent quality, and issues of safety and health. A new vision-based impedance controller is proposed for polishing robots to automate the labor-intensive works. The desired impedance model is specified as the control objective to regulate the dynamic relationship between the interaction force and the motion of the robot end effector, where the motion is measured with the vision feedback. The use of the vision feedback guarantees the performance of the robot from two aspect. First, the vision feedback from the high-resolution camera ensures the accuracy of measurement of the robot end effector and hence guarantees the quality of polishing. Second, the concept of image moment is introduced such that the image Jacobian matrix is non-singular in a global sense, which guarantees the large working range of the robot. The dynamic stability of the closed-loop system is rigorously proved with Lyapunov methods, and experimental results are presented to illustrate the performance of the proposed controller.

In this paper we propose a new method to detumble a malfunctioning satellite. Large space debris such as malfunctioning satellites generally rotates with nutational motion. Thus several researches have proposed the methods to use a space robot for capturing and deorbiting these debris. The most of the past studies considered the method to detumble an uncontrollable satellite and then capture a single spinning satellite. However these methods require physical contact with malfunctioning satellites, which has a risk of accident. Therefore, we propose a method with an eddy current brake. The eddy current brake system can produce braking force to the target without any physical contact. Thus, we can reduce the risk of critical collision between the space robot and the target object. This paper firstly reviews dynamics of a tumbling satellite and proposes a detumbling strategy with the eddy current brake. We carry out a fundamental experiment to evaluate the performance of the braking force of the developed eddy current brake system, and then we simulate detumbling operation by using the experimental data and show an effectiveness of the proposed detumbling method.

A hybrid simulator is a simulator which embeds a hardware experiment in a numerical simulation. However, the hybrid simulator is generally subjected to an inherent problem of energy increase in collision of two hardwares in the loop because of delay times. To solve this problem, Osaki et al. proposed a compensation method of delay time that can realize a variable coefficient of restitution [1]. However, the above method has a drawback that a model error of the system degrades the accuracy of the compensation method. This paper describes the influence of the model error and proposes a new approach to improve the accuracy of delay time compensation based on the coefficient of restitution by introducing a new model of the system. A collision experiment by the hybrid simulator is carried out to validate the accuracy of the proposed compensation method compared with the conventional compensation method.

This paper presents a robust and efficient method of generating manipulation motion skill for non-force-feedback high speed constrained compliant robot motion. Using a non-structured teaching environment, the inherent task in the captured demonstration force and position data is estimated and reconstructed from three sets of complimentary models, including analytical mathematical modelling, empirical modelling and human skill demonstration modelling. The approach addresses task specification accuracy deficiencies, and involves outward interface simplifications, with embedded rigorous analytical methodologies that enable users to realise complex and robust constrained compliant robot motion without dealing with the low level motion generation aspects. Function based task representation supports an intuitive approach to generate robust constrained motion by skill superimposition, as exemplified by peg-in-hole with crank turning.

This paper addresses an engineering attempt of utilizing multiple robot arms in assemblies of deformable parts. The developed robot system simulates a practically existing assembly process in an automobile plant where wrapped cables (wire harness) have to be fixed on the body of a car. This operation has been performed by skilled workers and is considered to be difficult for automatization. During the development of the system, solutions are proposed for various engineering problems confronted. The problems are all due to the deformable properties of a wire harness. Many of them are general for other operations involved with deformable objects. The proposed methods in the research are verified by a successful demonstration of wire harness assembly under a condition similar to a real plant.

This paper presents a MPID (Modified Proportional-Integral-Derivative) controller used to suppress vibration of flexible manipulator moving vertically. To attain the vibration suppression a vibration variable should play role in the control signal which drives the flexible manipulator. The proposed controller is based on the conventional PID control but the integral term is replaced with another one which mainly depends on the vibration of the end effector. Camera becomes an inherent part of space manipulator, using the MPID controller with the visual information recorded by the camera makes the strain measuring circuits and amplifiers unnecessary. This means reducing the hardware used in the control of flexible manipulator. The novelty of the results lies in the fact that the measurement of the rate of change in deflection, which is used as a vibration variable, has been done without the need of numerical differentiation. Experiments successfully demonstrate that using the visual data with an observer based on Kalman filer can achieve a noticeable damping of the end effector vibration of the flexible manipulator.

In this paper, a vision-based vibration control approach is proposed to deal with the vibration appearing at the end-point of a flexible-link manipulator. It is achieved by utilizing the information obtained from end-effector camera. Particularly, the image features necessary for this approach are not specified to those obtained from assumed objects like artificial markers. This property enables the approach to be applied in a completely unknown environment. In that case, end-effector camera automatically collects the visual information needed for vibration damping. This paper shows feasibility of this strategy.

Robot operation requires a succinct interface with fewer parameters to specify. This paper presents an algorithm to interface an integrated torque control law, and human task demonstration data. The data is comprised of 6 degrees of freedom, pendant force and position data, and hand posture tracking data. The aim is to automatically specify all the required parameters for the control law, which include constrained and non-constrained trajectories, reference task frames and distal end compliance. The algorithm is embedded with functions to optimise, filter, synchronise and enhance safety.

This paper addresses a vision-based endpoint trajectory and vibration control approach for flexible manipulators. In this approach, both the trajectory and vibration control are implemented by using an endpoint camera. In order to estimate link vibrations from the visual information, Kalman filter is used. To make the endpoint to follow a specified trajectory, image interpolation approach is introduced. The experimental results prove the effectiveness of the proposed control strategy.

This paper discusses an approach to apply visual servoing to a 3D flexible-link manipulator. The proposed approach is composed of an end-effector position control based on eye-in-hand visual servoing and a vibration suppression control. In order to decouple the end-effector position control and the vibration control, the end-effector position control is applied in a low frequency domain, while the vibration control is applied in a high frequency domain. To decouple the frequency domain, a high-pass filter and a low-pass filter are used. Stability of the whole system is analyzed by investigating the pole of the system. Using the proposed approach, the 3D flexible-link manipulator succeeds to insert a peg into a hole whose clearance is 0.5 mm. However, it is difficult for the flexible-link manipulator to insert the peg into a hole whose clearance is 0.1 mm, if there is no force control. Therefore, an impedance control is also tested together with the end-effector position control and the vibration control. Using the modified approach, the flexible-link manipulator succeeds to insert the peg into the hole whose clearance is 0.1 mm

In this paper, we report on passive running of planar one-legged, biped, and quadruped robots. The topic includes the analysis of passive running gaits and their orbital stabilization. For one-legged robot, two stabilizing controllers that asymptotically stabilize periodic passive gaits are derived. In particular, the second controller is based on energy-preserving principle and its original form generates interesting quasi-periodic running gaits, which can be seen in Hamiltonian system. The controller is extend to a planar biped robot with torso, which does not have any passive running gaits. Combining simple attitude controller at stance phase generates stable periodic running gaits. For a planar quadruped robot, with an advanced gait searching algorithm, some fundamental properties of the passive running gaits, such as stability or symmetry, are observed.