This paper revisits the numerical inverse kinematics (IK) problem, leveraging modern computational resources and refining the seed selection process to develop a solver that is competitive with analytical-based methods. The proposed method discretizes the robot configuration space via Centroidal Voronoi Tessellation (CVT) indexed in a KDTree, ranks candidate joint configurations by minimal joint-space adjustment, and iteratively re-attempt with the next-nearest seeds in pose space according to CVT Voronoi distance. The joint space adjustment-based seed selection increases the likelihood of rapid convergence, while the re-attempt strategy effectively helps circumvent local minima and joint limit constraints. Comparison results with both traditional numerical solvers and learning-based methods demonstrate the strengths of the proposed approach in terms of success rate, time efficiency, and accuracy. Additionally, we conduct detailed ablation studies to analyze the effects of various parameters and solver settings, providing practical insights for customization and optimization. The proposed method consistently exhibits high success rates and computational efficiency. It is suitable for time-sensitive applications.

Robotic grasping is facing a variety of real-world uncertainties caused by non-static object states, unknown object properties, and cluttered object arrangements. The difficulty of grasping increases with the presence of more uncertainties, where commonly used learning-based approaches struggle to perform consistently across varying conditions. In this study, we integrate the idea of similarity matching to tackle the challenge of grasping novel objects that are simultaneously in motion and densely cluttered using a single RGBD camera, where multiple uncertainties coexist. We achieve this by shifting visual detection from global to local states and operating grasp planning from static to dynamic scenes. Notably, we introduce optimization methods to enhance planning efficiency for this time-sensitive task. Our proposed system can adapt to various object types, arrangements and movement speeds without the need for extensive training, as demonstrated by real-world experiments.

In assembly tasks, multiple operations, such as positioning, snap-fitting, and screw fastening, are often required for a single workpiece. The multiple operations add complexity to the planning process. To address this challenge, we propose an assembly sequence planning method that considers the combination of multiple operations associated with each workpiece. We define the sequence of these operations as a "workflow" and search for an optimal assembly sequence while respecting the workflow constraints of the workpieces. Beyond handling multi-operation constraints, our method optimizes the robot’s motion costs by assigning weights to the search tree and minimizing these costs accordingly. To evaluate the effectiveness of the proposed approach, we compare assembly planning results for multi-operation tasks with and without workflow decomposition. Additionally, we analyze the influence of motion cost minimization on planning performance and computational efficiency. Experimental results verified the effectiveness of the proposed method in improving assembly planning efficiency.

Diffusion models demonstrate superior performance in capturing complex distributions from large-scale datasets, providing a promising solution for quadrupedal locomotion control. However, the robustness of the diffusion planner is inherently dependent on the diversity of the pre-collected datasets. To mitigate this issue, we propose a two-stage learning framework to enhance the capability of the diffusion planner under limited dataset (reward-agnostic). Through the offline stage, the diffusion planner learns the joint distribution of state-action sequences from expert datasets without using reward labels. Subsequently, we perform the online interaction in the simulation environment based on the trained offline planner, which significantly diversified the original behavior and thus improves the robustness. Specifically, we propose a novel weak preference labeling method without the ground-truth reward or human preferences. The proposed method exhibits superior stability and velocity tracking accuracy in pacing, trotting, and bounding gait under different speeds and can perform a zero-shot transfer to the real Unitree Go1 robots. The project website for this paper is at https://shangjaven.github.io/preference-aligned-diffusion-legged/.

In this work, we conducted research on deformable object manipulation by robots based on demonstration-enhanced reinforcement learning (RL). We present FADERL (Fuzzy-Augmented Demonstration-Embedded Reinforcement Learning), a novel framework for robotic manipulation of deformable objects that significantly improves reinforcement learning efficiency through synergistic unification of High-Dimensional Takagi-Sugeno-Kang (HTSK) fuzzy systems, Generative Adversarial Behavior Cloning (GABC), and Conditional Policy Learning (CPL). Compared to the Rainbow-DDPG baseline, FADERL achieves 2.01× higher global average reward and reduces standard deviation to 45% while requiring fewer computational resources. To address the high cost of human demonstration collection, we introduce a Nonlinear Model Predictive Control (NMPC)-based data augmentation method that generates high-quality demonstrations at minimal cost. Simulation results demonstrate that NMPC-generated demonstrations enable FADERL to achieve performance comparable to human demonstrations. Physical experiments on fabric manipulation tasks—diagonal folding, central-axis folding, and flattening—achieve success rates of 83.3%, 80.0%, and 96.7% respectively, validating our approach’s effectiveness in real-world scenarios. Unlike computationally intensive large-model approaches, FADERL provides a lightweight, task-specific solution with efficient adaptability, making it suitable for practical robotic applications in manufacturing, medical surgery, and service robotics.

Grasping unknown objects from a single view has remained a challenging topic in robotics due to the uncertainty of partial observation. Recent advances in large-scale models have led to benchmark solutions such as GraspNet-1Billion. However, such learning-based approaches still face a critical limitation in performance robustness for their sensitivity to sensing noise and environmental changes. To address this bottleneck in achieving highly generalized grasping, we abandon the traditional learning framework and introduce a new perspective: similarity matching, where similar known objects are utilized to guide the grasping of unknown target objects. We newly propose a method that robustly achieves unknown-object grasping from a single viewpoint through three key steps: 1) Leverage the visual features of the observed object to perform similarity matching with an existing database containing various object models, identifying potential candidates with high similarity; 2) Use the candidate models with pre-existing grasping knowledge to plan imitative grasps for the unknown target object; 3) Optimize the grasp quality through a local fine-tuning process. To address the uncertainty caused by partial and noisy observation, we propose a multi-level similarity matching framework that integrates semantic, geometric, and dimensional features for comprehensive evaluation. Especially, we introduce a novel point cloud geometric descriptor, the C-FPFH descriptor, which facilitates accurate similarity assessment between partial point clouds of observed objects and complete point clouds of database models. In addition, we incorporate the use of large language models, introduce the semi-oriented bounding box, and develop a novel point cloud registration approach based on plane detection to enhance matching accuracy under single-view conditions. Real-world experiments demonstrate that our proposed method significantly outperforms existing benchmarks in grasping a wide variety of unknown objects in both isolated and cluttered scenarios, showcasing exceptional robustness across varying object types and operating environments.

Deep learning methods have recently exhibited impressive performance in object detection. However, such methods needed much training data to achieve high recognition accuracy, which was time-consuming and required considerable manual work like labeling images. In this paper, we automatically prepare training data using robots. Considering the low efficiency and high energy consumption in robot motion, we proposed combining robotic in-hand observation and data synthesis to enlarge the limited data set collected by the robot. We first used a robot with a depth sensor to collect images of objects held in the robot’s hands and segment the object pictures. Then, we used a copy-paste method to synthesize the segmented objects with rack backgrounds. The collected and synthetic images are combined to train a deep detection neural network. We conducted experiments to compare YOLOv5x detectors trained with images collected using the proposed method and several other methods. The results showed that combined observation and synthetic images led to comparable performance to manual data preparation. They provided a good guide on optimizing data configurations and parameter settings for training detectors. The proposed method required only a single process and was a low-cost way to produce the combined data. Interested readers may find the data sets and trained models from the following GitHub repository: github.com/wrslab/tubedetNote to Practitioners—The background of this study is a requirement in lab automation – Using robots to arrange randomly placed tubes automatically. Before sending test tubes to an examination machine for gradient tests, humans need to categorize and organize the tubes into specific patterns to fit the machine’s internal design. Employing humans is difficult as the tube arrangement requirements are time-varying. A preferred solution is using robots to replace humans. The robots should have a vision system to detect the tubes and a manipulation system to perform physical arranging actions. They will be used in busy seasons while deployed for other tasks in leisure time. Deep neural networks like YOLO are effective for the tube detection task. However, preparing the training data is challenging and unsuitable for lab end users. Pre-trained neural networks are options but have limited tube detection ability and cannot deal with newly included tube types. The method developed in this work helps solve the training data preparation problem. With its support, the robot can automatically prepare training data that has comparable quality to manually labeled ones in a single-process and low-cost way.

This paper presented a regrasp planning method to eliminate grasp uncertainty while considering the geometric constraints of a fixture. The method automatically finds the Stable Placement Poses (SPPs) of an object on a Triangular Corner Fixture (TCF), elevates the object from its SPPs to dropping poses and finds the Deterministic Dropping Poses (DDPs), builds regrasp graphs by using the SPP-DDP pairs and their associated grasp configurations, and searches the graph to find regrasp motion sequences for precise assembly. Since the SPPs and their associated regrasps are constrained by the TCF’s geometry and have high precision, the final object poses regrasped via it has low uncertainty and can be directly used for assembly by position control. In the experimental section, we study the performance of analytical and learning-based methods for estimating the DDPs of different objects and quantitatively examine the proposed method’s ability to suppress uncertainty using assembly tasks like peg-in-hole insertion and sheathing tubes, aligning holes, mounting bearing housings, etc. The results demonstrate the method’s robustness and efficacy. Note to Practitioners—In production lines, robots interact with peripheral devices to improve efficiency and reduce uncertainty. In this work, we focus on a particular peripheral device – a Triangular Corner Fixture (TCF) made by three inclined and mutually perpendicular plates. We study using the TCF to improve manipulation precision. The inclined plates of the TCF form a gravity bucket that holds dropped objects in stable states under gravity. In a real scenario, a robot picks up an object and releases it above the TCF. The released object will reach a stable state on the TCF. Then, the robot regrasps and moves the stabilized object to the target pose with reduced uncertainty. Using the method proposed in this paper, a robot can automatically finish the above procedure by finding all the object’s stable states in the TCF, planning grasp configurations, invalidating infeasible states and grasps, building regrasp graphs and searching the graph to find a regrasp motion sequence that moves the object to a goal pose with high precision for assembly. In industrial applications, the proposed method has the potential to improve the flexibility of robotic systems for high-precision tasks. In the research fields, it may promote the research on sensorless manipulation and extrinsic manipulation, and push forward the studies in robotic regrasp.

Elasto-plastic metal wire curving task is commonly seen in manufacturing and medical fields. This paper presents a combined task and motion planner (TAMP) for a robot arm to work aside a bending machine and carry out 3D metal wire curving tasks. We assume a collaborative robot that is safe for humans but has a weak payload and develop the combined planner for the robot to use the bending machine. The contributions of the study are three-fold. First, we propose a coarse-to-fine optimization-based method to convert a 3D curve to a structured bending set. Second, we build a planner to generate the feasible bending sequence, machine operation, robotic grasp poses, and arm motion while considering constraints from the bending machine and the robot. Third, we use visual feedback to build and dynamically update the springback model of a metal wire and use the model to predict and compensate for bending errors caused by springback. Compared with previous work, the proposed planner does not require the robot arm to have a large payload, making it suitable for lightweight collaborative robots. We evaluate the system using both simulated and real-world 3D curving tasks. The results show that the proposed planner can solve robotic 3D curving problems with satisfying time efficiency and precision. It is flexible and applicable to different robots and metal wire materials without a significant change. The method is expected to accelerate the high-variation low-volume manufacture of 3D metal wire curves.Note to Practitioners—Using robots to bend metal wires has been an old topic in robotics and automation. In previous robotic metal wire bending systems, the robot motion was usually pre-programmed to feed parts to bending machines. It was not easy to be extended to multiple goal shapes. Also, the bending was limited to a few discrete action points instead of an arbitrary curve. The method developed in this paper solved the problem by adding up optimized goal shape parameterization, combined task and motion planning, and springback compensation. It helps to auto-program robot actions and ensures satisfying precision by correcting bending results online with visual feedback. Practitioners are encouraged to use the planner for either offline programming or online motion generation. The springback estimation and compensation are independent of motion generation and can be connected to both motions pre-programmed offline or generated online. However, it is advisable to employ robots with higher DoFs (Degree of Freedom) and avoid the online planning of curves with many bending actions.

This paper presents a system integration approach for a 6-DoF (Degree of Freedom) collaborative robot to operate a pipette for liquid dispensing. Its technical development is three-fold. First, we designed an end-effector for holding and triggering manual pipettes. Second, we took advantage of direct teaching to specify global labware poses and planned robotic motion based on them. Third, we leveraged hand-mounted cameras and visual classifiers to predict and correct positioning errors, which allowed precisely attaching pipettes and tips without calibration. Through experiments and analysis, we confirmed that the developed system, especially the planning and visual recognition methods, could help secure high-precision and flexible liquid dispensing. The developed system is suitable for low-frequency, high-repetition biochemical liquid dispensing tasks. We expect it to promote the deployment of collaborative robots for laboratory automation and thus improve the experimental efficiency without significantly customizing a laboratory environment.Note to Practitioners—The proposed system helps to automate low-frequency, high-repetition biochemical experiments using a vertical articulated robot with uncalibrated hand-mounted cameras and TCP (Tool Center Point). It can be quickly deployed in tight lab spaces or beside existing automation instruments for experiments. Implementing the system does not require high-quality cameras or precise manufacturing. The system relies on software, particularly vision compensation, and the flexibility provided by unfixed racks to successfully perform pipetting tasks. We particularly recommend using collaborative robots to implement the proposed system. Collaborative robots typically meet safety requirements and do not need to be enclosed in a cage. This allows them to be deployed in the same workspace as human researchers, enabling efficient lab experiments. Human researchers can prepare tips and microwell plates and place them in front of the robot with a certain degree of freedom, which is less burdensome than traditional automatic devices that require precise placement of tips and plates in specified positions. Practitioners can view an example of the robot working with a plant phenotyping system for screening chemicals in the supplementary video.

This paper presents a stiffness-changeable soft finger using chain mail jamming. This finger can achieve adaptive grasping and in-hand manipulation by reshaping and exerting changeable gripping force. The jamming phenomenon happens when particles in a chamber get interlocked where confining pressure is exerted at their boundaries, which is widely used to construct mechanisms with changeable stiffness. Compared with the traditional granular media, chain mail has a lower packing fraction and provides a stronger tensile force. In this paper, we proposed to apply chain mail jamming to the field of robotic finger design. Especially, we propose the design of the finger, the fabrication process, the method of predicting gripping force, and the grasping strategies. The experiments quantitatively verify the model of gripping force prediction. The demonstrations validate the advantages of adaptive grasp by picking a variety of items including foods, goods, and industrial components, and show the application of in-hand manipulation.

To safely and efficiently extract an object from the clutter, this paper presents a bimanual manipulation planner in which one hand of the robot is used to slide the target object out of the clutter while the other hand is used to support the surrounding objects to prevent the clutter from collapsing. Our method uses a neural network to predict the physical phenomena of the clutter when the target object is moved. We generate the most efficient action based on the Monte Carlo tree search. The grasping and sliding actions are planned to minimize the number of motion sequences to pick the target object. In addition, the object to be supported is determined to minimize the position change of surrounding objects. Experiments with a real bimanual robot confirmed that the robot could retrieve the target object, reducing the total number of motion sequences and improving safety.

A vacuum lifter is widely used to hold and pick up large, heavy, and flat objects. Conventionally, when using a vacuum lifter, a human worker watches the state of a running vacuum lifter and adjusts the object’s pose to maintain balance. In this work, we propose using a dual-arm robot to replace the human workers and develop planning and control methods for a dual-arm robot to raise a heavy plate with the help of a vacuum lifter. The methods help the robot determine its actions by considering the vacuum lifer’s suction position and suction force limits. The essence of the methods is two-fold. First, we build a Manipulation State Graph (MSG) to store the weighted logical relations of various plate contact states and robot/vacuum lifter configurations, and search the graph to plan efficient and low-cost robot manipulation sequences. Second, we develop a velocity-based impedance controller to coordinate the robot and the vacuum lifter when lifting an object. With its help, a robot can follow the vacuum lifter’s motion and realize compliant robot-vacuum lifter collaboration. Real-world experiments are carried out to investigate the proposed planning and control methods. The results show that a robot can effectively and flexibly work together with a vacuum lifter to manipulate large and heavy plate-like objects with the methods’ support. Note to Practitioners—This paper is motivated by the vacuum lifters used for transporting heavy plates in a factory that produces building materials. In the factory, a human worker attaches the suction cup of a vacuum lifter to a plate and controls the vacuum lifter to pull the plate up. Meanwhile, another human worker moves and lifts the plate to a goal pose, following the vacuum lifter while maintaining balance. The job is dangerous as the plate is heavy, and the vacuum lifter is not always strong enough to hold the plate firmly. Inspired by the usage and safety problem, we in this paper develop a planning and control method for a dual-arm robot to replace humans. The robot coordinates its motion to work with the vacuum lifter and performs lifting tasks. The vacuum lifter could remain operated by a human worker or be actuated by signals from the dual-arm robot or other third-party machines. The work is complementary to our previous study that developed planners for robots to use pulley blocks. They together provide extensive knowledge for using low-payload collaboratively robots to manipulate heavy plates.

When humans want to understand an object’s 3D shape, they watch the object from different viewpoints. Changing the viewpoint is either performed actively, i.e., moving eye sights or the human head, or passively, i.e., holding and reposing the object. Inspired by the humans’ passive policy, we propose a method to plan the motion for a dual-arm robot to hold and repose an object, capture multiple views using a stationary depth sensor mounted on the robot head, and obtain the object’s 3D shape from the multiple views. Primarily, we develop algorithms to determine the Next Best Configuration (NBC) for observation and Next Best Regrasp/Grasp (NBR/G) poses while considering elements like the confidence of captured partial point clouds, robotic manipulability, robotic motion distances, and sensing ranges. We study the necessity and influence of these elements on the time costs and surface coverage quality in the experimental section using several representative objects. The results show that the elements play essential roles in supporting specific actions or suppressing certain costs. They help to secure efficient robot motion and satisfactory 3D shape recovery quality.Note to Practitioners—This paper is motivated by the difficulties in using commercial 3D scanners. A commercial 3D scanner set usually comprises a scanning sensor, a rotating table, and editing software. To scan the 3D shape of an object, a human needs to place the object on the rotating table with different poses, let the scanner obtain several partial point clouds, and use the editing software to merge them into a final model. The human must carefully design the different poses by considering both the object’s self obstructions and stable placements, which is tiring and difficult to be applied to large-scale tasks like building 3D shape databases containing many objects. On the other hand, although several robotic solutions exist for automatic scanning, they either use an eye-in-hand scanner to scan a stationary object or an arm to hold and move an object for scanning. In the former case, the bottom or downward faces of the object cannot be covered. In the latter case, the surface blocked by the fingers of the holding hand will be lost. The method proposed by this paper plans dual-arm robot motion to grasp and move objects for scanning. It automatically determines pick-up, rotation, and handover to maximize scanning coverage. Compared with commercial scanners and existing robotic solutions, the method performs automatic scanning with high coverage and is more advantageous for scanning many objects without human intervention.

The performance of a robot is closely related to its structure. From the initial design of link lengths to structural optimization, it is still the research hotspot in recent years. To make the manipulator lightweight and ensure its working range and flexibility, researchers have proposed many optimization methods, most of which are for specific working scenarios, requirements, and robot structures, therefore their generality is limited. The optimization of the manipulator should be a comprehensive method. That is, we should pay attention to the joint configuration and each link length at the beginning of the design. Particularly, the geometric parameters of each link, which not only affect the range of the workspace but also have a direct impact on the working space, working efficiency, and flexibility of the manipulator. In this paper, a generalized optimization framework is proposed for multi-link manipulators. Starting from the optimization of manipulator link lengths, firstly, the geometry of the manipulator and workspace is parameterized; then the performance indicators are established; lastly, the geometric size of the manipulator is optimized according to the workspace limits and task requirements. Besides, we verified its feasibility and generality by applying this method to different TBM scenarios.

This paper presents a combined task and motion planner for a robot arm to carry out 3D metal wire curving tasks by collaborating with a bending machine. We assume a collaborative robot that is safe to work in a human environment but has a weak payload to bend objects with large stiffness, and developed a combined planner for the robot to use a bending machine. Our method converts a 3D curve to a bending set and generates the feasible bending sequence, machine usage, robotic grasp poses, and pick-and-place arm motion considering the combined task and motion level constraints. Compared with previous deformable linear object shaping work that relied on forces provided by robotic arms, the proposed method is suitable for the material with high stiffness. We evaluate the system using different tasks. The results show that the proposed system is flexible and robust to generate robotic motion to corporate with the designed bending machine.

When generating robot motions based on instructions such as cooking recipes, ambiguity of the instructions and lack of necessary information are problematic for the robot. To solve this problem, we propose an efficient motion planning approach for a dual-arm robot by constructing a graph repre-senting a motion sequence based on a recipe consisting of verbal instructions and cooking images. A functional unit is generated based on the linguistic instructions in the recipe. Since most recipes lack the necessary information for executing the motion, we first consider extracting the information about the cooking motion like cutting from the food images of the recipe and supplementing it. In addition, to supplement the actions that humans perform unconsciously, we generate functional units for actions not explicitly mentioned in the recipe based on the current situation of the cooking process, and then connect them to the functional units generated from the recipe. Moreover, during the connection we consider the motion of the robot's arms in parallel for an efficient execution of the recipe, similar to those of a human. Through experiments, we demonstrate that for a given recipe, the proposed method can be used to generate a cooking sequence with the supplementary information needed, and executed by a dual-arm robot. The results show that the proposed method is effective and can simplify robot teaching in cooking tasks.

The work is inspired by the assembly of Soma block puzzles. Soma block puzzles usually include unstable intermediate states that require additional support to maintain stability temporarily. In the puzzles’ solution manual, we can observe that designers consider the characteristics that humans have two hands and can avoid an unstable intermediate state by using one hand to support the finished component and using the other hand to assemble an upcoming workpiece. Motivated by human behavior, this paper develops a planner that automatically finds an optimal assembly sequence for a dual-arm robot to build a woodblock structure while considering various constraints and supporting grasps from a second hand. It uses the mesh model of wood blocks and the final assembly state to generate possible assembly sequences and evaluate the optimal assembly sequence by considering the stability, graspability, assemblability, and the need for a second hand. Especially, the need for a second hand is resolved when supports from worktables and other workpieces are not enough to produce a stable assembly. A second hand can hold and support the unstable components so that the robot can further assemble new workpieces until the structure state becomes stable again. The output of the planner includes the optimal assembly orders, candidate grasps, assembly directions, and the supporting grasps (if needed). The output can help guide a dual-arm robot to perform motion planning and thus generate assembly motion. Experiments using various blocks and structures show the effectiveness of the proposed planner. Note to Practitioners—The presented planner can generate an optimal assembly order for a large variety of structures like decoration accessories, furniture, home interiors, frames, etc., in the practices. They can also be used for scenarios that need stacking or piling up multiple objects. The generated optimal assembly order is more friendly to dual-arm robot systems than previous assembly planners that ignored the merits of robotic collaboration. Also, the proposed assembly planner generates the necessary information for the motion planner, such as grasp poses and optimal assembly directions. A motion planner can directly use the generated results to plan robotic assembly motion.. The proposed assembly planner is expected to significantly reduce human effort and increase the efficiency of robotic assembly lines.

This paper proposes a combined planning and optimization method that enables a dual-arm robot to lift up and flip heavy plates using crane pulley blocks. The problem is motivated by the low payload of modern collaborative robots. Instead of directly manipulating heavy plates that collaborative robots cannot afford, the paper develops a planner for collaborative robots to operate crane pulley blocks. The planner assumes a target plate is pre-attached to the crane hook. It optimizes dual-arm action sequences and plans the robot’s dual-arm motion that pulls the rope of the crane pulley blocks to lift up the plate. The crane pulley blocks reduce the payload that each robotic arm needs to bear. When the plate is lifted up to a satisfying pose, the planner plans a sliding-pushing motion for one of the robot arms to tumble over the plate while considering force and moment constraints. The article presents the technical details of the planner and several experiments and analysis carried out using a dual-arm robot made by two Universal Robots UR3 arms. The influence of various parameters and optimization goals are investigated and compared in depth. The results show that the proposed planner is flexible and efficient. This paper is motivated by a cleaning process in a factory that produces sewage press machines. The pressboard of sewage press machines could be as heavy as 1000 kg. Human workers need to flip and clean both sides of the board before installing them to the main axis of a sewage machine. Their solution is using a gantry crane. They attach the board to the crane hook using bearing belts, activate the crane to lift up the board. When the board is raised to a satisfying pose, the workers turn the board over by pushing it. Motivated by human workers’ actions, we developed the planner presented in this paper. We assumed crane pulley blocks in the experiments and analysis, but in practice, they may be replaced with electronic ones to improve effort and efficiency. Using the electronic ones will be a sub-problem since there is no need for pulling ropes. The proposed method is expected to help a company’s technicians better judge if they need a heavy payload manipulator or keep their current crane equipment while employing several intelligent collaborative robots to operate them. As a result, it may help to accelerate the upgrade of manufacturing sites while reducing reforming budgets. Note to Practitioners—This paper is motivated by a cleaning process in a factory that produces sewage press machines. The pressboard of sewage press machines could be as heavy as 1000 kg. Human workers need to flip and clean both sides of the board before installing them to the main axis of a sewage machine. Their solution is using a gantry crane. They attach the board to the crane hook using bearing belts, activate the crane to lift up the board. When the board is raised to a satisfying pose, the workers turn the board over by pushing it. Motivated by human workers’ actions, we developed the planner presented in this paper. We assumed crane pulley blocks in the experiments and analysis, but in practice, they may be replaced with electronic ones to improve effort and efficiency. Using the electronic ones will be a sub-problem since there is no need for pulling ropes. The proposed method is expected to help a company’s technicians better judge if they need a heavy payload manipulator or keep their current crane equipment while employing several intelligent collaborative robots to operate them. As a result, it may help to accelerate the upgrade of manufacturing sites while reducing reforming budgets.

This article investigates the synchronization of fractional-order multi-weighted complex networks (FMWCNs) with order$\alpha \in (0,1)$. A useful fractional-order inequality${}_{t_{0}}^{C} D_{t}^{\alpha } V(x(t))\leq -\mu V(x(t))$is extended to a more general form${}_{t_{0}}^{C} D_{t}^{\alpha } V(x(t))\leq -\mu V^{\gamma }(x(t)),\gamma \in (0,1]$, which plays a pivotal role in studies of synchronization for FMWCNs. However, the inequality${}_{t_{0}}^{C} D_{t}^{\alpha } V(x(t))\leq -\mu V^{\gamma }(x(t)),\gamma \in (0,1)$has been applied to achieve the finite-time synchronization for fractional-order systems in the absence of rigorous mathematical proofs. Based on reduction to absurdity in this article, we prove that it cannot be used to obtain finite-time synchronization results under bounded nonzero initial value conditions. Moreover, by using feedback control strategy and Lyapunov direct approach, some sufficient conditions are presented in the forms of linear matrix inequalities (LMIs) to ensure the synchronization for FMWCNs in the sense of a widely accepted definition of synchronization. Meanwhile, these proposed sufficient results cannot guarantee the finite-time synchronization of FMWCNs. Finally, two chaotic systems are given to verify the feasibility of the theoretical results.

This article develops a mechanical screwing tool and its manipulation policies for two-finger parallel robotic grippers. The tool is based on a combined scissor-like element (SLE) and double-ratchet mechanism that converts the gripping motion of two-finger parallel grippers into a continuous rotation to realize tasks like fastening screws. The tool is entirely mechanical. There is no need for external cable connections. The manuscript includes two parts. For one thing, it shows the details of the tool design, optimizes the tool’s dimensions and effective stroke lengths, and studies the contacts and forces to achieve stable grasping and screwing. For another, it presents the related manipulation and control policies, including recognizing the tool, changing tool poses, and completing screw fastening tasks. The designed tool, together with the related manipulation and control policies, are analyzed and verified in several real-world applications. The results show that the tool has satisfying mechanical properties. Robots with parallel grippers can robustly and flexibly use the tool to fasten screws. The tool can also be used collaboratively with other tools to finish difficult tasks. In the future, similar tools are expected to replace special-purpose end-effectors or tool changers for more flexible robot integration.

This article develops a flexible and robust robotic system for autonomously drawing on 3-D surfaces. The system takes 2-D drawing strokes and a 3-D target surface (mesh or point clouds) as input. It maps the 2-D strokes onto the 3-D surface and generates a robot motion to draw the mapped strokes using visual recognition, grasp pose reasoning, and motion planning. The system is flexible compared to conventional robotic drawing systems as we do not fix drawing tools to the end of a robot arm. Instead, a robot recognizes and picks up pens online and holds the pens to draw 3-D strokes. Meanwhile, the system has high robustness thanks to the following crafts: First, a high-quality mapping method is developed to minimize deformation in the strokes. Second, visual detection is used to reestimate the drawing tool’s pose before executing each drawing motion. Third, force control is employed to compensate for noisy visual detection and calibration and ensure a firm touch between the pen tip and the surface. Fourth, error detection and recovery are implemented to deal with slippage and other anomalies. The planning and executions are performed in a closed-loop manner until the strokes are successfully drawn. We evaluate the system and analyze the necessity of the various crafts using different real-world tasks. The results show that the proposed system is flexible and robust to generate robotic motion that picks up the pens and successfully draws 3-D strokes on given surfaces.

Robotic assembly tasks involve complex and low-clearance insertion trajectories with varying contact forces at different stages. While the nominal motion trajectory can be easily obtained from human demonstrations through kinesthetic teaching, teleoperation, simulation, among other methods, the force profile is harder to obtain especially when a real robot is unavailable. It is difficult to obtain a realistic force profile in simulation even with physics engines. Such simulated force profiles tend to be unsuitable for the actual robotic assembly due to the reality gap and uncertainty in the assembly process. To address this problem, we present a combined learning-based framework to imitate human assembly skills through hybrid trajectory learning and force learning. The main contribution of this work is the development of a framework that combines hierarchical imitation learning, to learn the nominal motion trajectory, with a reinforcement learning-based force control scheme to learn an optimal force control policy. To further improve the imitation learning part, we develop a hierarchical architecture, following the idea of goal-conditioned imitation learning, to generate the trajectory learning policy on the skill level offline. Through experimental validations, we corroborate that the proposed learning-based framework is robust to uncertainty in the assembly task, can generate high-quality trajectories, and can find suitable force control policies, which adapt to the task’s force requirements more efficiently.

This paper presents a motion planning algorithm that enables robots to efficiently pick up objects by considering simultaneous multi-object grasping. At the center of the algorithm is a cost function that helps to determine one of the following three grasping policies considering distance and friction constraints – Grasping a single object; Grasping two objects simultaneously; Grasping two object simultaneously after pushing one of the objects close to the other. After recognizing the object distributions on a table by using a depth camera and Mask R-CNN, our algorithm will select grasp policies from the three candidates considering the cost function, and plan a policy sequence that can most quickly finish picking all the objects using dynamic programming. Both simulation and real-world experiments are carried out to examine the performance of the proposed planner. Results show that the proposed method significantly improves the efficiency of robotic picking compared to conventional single-object-based methods.

This paper proposes a robot assembly planning method by automatically reading the graphical instruction manuals designed for humans. Essentially, the method generates an Assembly Task Sequence Graph (ATSG) by recognizing a graphical instruction manual. An ATSG is a graph describing the assembly task procedure by detecting types of parts included in the instruction images, completing the missing information automatically, and correcting the detection errors automatically. To build an ATSG, the proposed method first extracts the information of the parts contained in each image of the graphical instruction manual. Then, by using the extracted part information, it estimates the proper work motions and tools for the assembly task. After that, the method builds an ATSG by considering the relationship between the previous and following images, which makes it possible to estimate the undetected parts caused by occlusion using the information of the entire image series. Finally, by collating the total number of each part with the generated ATSG, the excess or deficiency of parts are investigated, and task procedures are removed or added according to those parts. In the experiment section, we build an ATSG using the proposed method to a graphical instruction manual for a chair and demonstrate the action sequences found in the ATSG can be performed by a dual-arm robot execution. The results show the proposed method is effective and simplifies robot teaching in automatic assembly.

To cope with the requirements on efficiency and labour-saving of the out-pipe surface grinding tasks in the wild, several proposals are revealed and discussed. The one benefiting from the characteristics of planetary gear transmission and friction actuating mechanism are expatiated. To realize full coverage of out-pipe surface, the self-rotation and revolution motions of every polishing tool (cutter) are actuated by the same motor, and the friction force produced in grinding process acts as suitable tractive force for the forward travel of the grinding system. The friction statics analysis is established to illustrate the force transmission. Compression spring system are utilized to realize force equilibrium and support passive diameter adaptability. The proposed robotic grinding system is characterized by less actuator, online grinding capability and high working efficiency. It has clear advantages regarding manufacturing costs and control complexity. As the result of prototype experiments, performance of smooth grinding the out-pipe surface is confirmed.

This article develops model-based grasp planning algorithms. It focuses on industrial end-effectors like grippers and suction cups, and plans grasp configurations considering computer aided design (CAD) models of target objects. The developed algorithms can stably find many high-quality grasps, with satisfying precision and little dependency on the quality of CAD models. The undergoing core technique is superimposed segmentation, which preprocesses a mesh model by peeling it into superimposed facets. The algorithms use the facets to locate contacts and synthesize grasp poses for popular industrial end-effectors. Several tunable parameters are prepared to adapt the algorithms to meet various requirements. The experimental section studies the influence of the tunable parameters and analyzes the cost, precision, and robustness of the proposed algorithms and their planned grasps, with both simulations and real-world systems. Besides, the proposed algorithms are applicable to mesh models reconstructed from point clouds obtained by depth sensors. Some experiments and analysis are also carried out to study and demonstrate the ability.

The automatic parts feeding of multiple objects is an unsolved problem in the manufacturing industry. In this paper, we tackle the problem by proposing a multi-robot system. The system comprises three sub-components which perform bin-picking, regrasping, and kitting. The three subcomponents divide and conquer the automatic multiple parts feeding problem by considering a coarse-to-fine manipulation process. Multiple robot arms are connected in series as a pipeline. The robots are separated into three groups to perform the roles of each sub-component. The accuracy of the state and manipulation are getting higher along with the changes of the sub-components in the pipeline. In the experimental section, the performance of the system is evaluated by using the Mean Picks Per Hour (MPPH) metric and success rate, which are compared to traditional parts feeder and manual labor. The results show that the Mean Picks Per Hour (MPPH) of the proposed system is 351 with eleven various-shaped industrial parts, which is faster than the state-of-the-art robotic bin-picking system. The lead time of the proposed system for new parts is less than that of a traditional parts feeders and/or manual labor.

This paper considers basket grasps, where a two-finger robot hand forms a basket that can safely lift and carry rigid objects in a 2-D gravitational environment. The two-finger basket grasps form special points in a high-dimensional configuration space of the object and two-finger robot hand. This paper establishes that all two-finger basket grasps can be found in a low-dimensional contact space that parametrizes the two-finger contacts along the supported object boundary. Using contact space, each basket grasp is associated with its depth that provides a security measure while carrying the object, as well as its safety margin away from a critical finger opening where the object drops-off into its intended destination. Geometric techniques that compute the depth and drop-off finger opening are described and illustrated with detailed graphical and numerical examples.

In this paper, we address efficiently and robustly collecting objects stored in different trays using a mobile manipulator. A resolution complete method, based on precomputed reachability database, is proposed to explore collision-free inverse kinematics (IK) solutions and then a resolution complete set of feasible base positions can be determined. This method approximates a set of representative IK solutions that are especially helpful when solving IK and checking collision are treated separately. For real world applications, we take into account the base positioning uncertainty and plan a sequence of base positions that reduce the number of necessary base movements for collecting the target objects, the base sequence is robust in that the mobile manipulator is able to complete the part-supply task even there is certain deviation from the planned base positions. Our experiments demonstrate both the efficiency compared to regular base sequence and the feasibility in real world applications.

Synergy provides a practical approach for expressing various postures of a multi-fingered hand. However, a conventional synergy defined for reproducing grasping postures cannot perform in-hand manipulation, e.g., tasks that involve simultaneously grasping and manipulating an object. Locking the position of particular fingers of a multi-fingered hand is essential for in-hand manipulation tasks either to hold an object or to fix unnecessary fingers. When using conventional synergy based control to manipulate an object, which requires locking some fingers, the coordination of joints is heavily restricted, decreasing the dexterity of the hand. We propose a functionally divided manipulation synergy (FDMS) method, which provides a synergy-based control to achieves both dimensionality reduction and in-hand manipulation. In FDMS, first, we define the function of each finger of the hand as either "manipulation" or "fixed." Then, we apply synergy control only to the fingers having the manipulation function, so that dexterous manipulations can be realized with a few control inputs. Furthermore, we propose the Synergy Switching Framework as a method for applying a finely defined FDMS to sequential task changes. The effectiveness of our method is experimentally verified.

This article develops a robotic manipulation planner for human-robot collaborative assembly. Unlike previous methods that study an independent and fully artificial intelligent (AI)-equipped autonomous system, this article explores the subtask distribution between a robot and a human and studies a human-in-the-loop robotic system for collaborative assembly. The system distributes the subtasks of an assembly to robots and humans by exploiting their advantages and avoiding their disadvantages. The robot in the system will work on pick-and-place tasks and provide workpieces to humans. The human collaborator will work on fine operations, such as aligning, fixing, and screwing. A constraint-based incremental manipulation planning method is proposed to generate the motion for the robots. The performance of the proposed system is demonstrated by asking a human and the dual-arm robot to collaboratively assemble a cabinet. The results show that the proposed system and planner are effective and efficient and can assist humans in finishing the assembly task comfortably.

This paper presents a manipulation planning algorithm for robots to reorient objects. It automatically finds a sequence of robot motion that manipulates and prepares an object for specific tasks. Examples of the preparatory manipulation planning problems include reorienting an electric drill to cut holes, reorienting workpieces for assembly, and reorienting cargo for packing, etc. The proposed algorithm could plan single- and dual-arm manipulation sequences to solve the problems. The mechanism under the planner is a regrasp graph, which encodes grasp configurations and object poses. The algorithms search the graph to find a sequence of robot motion to reorient objects. The planner is able to plan both single- and dual-arm manipulation. It could also automatically determine whether to use a single arm, dual arms, or their combinations to finish given tasks. The planner is examined by various humanoid robots like Nextage, HRP2Kai, HRP5P, etc., using both simulation and real-world experiments.

Most online initialization and self-calibration methods for visual-inertial odometry (VIO) are only able to estimate the extrinsic parameters (orientation and translation) between one camera and inertial measurement unit (IMU) pair. They are not applicable to stereo VIO where both camera-IMU and camera-camera pairs exist. In this article, we address the issue by taking advantage of the geometric constraints among the multiple sensors. An online method is proposed to estimate the initial values of velocity, gravity, IMU biases, and simultaneously calibrate the extrinsic parameters of camera-camera and camera-IMU pairs for bootstrapping a smoothing-based stereo VIO system. The method includes a three-step process to incrementally solve several linear equations in a coarse-to-fine manner. It back-propagates historically estimated results to update weight factors and remove outliers, and employs a convergence criterion to monitor and terminate the process. It also includes an optional global optimization for further refinement. The method is evaluated in terms of accuracy, robustness, convergence, consistency, and tunable parameters using both simulated and public datasets. Experimental results show that the proposed method can accurately estimate the initial values and the extrinsic parameters.

In this paper, we propose a control algorithm to collectively transport an object using a group of relatively low-cost robots. We address this problem using the robust caging, which features reliable object closure with minimum number of robots, and requires no high-precision control capability on the individual robot. Given a 2-D convex object, the proposed method uses the quality of complete robustness to first optimize the number of robots in the initial formation, and then reorient and move the formation. The method is free of force analysis, and therefore less prone to sensor errors and failures. Compared with state-of-the-art multirobot object transport approaches, which require more robots and rely heavily on high-precision control, such as force and torque feedback control, our method uses fewer robots and has high tolerance to control noises. We performed both simulation and real-time experiments to demonstrate the performance of our method. We conclude that the proposed robust caging is promising under reduced number of robots and a certain level of control noises in multirobot object transport tasks.

In this research, we tackle the challenge of picking only one object from a randomly stacked pile where the objects can potentially be tangled. No solution has been proposed to solve this challenge due to the complexity of picking one and only one object from the bin of tangled objects. Therefore, we propose a method for avoiding the situation where a robot picks multiple objects. In our proposed method, first, grasping pose candidates are computed by using the graspability index. Then, a Convolutional Neural Network (CNN) is trained to predict whether or not the robot can pick one and only one object from the bin. Additionally, since a physics simulator is used to collect data to train the CNN, an automatic picking system can be built. The effectiveness of the proposed method is confirmed through experiments on robot Nextage and compare with previous bin-picking methods.

Planning dual-arm assembly of more than three objects is a challenging Task and Motion Planning (TAMP) problem. The assembly planner shall consider not only the pose constraints of objects and robots, but also the gravitational constraints that may break the finished part. This paper proposes a planner to plan the dual-arm assembly of more than three objects. It automatically generates the grasp configurations and assembly poses, and simultaneously searches and backtracks the grasp space and assembly space to accelerate the motion planning of robot arms. Meanwhile, the proposed method considers gravitational constraints during robot motion planning to avoid breaking the finished part. In the experiments and analysis section, the time cost of each process and the influence of different parameters used in the proposed planner are compared and analyzed. The optimal values are used to perform real-world executions of various robotic assembly tasks. The planner is proved to be robust and efficient through the experiments.

This paper explores the use of robots to autonomously assemble parts with variations in colors and textures. Specifically, we focus on peg-in-hole assembly with some initial position uncertainty and holes located on surfaces of different colors and textures. Two in-hand cameras and a force-torque sensor are used to account for the position uncertainty. A program sequence comprising learning-based visual servoing, spiral search, and impedance control is implemented to perform the peg-in-hole task with feedback from the above sensors. Contributions are mainly made in the learning-based visual servoing component of the sequence, where a deep neural network is trained with various sets of synthetic data generated using the concept of domain randomization to predict where a hole is. In the experiments and analysis section, the network is analyzed and compared, and a real-world robotic system to insert pegs to holes using the proposed method is implemented. The results show that the implemented peg-in-hole assembly system can perform successful peg-in-hole insertions on surfaces with various colors and textures. It can generally speed up the entire peg-in-hole process, especially when the initial position uncertainty is large.

This paper develops an adaptive gripper for peg-in-hole tasks. Conventional grippers require complicated compliant mechanisms or complicated control strategy and force sensing to successfully insert pegs into holes. Different from them, this paper proposes a simple gripper with an L-shape finger as a low-cost peg-in-hole solution. The basic idea is to divide a peg-in-hole process into a preparation phase and an execution phase, and eliminate uncertainty step-by-step by pushing using the L-shape finger in the preparation phase. The robustness of the gripper for peg-in-hole tasks is examined by repeated executions for different pegs in the International Robotic Exhibition 2017 (IREX) in Tokyo. The experimental section presents details of the executions, and qualitatively shows the high performance of the proposed gripper.

This paper presents an efficient algorithm to test whether a planar object can be caged by a formation of point agents (point fingertips or point mobile robots). The algorithm is based on a space mapping between the 2-D work space (W space) and the 3-D configuration space (C space) of the given agent formation. When performing caging test on a planar object, the algorithm looks up the space mapping to recover the C space of the given agent formation, labels the recovered C space, and counts the number of labeled surfaces to judge the success of caging. The algorithm is able to work with various planar shapes, including objects with convex boundaries, concave boundaries, or holes. It can also respond quickly to varying agent formations and different object shapes. Experiments and analysis on different objects and fingertip formations demonstrate the completeness, robustness, and efficiency of our proposal.

This paper develops intelligent algorithms for robots to reorient objects. Given the initial and goal poses of an object, the proposed algorithms plan a sequence of robot poses and grasp configurations that reorient the object from its initial pose to the goal. While the topic has been studied extensively in previous work, this paper makes important improvements in grasp planning by using over-segmented meshes, in data storage by using relational database, and in regrasp planning by mixing real-world roadmaps. The improvements enable robots to do robust regrasp planning using 10,000s of grasps and their relationships in interactive time. The proposed algorithms are validated using various objects and robots.

Robots are increasingly operating in indoor environments designed for and shared with people. However, robots working safely and autonomously in uneven and unstructured environments still face great challenges. Many modern indoor environments are designed with wheelchair accessibility in mind. This presents an opportunity for wheeled robots to navigate through sloped areas while avoiding staircases. In this paper, we present an integrated software and hardware system for autonomous mobile robot navigation in uneven and unstructured indoor environments. This modular and reusable software framework incorporates capabilities of perception and navigation. Our robot first builds a 3D OctoMap representation for the uneven environment with the 3D mapping using wheel odometry, 2D laser and RGB-D data. Then we project multilayer 2D occupancy maps from OctoMap to generate the the traversable map based on layer differences. The safe traversable map serves as the input for efficient autonomous navigation. Furthermore, we employ a variable step size Rapidly Exploring Random Trees that could adjust the step size automatically, eliminating tuning step sizes according to environments. We conduct extensive experiments in simulation and real-world, demonstrating the efficacy and efficiency of our system. (Supplemented video link: https://youtu.be/6XJWcsH1fk0).

Pick-and-place regrasp is an important manipulation skill for a robot. It helps a robot accomplish tasks that cannot be achieved within a single grasp, due to constraints such as kinematics or collisions between the robot and the environment. Previous work on pick-and-place regrasp only leveraged flat surfaces for intermediate placements, and thus is limited in the capability to reorient an object. In this paper, we extend the reorientation capability of a pick-and-place regrasp by adding a vertical pin on the working surface and using it as the intermediate location for regrasping. In particular, our method automatically computes the stable placements of an object leaning against a vertical pin, finds several force-closure grasps, generates a graph of regrasp actions, and searches for the regrasp sequence. To compare the regrasping performance with and without using pins, we evaluate the success rate and the length of regrasp sequences while performing tasks on various models. Experiments on reorientation and assembly tasks validate the benefit of using support pins for regrasping.

We present a novel method for caging grasps in this paper by stretching ropes on the surface of a 3D object. Both topology and shape of a model to be grasped has been considered in our approach. Our algorithm can guarantee generating local minimal rings on every topological branches of a given model with the help of a Reeb graph. Cages and grasps can then be computed from these rings, and physical experimental tests have been conducted to verify the robustness of our approach.

Pick-and-place regrasp extends the manipulation capability of a robot by using a sequence of regrasps to accomplish tasks that are not possible using a single grasp due to constraints such as kinematics or collisions between the robot and the environment. Previous work on pick-and-place only leveraged static passive devices for intermediate placements, and thus is limited in the flexibility and robustness to reorient an object. In this paper, we extend the reorientation capability of a pick-and-place regrasp by adding an actively actuated gripper fixed in the working cell, and using it as the intermediate location for regrasping. In particular, our method automatically computes the stable placements of an object being hold in the gripper support, finds a rich set of force-closure grasps, performs k-means based grasp clustering, generates a graph of regrasp actions, and searches for the optimal regrasp sequence. To compare the regrasping performance with typical passive supports, we evaluate the success rate while performing tasks on various models. Experiments on reorientation tasks validate the benefit of using an actively actuated gripper for regrasp placement.

The goal of this paper is to develop a regrasp planning algorithm general enough to perform statistical analysis with thousands of experiments and arbitrary mesh models. We focus on pick-and-place regrasp which reorients an object from one placement to another by using a sequence of pick-ups and place-downs. We improve the pick-and-place regrasp approach developed in 1990s and analyze its performance in robotic assembly with different work surfaces in the workcell. Our algorithm will automatically compute the stable placements of an object, find several force-closure grasps, generate a graph of regrasp actions, and search for regrasp sequences. We demonstrate the advantages of our algorithm with various mesh models and use the algorithm to evaluate the completeness, the cost and the length of regrasp sequences with different mesh models and different assembly tasks in the presence of different work surfaces. Our results show that spare work surfaces are beneficial to assembly. Tilted work surfaces are only sometimes beneficial, depending on the objects.

“Grasping by caging” has been considered as a powerful tool to deal with uncertainty. In this paper, we continue to explore into “grasping by caging” and propose a new solution by using eigen-shapes and space mapping. For one thing, eigen-shapes fix dexterous hands into a series of finger formations and help to reduce dimensionality and computational complexity. For the other, space mapping builds a mapping between rasterized grids in 2-D Work space (W space) and rasterized voxels in 3-D Configuration space (C space) and helps to rapidly reconstruct C space so that we can efficiently measure the robustness of caging and find an optimal caging configuration for grasping. Our algorithm can work rapidly and squeezingly cage any 2-D shapes, including objects with either convex boundaries, concave boundaries, 1-order or high-order boundaries and even objects with inner holes. We implement the algorithm with MATLAB and carry out experiments with WEBOTS simulation to test its robustness to uncertainties. The results show that our algorithm can work well with various object shapes and can be robust to noisy control and noisy perception. It is promising in the power grasping tasks of dexterous hands.

Caging can offer robustness to uncertainties in grasping. If a robotic hand is designed based on the idea of caging, it would probably work well with noisy perception devices and low-quality control. This paper takes into account these merits and designs and implements a gripping hand based on the idea of caging. The gripping hand is concise and offers a low-cost alternative to co-operate with noisy data and low-quality control. According to previous work, we need four fingers to cage any 2D objects. That is to say, if each finger has one, two or three degree of freedoms, we will totally need four, eight or twelve actuators. The large number of actuators would be costly. This paper simplify the number of actuators into one by quantitatively analyzing finger formations with caging tests conducted on both random objects and objects from MPEG-7 shape database. It successfully lowers costs while maintains high performance. Following the simplified one-actuator design we implement a gripping hand by modifying a SCHUNK RH707 hand and carried out experiments with a manipulator built on the Neuronics Katana arm. The one-actuator gripping hand could work well with common depth cameras (Swiss Ranger) and pick up various objects. It bridges the gap between caging theories and applications and demonstrates the merits of caging.

This paper presents a caging approach which deals with planar boundary clouds collected from a laser scanner. Given the boundary clouds of a target object and two fixed finger positions, our aim is to find potential third finger positions that can prevent target from escaping into infinity. The major challenge in working with boundary clouds lies in their uncertainty in geometric model fitting and the failure of critical orientations. In this paper, we track canonical motions according to the rotational intersection of Configuration space fingers and rasterize Work space with grids to compute the third caging positions. Our approach can generate the capture region with max(O(np),O(h2)) ≤ O(n2) cost where n denotes the resolution of grid rasterization, p denotes the resolution of canonical rasterization and h denotes the resolution of boundary rasterization or the number of boundary cloud points. Moreover, we propose a rough approximation which measures a subset of the possible positions by contracting rotations, indicating computational complexity of max(O(n),O(h2)). In the experimental part, our proposal is compared with state-of-the-art works and applied to many other objects. The approach makes caging fast and effective.

This paper presents a novel approach to deal with uncertainty in grasping. The basic idea is to initiate a caging manipulation state and then shrink fingers into immobilization to perform a practical grasping. Thanks to flexibility from caging, this procedure is intrinsically safe and gains tolerance towards uncertainty. Besides, we demonstrate that the minimum caging is immobilization and consequently propose using three or four fingers to manipulate planar convex objects in a grasping-by-caging way. Experimental results with physical simulation show the robustness and efficacy of our approach. We expect its leading benefits in saving finger number, conquering low-friction materials and especially, dealing with pose/shape uncertainty.

Although lots of planning algorithms have focused on the planning of fixed manipulators and mobile robots in moderate dynamic environments, seldom planning algorithms can be employed to deal with mobile agents in the presence of large scenario scales and unpredictable changing obstacles. Path planning for mobile robots in unpredictable environments would be an extreme challenge since computational complexity increase dramatically with high dimensionality, unpredictability and large scales. In this paper, a novel and real-time approach is proposed to solve this problem by generating subgoals dynamically according to time and potential values. This dynamic subgoal based approach includes two procedures, the subgoal generator and the inter-subgoal or inner replanner. On the one hand, a set of high-level subgoals is generated dynamically by an improved single shot strategy that could tailor itself adaptively. On the other hand, a roadmap is built during the preprocessing phase by employing a localized Dynamic Roadmap Mapping (local-DRM) for inter-subgoal replanning. Finally these two procedures will collaborate according to the potential field criterion to ensure completeness. Our approach can not only generate paths rapidly enough to satisfy the requirements of an anytime planer but also work for large scenario scales. Experimental results on different kinds of mobile agents, in large scenario scales and in the presence of unpredictable changing obstacles show that our approach can find out a collision free path on an average of 0.11s for a single planning, indicating an anytime planner.

Replanning is a powerful tool for high dimensional mobile agents in changing environments. However, most works employ replanning periodically. In order to fully exert the merits of this powerful tool, we should concentrate on the time interval employed for each replanning (that is “when to replan”) and carry out replanning adaptively. In this paper, an adaptive strategy is proposed to govern replanning in hard changing environments. The key point of this adaptive replanning strategy is to perform local environment accumulation by using grids method, which is a derivative of degenerated potential field. Since the accumulation is only performed locally in the regions between subgoals and only computed towards the changes of obstacles, it increases little computational complexity to parent anytime planners. Our adaptive replanning strategy works as a plug-in to state-of-the-art algorithms and can generate heuristics by using information from projected spaces to overcome high dimensionality. Experiments on different mobile agents in various hard changing environments (environments with crowded and unforseen obstacles) with IDRM-gRRT and IRRT-gRRT showed that the adaptive strategy can improve the performance and robustness of parent anytime planners significantly.

Although primates can facilely maintain long-duration tracking of an object without infection of occlusion or other near similar distracters, it remains a challenge for computer vision system. Studies in psychology suggest that the ability of primates to focus selective attention on the spatial properties of an object is necessary to observe object quickly and efficiently while focus selective attention on the feature properties of object is necessary to render it more prominent from the distracters. In this paper, we propose a novel spatial-feature attentional visual tracking (SFAVT) algorithm to encode both. In SFAVT, tracking is treated as an on-line binary classification problem where spatial attention is employed in early selective procedure to construct foreground/background appearance model by identifying image patches with good localization properties, and in late selective procedure to update models by maintaining image patches with good discrimitive motion properties. Meanwhile, feature attention works in mode seeking procedure to help select feature spaces that best separate a target from background. The on-line tuned adaptive appearance models by those selected feature spaces are used to train a classifier for target localization, then. Experiments under various real-world conditions show that this algorithm is able to track an object influenced by dramatic distracters while is of comparable time efficiency with meanshift.

Robotic path planning in changing environments with difficult regions is an extremely challenge. Since the structure of configuration space (C-space) will change when obstacles move in workspace (W-space), the planner should have the capacity of building approximate structure of C-space, while avoiding intense computational complexity. Further, difficult regions will also change their positions, which requires the planner should be able to identify them fast and increase the free nodes inside them efficiently. This paper presents a novel approach for path planning in changing environments using predictive model, which is inspired by the idea of active learning. With the help of W-C nodes mapping, this predictive model is built to capture the approximate structure of C-space, while avoiding intense computational complexity. This model include two steps: K-near Dynamic Bridge Builder (K-near DBB) is proposed to identify difficult passages in the space first, and then Inner Parzen Window is adopted to sample points in these difficult regions without invoking any collision checker. Experiments are carried out with two 6-DOF manipulators, and our approach can find a path with high time efficiency and low error rate, even if the environment is complex.