Exploring the relationship among objects and giving the correct operation sequence is vital for robotic manipulation. However, most previous algorithms only model the relationship between pairs of objects independently, ignoring the interaction effect between them, which may generate redundant or missing relations in complex scenes, such as multi-object stacking and partial occlusion. To solve this problem, a Gated Graph Neural Network (GGNN) is designed for visual manipulation relationship detection, which can help robots detect targets in complex scenes and obtain the appropriate grasping order. Firstly, the robot extracts feature from the input image and estimate object categories. Then GGNN is used to effectively capture the dependencies between objects in the whole scene, update the relevant features, and output the grasping sequence. In addition, by embedding positional encoding into pair object features, accurate context information is obtained to reduce the adverse effects of complex scenes. Finally, the constructed algorithm is applied to the physical robot for grasping. Experiment results on the Visual Ma-nipulation Relationship Dataset (VMRD) and the large-scale relational grasp dataset named REGRAD show that our method significantly improves the accuracy of relationship detection in complex scenes, and can be well generalized in the real world.

Business organization performs its business activities within a headquarter-branch relationship, especially for multinationals, by establishing work places in accordance with business requirements. The data-driven decision making stimulates the importance of data analysis. Machine learning (ML), as a method of data analysis, automates analytical model building based on the idea that systems can train and learn from data to make decisions with minimal human intervention. For high accuracy, traditional ML algorithms make design tradeoffs, conceding privacy and reliability, and are thereby unable to satisfy strong security demands. To resolve design tensions, in this article, we propose RePEL, which harmonizes functional encryption (FE) and blockchain on top of encrypted learning. RePEL allows the headquarter to only share partial information about business data collected from branches while manages data transfer with the consensus mechanism in the blockchain, which works in a coordinated way for preserving conditional privacy and high reliability. We instantiate a RePEL design with the three-layer framework and formally reduce its security to a provably secure FE scheme. We further deploy Feel_BC to implement a RePEL prototype system, so as to realize the performance evaluation. Experimental results show that RePEL, with the basic premise of privacy and reliability, can achieve high accuracy and reasonable throughputs.

Autonomous tool construction is a significant but challenging task in robotics. This task can be interpreted as when given a reference tool, selecting some available candidate parts to reconstruct it. Most of the existing works perform tool construction in the form of action part and grasp part, which is only a specific construction pattern and limits its application to some extent. In general scenarios, a tool can be constructed in various patterns with different part pairs. Therefore, whether a part pair is most suitable for constructing the tool depends not only on itself, but on other parts in the same scene. To solve this problem, we construct a Gated Graph Neural Network (GGNN) to model the relations between all part pairs, so that we can select the candidate parts in consideration of the global information. Afterwards, we embed the constructed GGNN into a RCNN-like structure to finally accomplish tool construction. The whole model will be named Tool Construction Graph RCNN (TC-GRCNN). In addition, we develop a mechanism that can generate large-scale training and testing data in simulation environments, by which we can save the time of data collection and annotation. Finally, the proposed model is deployed on the physical robot. The experiment results show that TC-GRCNN can perform well in the general scenarios of tool construction.

In task-oriented grasping, the robot is supposed to manipulate the objects in a task-compatible manner, which is more important but more challenging than just stably grasping. However, most of existing works perform task-oriented grasping only in single object scenes. This greatly limits their practical application in real world scenes, in which there are usually multiple stacked objects with serious overlaps and occlusions. To perform task-oriented grasping in object stacking scenes, in this paper, we firstly build a synthetic dataset named Object Stacking Grasping Dataset (OSGD) for task-oriented grasping in object stacking scenes. Secondly, a Conditional Random Field (CRF) is constructed to model the semantic contents in object regions. The modelled semantic contents can be illustrated as incompatibility of task labels and continuity of task regions. This proposed approach can greatly reduce the interference of overlaps and occlusions in object stacking scenes. To embed the CRF-based semantic model into our grasp detection network, we implement the inference process of CRFs as a RNN so that the whole model, Task-oriented Grasping CRFs (TOG-CRFs) can be trained end to end. Finally, in object stacking scenes, the constructed model can help robot achieve 69.4% success rate for task-oriented grasping.

Autonomous robotic grasping plays an important role in intelligent robotics. However, how to help the robot grasp specific objects in object stacking scenes is still an open problem, because there are two main challenges for autonomous robots: (1) it is a comprehensive task to know what and how to grasp; (2) it is hard to deal with the situations in which the target is hidden or covered by other objects. In this paper, we propose a multi-task convolutional neural network for autonomous robotic grasping, which can help the robot find the target, make the plan for grasping and finally grasp the target step by step in object stacking scenes. We integrate vision-based robotic grasping detection and visual manipulation relationship reasoning in one single deep network and build the autonomous robotic grasping system. Experimental results demonstrate that with our model, Baxter robot can autonomously grasp the target with a success rate of 90.6%, 71.9% and 59.4% in object cluttered scenes, familiar stacking scenes and complex stacking scenes respectively.

Trajectory-based methods offer an effective methodology for generating the reduced-order models (ROMs) for nonlinear systems. These methods first sample on the state trajectories driven by the training inputs, then linearize and reduce the linearized systems around the sample points. However, these methods depend on an single global reduction subspace generated by combining all the projection subspaces of the sample points on the trajectories. In order to address this problem, a localized reduction technique has been proposed. This method weaves together a larger set of smaller localized ROMs for the trajectory samples. However, since these localized ROMs do not share the same coordinates, these localized ROMs cannot be interpolated to derive new ROMs. As a result, a large number of localized ROMs are needed to cover the necessary state space and guarantee adequate reduction accuracy. In this paper, we propose a new, efficient trajectory-based model order reduction algorithm for nonlinear systems via localized projection and global interpolation. We employ an efficient procedure to transform the smaller localized ROMs into a set of equivalent ROMs with nearly consistent global coordinate. The ROMs for the nonlinear systems are then obtained by globally interpolating the localized ROMs. Because we can perform interpolation between these localized ROMs, the required number of localized ROMs can be greatly reduced.