The usage of 3D vision algorithms, such as shape reconstruction, remains limited because they require inputs to be at a fixed canonical rotation. Recently, a simple equivariant network, Vector Neuron (VN) has been proposed that can be easily used with the state-of-the-art 3D neural network (NN) architectures. However, its performance is limited because it is designed to use only three-dimensional features, which is insufficient to capture the details present in 3D data. In this paper, we introduce an equivariant feature representation for mapping a 3D point to a high-dimensional feature space. Our feature can discern multiple frequencies present in 3D data, which, as shown by Tancik et al. (2020), is the key to designing an expressive feature for 3D vision tasks. Our representation can be used as an input to VNs, and the results demonstrate that with our feature representation, VN captures more details, overcoming the limitation raised in its original paper.

The object manipulation is a crucial ability for a service robot, but it is hard to solve with reinforcement learning due to some reasons such as sample efficiency. In this paper, to tackle this object manipulation, we propose a novel framework, AP-NPQL (Non-Parametric Q Learning with Action Primitives), that can efficiently solve the object manipulation with visual input and sparse reward, by utilizing a nonparametric policy for reinforcement learning and appropriate behavior prior for the object manipulation. We evaluate the efficiency and the performance of the proposed AP-NPQL for four object manipulation tasks on simulation (pushing plate, stacking box, flipping cup, and picking and placing plate), and it turns out that our AP-NPQL outperforms the state-of-the-art algorithms based on parametric policy and behavior prior in terms of learning time and task success rate. We also successfully transfer and validate the learned policy of the plate pick-and-place task to the real robot in a sim-to-real manner.

In this paper, we propose a novel sim-to-real framework to solve bolting tasks with tight tolerance and complex contact geometry which are hard to be modeled. The sim-to-real has desirable features in terms of cost and safety, however, that of the assembly task is rare due to the lack of simulator, which can robustly render multi-contact assembly. We implement the sim-to-real transfer of nut tightening policy which is adaptive to uncertain bolt positions. This can be realized through developing a novel contact model, which is fast and robust to complex assembly geometry, and novel hierarchical controller with reinforcement learning (RL), which can perform the tasks with a narrow and complicated path. The fast and robust contact model is achieved by utilizing configuration space abstraction and passive midpoint integrator (PMI), which render the simulator robust even in a high stiffness contact condition. And we use sampling-based motion planning to construct a path library and design linear quadratic tracking controller as a low-level controller to be compliant and avoid local optima. Additionally, we use the RL agent as a high-level controller to make it possible to adapt to the bolt position uncertainty, thereby realizing sim-to-real. Experiments are performed to verify our proposed sim-to-real framework.

We propose a novel data-driven learning-based contact clustering (i.e., of contact points and contact normals) framework for rigid-body robot simulation, with its accuracy established/verified by real experimental data. We first construct an experimental robotic setup with force/torque (F/T) sensors to collect real contact motion/force data. We then design a multilayer perceptron (MLP) network for the contact clustering based on the full motion and force/torque information of the contacts. We also adopt the constraint-based optimization contact solver to facilitate the learning of our MLP network during the training. Our proposed data-driven/learning-based contact clustering framework is then verified against the experimental setup, compared with other techniques/simulators and shown to significantly (or meaningfully) enhance the accuracy of contact simulation as compared to them.

Electrical motor and hydraulic actuation widely-used in robotics are “internal actuation” with their actuators sitting at the joint between two links. This internal actuation is fundamentally limiting to construct a large-size dexterously-articulated robot, since any external force (and its own link weight) is to be accumulated to the base multiplied by the moment arm length, requiring extremely strong/sturdy base actuator/structure as the system size increases. In this paper, we propose a novel robotic system, LASDRA (large-size aerial skeleton with distributed rotor actuation), which, by utilizing distributed rotors as “external actuation”, can overcome this limitation of internal actuation and enables us to realize large-size dexterously-articulated robots. We present its design and modeling, joint locking strategy to increase its loading capability, and also a novel decentralized control scheme to allow for compliant operation with scalability against the number of links. Trajectory tracking and valve turning experiments are also performed to validate the theory.