Bin-packing is an important problem in the robotic warehouse domain. Traditionally, this problem has been studied only for rigid packages (e.g., boxes or rigid objects). In this work, we tackle the problem of bin-packing with deformable packages that have become a popular choice for fulfillment needs. We present a system that incorporates a dual robot arm bimanual setup, uniquely combining suction and sweeping motions to stably and reliably pack deformable packages in a bin. Additionally, we propose a comprehensive action prediction framework to optimize for bin-packing efficiency by predicting optimal actions for both robots involved. Our methodology leverages a two-pronged learning strategy, where initially, we train a model in a self-supervised manner to predict a scoring metric indicative of bin-packing efficiency and then leverage an online optimization scheme to compute optimal actions in real time. The model is pre-trained in simulation in MuJoCo and fine-tuned on small-scale data from a real-world laboratory setting. Our packing score prediction model predicts bin-packing score ∈ [0, 1] with an MSE of 0.003. Real-world experiments validate our method’s adaptability to novel scenarios and its effectiveness in packing operations. Project Website: https://sites.google.com/usc.edu/bimanual-binpacking/

Suction cups are popular for picking and transporting packages in warehouse applications. To maximize throughput, high transport speeds are desired. Many packages are deformable and may detach from the suction cups due to inertial loading if trajectories use excessive velocities. This paper introduces a novel methodology that analyzes package deformation through its curvature at the package-suction cup contact interface to generate a Factor-of-Safety (FOS) score for each waypoint in a given trajectory. By maintaining the FOS above a predetermined threshold, the trajectory planner is able to generate transport trajectories that are both safe and time-optimized. Experimental results show the method’s efficacy, demonstrating a 21.92% reduction in transport times compared to a conservative trajectory generation. Our FOS predictor identified trajectories that ensured safe package transport with 100% accuracy across all 627 real-world experiments.

In this work, we present an inverse reinforcement learning approach for solving the problem of task sequencing for robots in complex manufacturing processes. Our proposed framework is adaptable to variations in process and can perform sequencing for entirely new parts. We prescribe an approach to capture feature interactions in a demonstration dataset based on a metric that computes feature interaction coverage. We then actively learn the expert's policy by keeping the expert in the loop. Our training and testing results reveal that our model can successfully learn the expert's policy. We demonstrate the performance of our method on a real-world manufacturing application where we transfer the policy for task sequencing to a manipulator. Our experiments show that the robot can perform these tasks to produce human-competitive performance. Code and video can be found at: https://sites.google.com/usc.edu/irlfortasksequencing

Screw-driving is an important operation in numerous applications. In many situations, hole pose cannot be estimated very accurately. Autonomous screw-driving cannot be performed by traditional industrial manipulators in position control mode when the hole pose uncertainty is high. This paper presents a mobile manipulator system for performing autonomous screw-driving in the presence of uncertainties in the hole estimates. It utilizes active compliance in the form of impedance control of the robot and passive compliance in the screwing driving tool to deal with uncertainties. We present a physics-informed machine learning approach to automatically characterize the motion of the screw tip and explain how this motion leads to successful operation in the presence of uncertainty. We also present an approach for detecting failure modes and taking corrective actions. Code and video is available at: https://sites.google.com/usc.edu/physicsinformedscrewdriving

Tool manipulation is a crucial skill for robots to perform intricate tasks, and learning from demonstration methods can provide an effective means for robots to learn these skills. However, the process of collecting human demonstration data can be challenging and may lead to information loss, requiring a large number of demonstrations to learn the human's policy. In this work, we propose a novel framework for collecting information-rich human demonstration data for learning complex tool manipulation skills. Our framework can accommodate data collection from multiple modalities such as speech, gesture, motion, video, and 3D depth data. Additionally, the framework actively queries the human expert to improve the information content of the data. We showcase the effectiveness of our method in collecting demonstration data for a complex granular media transport task and performing the task on a real robot.

Composites are increasingly becoming a material of choice in the aerospace and automotive industries. Currently, many composite parts are produced by manually laying up sheets on complex molds. Composite sheet layup requires executing two main tasks: (1) grasping a sheet and (2) draping it on the mold. Automating the layup process requires automation of these two tasks. This paper is focused on the automation of the grasping task using robots. This requires an automated generation of grasp plans to enable robots to hold the sheet during the draping process. We present a simulation-based approach for determining robot grasp locations on the composite sheets. We also present an intervention controller that uses a real-time sheet tracking system during plan execution and can prevent failures. We demonstrate the performance of the developed system using a large complex part.