The design and development of morphing structures that transition from compact, transportable forms to stable, deployable configurations is crucial for advances in soft robotics, healthcare applications, and biomimetic systems. These structures often require customized functionalities and must self-deploy into precise target shapes. Therefore, the deformed shapes of such structures are usually prescribed and the parameters for their design are unknown. To obtain the fabrication parameters, the inverse problem needs to be solved, which quickly becomes quite challenging using conventional methods due to the high-dimensional nature of the inverse problem as well as the material and geometric nonlinearities. To overcome these challenges, we combine the best of the two worlds – physics and data – and present a data-driven approach for the inverse design of two-layered soft composites that utilize the principles of kirigami and strain mismatch to self-deploy into different three-dimensional shapes. At the center of our methodology is the generative adversarial network, designed to generate the necessary fabrication parameters. By using a pre-trained simulator network, we condition the generative model to generate feasible and accurate fabrication parameters that are used to make composites that deploy into the target shapes. Our findings demonstrate that the generative model is able to effectively predict kirigami patterns and pre-stretch values required to realize complex three-dimensional shapes from simple and diverse planar designs. By performing simulations and precise desktop experiments, we compare the target with deployed shapes and demonstrate the predictive capacity of the method.

Robotic manipulation of slender objects is challenging, especially when the induced deformations are large and nonlinear. Traditionally, learning-based control approaches, such as imitation learning, have been used to address deformable material manipulation. These approaches lack generality and often suffer critical failure from a simple switch of material, geometric, and/or environmental (e.g., friction) properties. This article tackles a fundamental but difficult deformable manipulation task: forming a predefined fold in paper with only a single manipulator. A sim2real framework combining physically-accurate simulation and machine learning is used to train a deep neural network capable of predicting the external forces induced on the manipulated paper given a grasp position. We frame the problem using scaling analysis, resulting in a control framework robust against material and geometric changes. Path planning is then carried out over the generated “neural force manifold” to produce robot manipulation trajectories optimized to prevent sliding, with offline trajectory generation finishing 15$\times$faster than previous physics-based folding methods. The inference speed of the trained model enables the incorporation of real-time visual feedback to achieve closed-loop model-predictive control. Real-world experiments demonstrate that our framework can greatly improve robotic manipulation performance compared to state-of-the-art folding strategies, even when manipulating paper objects of various materials and shapes.Note to Practitioners—This article is motivated by the need for efficient robotic folding strategies for stiff materials such as paper. Previous robot folding strategies have focused primarily on soft materials (e.g., cloth) possessing minimal bending resistance or relied on multiple complex manipulators and sensors, significantly increasing computational and monetary costs. In contrast, we formulate a robust, sim2real, physics-based method capable of folding papers of varying stiffness with a single manipulator. The proposed folding scheme is limited to papers of homogeneous material and folding along symmetric centerlines. Future work will involve formulating efficient methods for folding along arbitrary geometries and preexisting creases.

Precision agricultural robots require high-resolution navigation solutions. In this paper, we introduce a robust neural-inertial sequence learning approach to track such robots with ultra-intermittent GNSS updates. First, we propose an ultra-lightweight neural-Kalman filter that can track agricultural robots within 1.4 m (1.4–5.8× better than competing techniques), while tracking within 2.75 m with 20 mins of GPS outage. Second, we introduce a user-friendly video-processing toolbox to generate high-resolution (±5 cm) position data for fine-tuning pre-trained neural-inertial models in the field. Third, we introduce the first and largest (6.5 hours, 4.5 km, 3 phases) public neural-inertial navigation dataset for precision agricultural robots. The dataset, toolbox, and code are available at: https://github.com/nesl/agrobot.

Multi-flagellated bacteria utilize the hydrodynamic interaction between their filamentary tails, known as flagella, to swim and change their swimming direction in low Reynolds number flow. Simplified hydrodynamics model, like Resistive Force Theories (RFT), lacks the capability to capture the dynamics of certain interactions known as bundling and tumbling. However, for the development of efficient and steerable robots inspired by bacteria, it becomes crucial to exploit this interaction. In this paper, we present the construction of a macroscopic bio-inspired robot featuring two rigid flagella arranged as right-handed helices, along with a cylindrical head. By rotating the flagella in opposite directions, the robot's body can reorient itself through repeatable and controllable tumbling. To accurately model this bi-flagellated mechanism in low Reynolds flow, we employ a coupling of rigid body dynamics and the method of Regularized Stokeslet Segments (RSS). Unlike RFT, RSS takes into account the hydrodynamic interaction between distant filamentary structures. Furthermore, we delve into the exploration of the parameter space in terms of the flagellum geometry to optimize the propulsion and torque of the system. To achieve the desired reorientation of the robot, we propose a tumble control scheme that involves modulating the rotation direction and speed of the two flagella. The scheme enhance the steerability by enabling the robot to attain the desired heading angle with high accuracy. Notably, the overall scheme boasts a simplified design and control as it only requires two control inputs. With our macroscopic framework serving as a foundation, we envision the eventual miniaturization of this technology to construct mobile and controllable micro-scale bacterial robots.

As technology advances, the need for safe, efficient, and collaborative human-robot-teams has become increasingly important. One of the most fundamental collaborative tasks in any setting is the object handover. Human-to-robot handovers can take either of two approaches: (1) direct hand-to-hand or (2) indirect hand-to-placement-to-pick-up. The latter approach ensures minimal contact between the human and robot but can also result in increased idle time due to having to wait for the object to first be placed down on a surface. To minimize such idle time, the robot must preemptively predict the human intent of where the object will be placed. Furthermore, for the robot to preemptively act in any sort of productive manner, predictions and motion planning must occur in real-time. We introduce a novel prediction-planning pipeline that allows the robot to preemptively move towards the human agent's intended placement location using gaze and gestures as model inputs. In this paper, we investigate the performance and drawbacks of our early intent predictor-planner as well as the practical benefits of using such a pipeline through a human-robot case study.

Smart weeding systems to perform plant-specific operations can contribute to the sustainability of agriculture and the environment. Despite monumental advances in autonomous robotic technologies for precision weed management in recent years, work on under-canopy weeding in fields is yet to be realized. A prerequisite of such systems is reliable detection and classification of weeds to avoid mistakenly spraying and, thus, damaging the surrounding plants. Real-time multi-class weed identification enables species-specific treatment of weeds and significantly reduces the amount of herbicide use. Here, our first contribution is the first adequately large realistic image dataset AIWeeds (one/multiple kinds of weeds in one image), a library of about 10,000 annotated images of flax and the 14 most common weeds in fields and gardens taken from 20 different locations in North Dakota, California, and Central China. Second, we provide a full pipeline from model training with maximum efficiency to deploying the TensorRT-optimized model onto a single board computer. Based on AIWeeds and the pipeline, we present a baseline for classification performance using five benchmark CNN models. Among them, MobileNetV2, with both the shortest inference time and lowest memory consumption, is the qualified candidate for real-time applications. Finally, we deploy MobileNetV2 onto our own compact autonomous robot SAMBot for real-time weed detection. The 90% test accuracy realized in previously unseen scenes in flax fields (with a row spacing of 0.2-0.3 m), with crops and weeds, distortion, blur, and shadows, is a milestone towards precision weed control in the real world. We have publicly released the dataset and code to generate the results at https://github.com/StructuresComp/Multi-class-Weed-Classification.

Simulating the swimming of soft underwater robot remains challenging due to the absence of an efficient numerical framework that can effectively capture the geometrically nonlinear deformation of soft materials and structures when interacting with a liquid environment. Here, we address this by introducing a discrete differential geometry-based model that incorporates an implicit treatment of the elasticity of soft limbs and a fluid model with three different components: hydrodynamic drag, jetting, and virtual added mass. The physical engine can run faster than real-time on a single thread desktop processor. We experimentally validate this numerical simulation tool by performing tests using an untethered omni-directional star-shaped swimming soft robot that is capable of moving with multiple swimming gaits. Quantitative agreement between experiment and simulation indicates the potential application of such a numerical framework for robot design and for model-based control schemes.

Modern herbicide application in agricultural set-tings typically relies on either large scale sprayers that dispense herbicide over crops and weeds alike or portable sprayers that require labor intensive manual operation. The former method results in overuse of herbicide and reduction in crop yield while the latter is often untenable in large scale operations. This paper presents the first fully autonomous robot for weed management for row crops capable of computer vision based navigation, weed detection, complete field coverage, and automatic recharge for under $400. The target application is autonomous inter-row weed control in crop fields, e.g. flax and canola, where the spacing between croplines is as small as one foot. The proposed robot is small enough to pass between croplines at all stages of plant growth while detecting weeds and spraying herbicide. A recharging system incorporates newly designed robotic hardware, a ramp, a robotic charging arm, and a mobile charging station. An integrated vision algorithm is employed to assist with charger alignment effectively. Combined, they enable the robot to work continuously in the field without access to electricity. In addition, a color-based contour algorithm combined with preprocessing techniques is applied for robust navigation relying on the input from the onboard monocular camera. Incorporating such compact robots into farms could help automate weed control, even during late stages of growth, and reduce herbicide use by targeting weeds with precision. The robotic platform is field-tested in the flaxseed fields of North Dakota.

Coating rollers are widely popular in structural painting, in comparison with brushes and sprayers, due to thicker paint layer, better color consistency, and effortless customizability of holder frame and naps. In this paper, we introduce a cost-effective method to employ a general purpose robot (Sawyer, Rethink Robotics) for autonomous coating. To sense the position and the shape of the target object to be coated, the robot is combined with an RGB-Depth camera. The combined system autonomously recognizes the number of faces of the object as well as their position and surface normal. Unlike related work based on two-dimensional RGB-based image processing, all the analyses and algorithms here employ three-dimensional point cloud data (PCD). The object model learned from the PCD is then autonomously analyzed to achieve optimal motion planning to avoid collision between the robot arm and the object. To achieve human-level performance in terms of the quality of coating using the bare minimum ingredients, a combination of our own passive and builtin active impedance control is implemented. The former is realized by installing an ultrasonic sensor at the end-effector of robot working with a customized compliant mass-spring-damper roller to keep a precise distance between the end-effector and surface to be coated, maintaining a fixed force. Altogether, the control approach mimics human painting as evidenced by experimental measurements on the thickness of the coating. Coating on two different polyhedral objects is also demonstrated to test the overall method.