3D part segmentation is still an open problem in the field of 3D vision and AR/VR. Due to limited 3D labeled data, traditional supervised segmentation methods fall short in generalizing to unseen shapes and categories. Recently, the advancement in vision-language models' zero-shot abilities has brought a surge in open-world 3D segmentation methods. While these methods show promising results for$3 D$scenes or objects, they do not generalize well to 3D humans. In this paper, we present the first open-vocabulary segmentation method capable of handling 3D human. Our framework can segment the human category into desired fine-grained parts based on the textual prompt. We design a simple segmentation pipeline, leveraging SAM to generate multi-view proposals in 2D and proposing a novel Human-CLIP model to create unified embeddings for visual and textual inputs. Compared with existing pre-trained CLIP models, the HumanCLIP model yields more accurate embeddings for human-centric contents. We also design a simple-yet-effective MaskFusion module, which classifies and fuses multi-view features into 3D semantic masks without complex voting and grouping mechanisms. The design of decoupling mask proposals and text input also significantly boosts the efficiency of per-prompt inference. Experimental results on various 3D human datasets show that our method outperforms current state-of-the-art open-vocabulary 3D segmentation methods by a large margin. In addition, we showthat our method can be directly applied to various 3D representations including meshes, point clouds, and 3D Gaussian Splatting.

Due to labor shortages in specialty crop industries, a need for robotic automation to increase agricultural efficiency and productivity has arisen. Previous manipulation systems harvest well in uncluttered and structured environments. High tunnel environments are more compact and cluttered in nature, requiring a rethinking of the large form factor systems and grippers. We propose a novel co-designed framework incorporating a global detection camera and a local eye-in-hand camera that demonstrates precise localization of small fruits via closed-loop visual feedback and reliable error handling. Field experiments in high tunnels show that our system can reach 85.0% of cherry tomato fruit in 10.98s on average.

A neural network-based framework is developed and experimentally demonstrated for the problem of estimating the shape of a soft continuum arm (SCA) from noisy measurements of the pose at a finite number of locations along the length of the arm. The neural network takes as input these measurements and produces as output a finitedimensional approximation of the strain, which is further used to reconstruct the infinite-dimensional smooth posture. This problem is important for various soft robotic applications. It is challenging due to the flexible aspects that lead to the infinitedimensional reconstruction problem for the continuous posture and strains. Because of this, past solutions to this problem are computationally intensive. The proposed fast smooth reconstruction method is shown to be five orders of magnitude faster while having comparable accuracy. The framework is evaluated on two testbeds: a simulated octopus muscular arm and a physical BR2 pneumatic soft manipulator.

The agricultural setting poses additional challenges for robotic manipulation, as fruit is firmly attached to plants and the environment is cluttered and occluded. Therefore, accurate feedback about the grasp state is essential for effective harvesting. This study examines the different states involved in fruit picking by a robot, such as successful grasp, slip, and failed grasp, and develops a learning-based classifier using low-cost, computationally light sensors (IMU and IR reflectance). The Random Forest multi-class classifier accurately determines the current state and along with the sensors can operate in the occluded environment of a plant. The classifier was successfully trained and tested in the lab and showed 100% success at identifying slip and grasp failure and 80% success identifying successful picks on a real cherry tomato plant. By using this classifier, corrective actions can be planned based on the current state, thus leading to more efficient fruit harvesting.

This paper is concerned with the problem of estimating (interpolating and smoothing) the shape (pose and the six modes of deformation) of a slender flexible body from multiple camera measurements. This problem is important in both biology, where slender, soft, and elastic structures are ubiquitously encountered across species, and in engineering, particularly in the area of soft robotics. The proposed mathematical formulation for shape estimation is physics-informed, based on the use of the special Cosserat rod theory whose equations encode slender body mechanics in the presence of bending, shearing, twisting and stretching. The approach is used to derive numerical algorithms which are experimentally demonstrated for fiber reinforced and cable-driven soft robot arms. These experimental demonstrations show that the methodology is accurate (<5 mm error, three times less than the arm diameter) and robust to noise and uncertainties.

Interest in soft continuum arms has increased as their inherent material elasticity enables safe and adaptive interactions with the environment. However to achieve full autonomy in these arms, accurate three-dimensional shape sensing is needed. Vision-based solutions have been found to be effective in estimating the shape of soft continuum arms. In this paper, a vision-based shape estimator that utilizes a geometric strain based representation for the soft continuum arm’s shape, is proposed. This representation reduces the dimension of the curved shape to a finite set of strain basis functions, thereby allowing for efficient optimization for the shape that best fits the observed image. Experimental results demonstrate the effectiveness of the proposed approach in estimating the end effector with accuracy less than the soft arm’s radius. Multiple basis functions are also analyzed and compared for the specific soft continuum arm in use.

Soft robots undergo large nonlinear spatial deformations due to both inherent actuation and external loading. The physics underlying these deformations is complex, and often requires intricate analytical and numerical models. The complexity of these models may render traditional model-based control difficult and unsuitable. Model-free methods offer an alternative for analyzing the behavior of such complex systems without the need for elaborate modeling techniques. In this paper, we present a model-free approach for open loop position control of a soft spatial continuum arm, based on deep reinforcement learning. The continuum arm is pneumatically actuated and attains a spatial work-space by a combination of unidirectional bending and bidirectional torsional deformation. We use Deep-Q Learning with experience replay to train the system in simulation. The efficacy and robustness of the control policy obtained from the system is validated both in simulation and on the continuum arm prototype for varying external loading conditions.

This paper presents the design and testing of a bio-inspired soft pneumatic robot that can achieve locomotion along the outside of a cylinder. The robot uses soft pneumatic actuators called FREEs (Fiber Reinforced Elastomeric Enclosure), which can have a wide range of deformation behavior upon pressurization. The robot being soft and compliant can grasp and move along cylinders of varying dimensions. Two different types of FREEs are used in the robot namely (a) extending FREEs and (b) bending FREEs. These actuators are arranged in such a way that the bending actuators are used to grip the pipe while the extending actuators generate forward motion as well as bending for direction control. The modular design of the robot provides simplicity and ease of maintenance. The entire robot is made of flexible actuators and can withstand external impact with minimal to no damage. The maximum speed achieved for horizontal motion is 4.2 mm/s and for vertical motion is 2.1 mm/s.

Low weight and innate compliance make soft pneumatic actuators an attractive method for actuating wearable robots. Performance of soft pneumatic actuators can be tailored to an application by combining them in novel architectures. We modeled and constructed nested linear and pennate architectures using fiber-reinforced elastomeric enclosures (FREEs) with identical manufacturing parameters and total effective lengths to compare their suitability for a cable-driven exoskeleton for augmenting shoulder flexion. We determined actuator performance requirements using a static model for the transmission of actuator forces to the upper arm via Bowden cables. We experimentally characterized the architectures by measuring their force-displacement curves at a range of pressures, yielding greater force and displacement from the nested architecture in the domain required by our exoskeleton. Results also indicated a force threshold above which the pennate structure produced greater force at any given displacement. We validated the nested linear architecture using a prototype exoskeleton installed on a passive mannequin. Measured joint angles at varying pressures were close to predicted values, adjusted for measured losses due to cable anchor movement.

Soft robotic actuators are well suited for use in exoskeleton applications due to their innate compliance and low weight. We have developed a wearable soft robotic sleeve that uses fiber reinforced elastomeric enclosures (FREEs) to provide actuation and stiffness at the elbow for augmented lifting and carrying ability. The sleeve includes novel linear and helical actuator architectures to induce and resist joint movement respectively, and is intended to be comfortable, lightweight, and low profile. We developed test protocols to measure actuation and stiffness performance of different helical and linear architectures, and to compare helical and linear actuator groups when used individually and together. Our findings indicate that nested linear actuators have superior contraction ratios compared to parallel linear actuators, resulting in greater angular displacement. Stiffness from helical actuators increased with pressure and number of parallel actuators. A combined linear-helical actuator configuration considerably outperformed helical and linear actuator groups when used on their own.

Fiber Reinforced Elastomeric Enclosures (FREEs) are fundamental building blocks of pneumatic soft robots, with McKibben muscles as common examples. When a system of FREEs are arranged in a planar configuration, they undergo large bending deformations due to internal loads and moments. This paper models the bending behavior of FREEs using a modified three-spring Pseudo-Rigid Body (PRB) model, which is commonly used to capture large bending behavior of flexures. The springs of the PRB model are shown to effectively capture the axial and bending stiffness of contracting FREEs, while an internal force accounts for actuation due to internal pressurization. An optimization process is carried out to fit the PRB model parameters with the experimental response curve of different FREEs with varying length to diameter ratio and fiber angles. Unlike conventional models applied to compliant mechanisms, the model parameters are shown to depend on fluid pressure. The PRB model is used to demonstrate and analyze a pennate arrangement of contracting FREEs that exhibit variable gear ratio behavior. The pennate FREEs can serve as building blocks for the design of soft robots.

The Industrial and Enterprise Systems Engineering department at the University of Illinois at Urbana-Champaign has five major research areas: data analytics; decision and control systems; design and manufacturing; financial engineering; and operations research. During the summer of 2015, faculty and graduate students from each of the last four areas designed mini-projects and assignments to cover the research areas of the department in a required first-year course. The goal of the designed course was to provide a comprehensive overview of the department and an engaging experience for first-year students. Project-based learning and Kolb's cycle for experiential learning were used to inform the structure of the course. Pre-surveys and post-surveys were administered to gather feedback from students in the course. On the surveys, students reported an increase in understanding of each of the research areas and a positive experience in the course.

Fiber reinforced elastomeric actuators are popular actuators for soft robots because of their inherent safety, energy density and a large repertoire of spatial motion patterns. However, a small subset of these actuators alone known as McKibben pneumatic muscles with antisymmetric fiber orientations have been extensively analyzed in literature. This paper analyzes the large deformation kinematics of generalized McKibben actuators with asymmetric and arbitrarily varying fiber orientations by formulating a simple and accurate isoperimetric problem that involves constrained volume maximization problem. This model maximally decouples kinematics and kinetostatics thereby significantly reducing the numerical complexity involved in analysis. The accuracy of the model is verified by benchmarking with existing models for the McKibben actuator case, and with experiments for novel designs with no associated prior literature. This model is deemed to be useful in the design synthesis of fiber reinforced elastomeric actuators for a desired kinematic and kinetostatic requirement.

The growing interest in robots that interact safely with humans and surroundings have prompted the need for soft structural embodiments including soft actuators. This paper explores a class of soft actuators inspired in design and construction by Pneumatic Artificial Muscles (PAMs) or McKibben Actuators. These bio-inspired actuators consist of fluid-filled elastomeric enclosures that are reinforced with fibers along a specified orientation and are in general referred to as Fiber-Reinforced Elastomeric Enclosures (FREEs). Several recent efforts have mapped the fiber configurations to instantaneous deformation, forces, and moments generated by these actuators upon pressurization with fluid. However most of the actuators, when deployed undergo large deformations and large overall motions thus necessitating the study of their large-deformation kinematics. This paper analyzes the large deformation kinematics of FREEs. A concept called configuration memory effect is proposed to explain the smart nature of these actuators. This behavior is tested with experiments and finite element modeling for a small sample of actuators. The paper also describes different possibilities and design implications of the large deformation behavior of FREEs in successful creation of soft robots.

Soft actuators are found throughout nature from elephant trunks to round worms, demonstrating large specific forces without the need for sliding components. These actuators offer impact resilience, human-safe interaction, versatility of motion, and scalability in size. Biological structures often use a fiber-reinforcement around a fluid filled elastomeric enclosure, in which the elastomeric material will capture the distributed pressure and transfer it to the fibers, which will in turn direct the forces to the ends. We previously discovered an entire domain of fiber-reinforced elastomeric enclosures (FREEs), of which McKibben actuators are a small subset. The range of forces and moments possible with FREEs has not been previously investigated. 45 FREE actuators across the span of fiber angle configurations were fabricated and tested. The reaction force and moment of each actuator was determined across a gamut of pressures. Analytical models were generated using a variety of simplifying assumptions. These models were created to provide a closed form expression that models the force and moment data. The models were compared to the experimental values to determine their fit; this provides an understanding of which simplifying kinematic assumptions best represent the experimental results. Interpolated experimental results and the analytical models are all graphically represented for use as an intuitive design tool.

Complex controlled motions, soft human interaction, and minimal moving mass all drive the need for soft robots using fluid filled fiber reinforced elastomer enclosures (FREEs). While a narrow class of FREEs known as McKibben's actuators have been extensively studied, there is a wide unexplored class with complex sets of motion patterns. Combining these actuators in parallel can yield versatile motion patterns resulting in a large overall workspace. Mobility of individual actuators is enforced by inextensibility of fibers and incompressibility of fluids, which in turn drives the net attainable motions. In this paper, we map the mobility of individual FREE actuators to all possible resultant motions that a combination of three sets of actuators arranged in a triangular configuration would undergo. This understanding has resulted in a preliminary design tool that determines individual FREE topologies for a required set of motion patterns. Five case studies that result from this methodology are prototyped and tested for comparison of the predicted and obtained motion directions.