Underactuated robotic hands are extensively used in remote manipulation due to their ability to adapt to various object sizes and shapes. Their structural simplicity and small number of actuators required for operation make them highly versatile and responsive, which is crucial for effective teleoperation. In addition to grasping performance, haptic feedback, which integrates force and tactile sensing, is essential for dexterous manipulation. This study proposes a solution using fiber-optic tendons embedded with fiber Bragg gratings (FBGs), combining sensing and actuation to simultaneously perform power transmission, along with force and tactile sensing. Each finger employs a fiber-optic tendon with three FBGs: one measures tendon tension, and the other two at the fingertip detect contact force and temperature. The tendon is placed on the volar side of the finger and routed to an actuation module with a servomotor at the wrist for power transmission. This tendon enables finger flexion, while a passive extension mechanism with linear springs on the dorsal side facilitates extension. Experimental results demonstrate the feasibility of this approach, showing the hand's multifunctional capabilities, including haptic feedback and power transmission, as well as its potential for teleoperation. This approach improves the robotic hand's ability to provide real-time feedback, improving dexterity in remote manipulation.

The apparel industry is increasingly adopting automation technologies and investing in research to address global reshoring efforts and rising labor costs. One essential process for automation in garment factories is top stitch, which involves sewing two fabric sheets together along a seam line while maintaining a consistent distance from it. This paper presents an automated top stitch system that retrofits a legacy machine using a vision-based macro-mini approach. The system mainly comprises a two-degree-of-freedom automatic sewing machine as the legacy machine, a vision sensor module, a mini-actuator module, and an infrared sensor module. The vision sensor module detects the seam line on the fabric, even in the presence of positional uncertainties, such as, human errors. The infrared sensor module monitors the sewing sequence, while the mini-actuator module operates simultaneously with the sewing machine to compensate for any misalignment. The system demonstrates improvements, with the average and the maximum errors reduced by 79% and 67%, respectively, and the standard deviation improved by 73%, compared to those from the operation without our system. To the best of our knowledge, this is the first development of an autonomous top stitch system using a retrofitted legacy machine, presenting a novel architecture that integrates sensors and actuators through a macro-mini approach.

Robots have taken the place of human operators in hazardous and challenging jobs requiring high dexterity in manipulation, and robots with skin for force and tactile sensing that mimics the function of mechanoreception in animals will be highly dexterous in performing complex tasks. In this study, we propose the design of modular robotic skin, capable of detecting the magnitude and the location of a contact force simultaneously. Each skin module needs three degrees of freedom in sensing in order to estimate the horizontal and the vertical locations of the contact force as well as its magnitude. Force sensing in the proposed skin is enabled by a custom-designed triangular beam structure underneath the skin cover. A force applied to the skin cover causes the bending of the beam, which is detected by fiber optic strain sensors. The result shows the resolutions of 1.45 N for force estimation and 1.85 mm and 1.91 mm for contact localization in horizontal and vertical directions, respectively. We also demonstrate how the proposed skin can be used for remote and autonomous control of commercial robotic arms equipped with an array of the skin modules.

The ability to grasp a wider range of objects in size and shape directly relates to the performance of robotic grippers. Adapting to complex geometries of objects requires large degrees of freedom to allow complex configurations. However, complexity in controlling many individual joints leads to introduction of underactuated mechanisms, in which traditional finger designs composed of revolute joints allow only flexion/extension motions. In this article, we propose a length-adjustable linkage mechanism in the underactuated finger controlled by an antagonistic tendon pair. The resulting gripper can elongate the fingers for an increased task space or shorten them for a finer spatial resolution. For tactile sensing, hyperelastic soft sensors are used to stretch with finger elongation. Contact pressures measured by the soft sensors are used in force-feedback control for which either the joint angles or the link lengths are adjusted. Lastly, a multimodal control scheme that combines elongation and flexion modes is demonstrated with tasks of dexterous manipulation.

State estimation is one of the key requirements in robot control, which has been achieved by kinematic and dynamic models combined with motion sensors in traditional robotics. However, it is challenging to acquire accurate proprioceptive information in soft robots due to relatively high noise levels and hysteretic responses of soft actuators and sensors. In this article, we propose a method of estimating real-time states of soft robots by filtering noisy output signals and including hysteresis in the models using a Bayesian network. This approach is useful in constructing a state observer for soft robot control when both the kinematic model of the actuator and the model of the sensor are used. In our method, we regard a hysteresis function as a conditional random process model. We then introduce a dynamic Bayesian network composed of the actuator and the sensor models of the target system using distribution hysteresis mapping. Finally, we show that solving a Bayesian filtering problem is equivalent to suboptimal state estimation of the soft system. This article describes two ways for defining modeling and filtering; one is by Gaussian process regression combined with an extended Kalman filter, and the other is based on variational inference with a particle filter. While the first approach relaxes the uncertainty level in modeling to Gaussian, the second approach illustrates a general probability distribution. We experimentally validate the proposed methods through real-time state estimation of a sensor-integrated soft robotic gripper. The result shows significant improvement in state estimation compared to conventional estimation methods.

The performance of inflatable artificial muscles depends greatly on their designs and the output shapes resulting from the geometric constraints. Although there have been attempts to apply physical constraints on the air chamber to achieve larger stroke lengths and increased force-length ratios, it has been difficult to achieve the above two goals while maintaining a compact form factor. In this article, we propose flat inflatable artificial muscles that have large contraction ratios (up to 0.5) and show increased forces in wider ranges of contractions by adding an internal geometric constraint. Addition of an external constraint, such as rigid plates, further increased the maximum contraction ratio (up to 0.553) through a synergistic effect. We show that various force-length relations can be achieved by adjusting the height of the plates. Theoretical models based on the geometry and the principle of virtual work are experimentally validated using actuator prototypes made of heat-sealable plastic sheets. Also, compact capacitive sensors are integrated in design for proprioceptive feedback of the proposed actuators, and their feasibility and effectiveness are experimentally evaluated through closed-loop control.

Soft artificial skin sensors that can detect contact forces as well as their locations are attractive in various soft robotics applications. However, soft sensors made of polymer materials have inherent limitations of hysteresis and nonlinearity in response, which makes it highly difficult to implement traditional calibration techniques and yields poor estimation performance. In this paper, we propose intelligent algorithms based on machine learning and logics that can improve the performance of soft sensors. The proposed methods in this paper could be solutions to the aforementioned long-standing problems. They can also be used to simplify the system complexity by reducing the number of signal wires. Three machine learning techniques are discussed in this paper: an artificial neural network (ANN), the k-nearest neighbors (k-NN) algorithm, and a recurrent neural network (RNN). The Preisach model of hysteresis and simple logics were used to support these algorithms. We proved that classifying contact locations on a soft sensor is possible using simple algorithms in real time. Also, force estimation of a single contact was possible using an ANN with the Preisach method. Finally, we successfully estimated forces of multiple contact locations by predicting the outputs of mixed RNN results.

Presented are techniques for designing, modeling, and control of reliable pneumatic artificial muscle actuators with integrated low profile sensors for position feedback. The sensor is fabricated through a three-dimensional manufacturing process based on a modified lathe approach for controlling viscous and viscoelastic materials as well as on direct writing of liquid metal. Next, a new precision pneumatic muscle design and its integration with the sensor is illustrated. A theoretical model and experimental characterization of the muscle-sensor package are presented with high correlation and repeatability. Finally, a position feedback sliding mode controller is implemented with a position error of <0.9% of maximum muscle contraction.

Soft Pneumatic Actuators (SPAs) have recently become popular for use as fingers in robotic hands because of their inherent compliance, low cost, and ease of construction. We seek to overcome two key limitations which limit SPAs' abilities to grasp and manipulate objects: 1) Current SPAs lack position or force sensor feedback, which prevents controlling them precisely (e.g. to achieve a hand preshape or apply a specified pushing force), and 2) the tip of the SPA is compliant and has high friction against common surfaces, causing the SPA to stick against surfaces when grasping objects from above. To overcome the first limitation we propose methods to integrate soft eGaIn sensors into SPAs and controllers that use these sensors' feedback for position and force control. To overcome the second limitation, we explore embedding rigid fingernails into the tip of the SPA so that the finger does not stick against surfaces and can wedge under objects. Our experiments suggest that we can achieve low steady-state error and overshoot in position and force using feed-forward models that relate pressure, force, and curvature along with a PID controller. We also compare several fingernail designs and show that the best-performing design significantly outperforms having no fingernails when grasping a set of common objects from a table.

This paper describes the design and manufacturing of soft artificial skin with an array of embedded soft strain sensors for detecting various hand gestures by measuring joint motions of five fingers. The proposed skin was made of a hyperelastic elastomer material with embedded microchannels filled with two different liquid conductors, an ionic liquid and a liquid metal. The ionic liquid microchannels were used to detect the mechanical strain changes of the sensing material, and the liquid metal microchannels were used as flexible and stretchable electrical wires for connecting the sensors to an external control circuit. The two heterogeneous liquid conductors were electrically interfaced through flexible conductive threads to prevent the two liquid from being intermixed. The skin device was connected to a computer through a microcontroller instrumentation circuit for reconstructing the 3-D hand motions graphically. The paper also presents preliminary calibration and experimental results.

We present the design, fabrication, and characterization of a fiber optically sensorized robotic hand for multi purpose manipulation tasks. The robotic hand has three fingers that enable both pinch and power grips. The main bone structure was made of a rigid plastic material and covered by soft skin. Both bone and skin contain embedded fiber optics for force and tactile sensing, respectively. Eight fiber optic strain sensors were used for rigid bone force sensing, and six fiber optic strain sensors were used for soft skin tactile sensing. For characterization, different loads were applied in two orthogonal axes at the fingertip and the sensor signals were measured from the bone structure. The skin was also characterized by applying a light load on different places for contact localization. The actuation of the hand was achieved by a tendon-driven under-actuated system. Gripping motions are implemented using an active tendon located on the volar side of each finger and connected to a motor. Opening motions of the hand were enabled by passive elastic tendons located on the dorsal side of each finger.

Recent advances in soft sensors using microfluidic liquid conductors enabled sensing of large deformation of soft structures. However, the use of liquids as conductive media carries a risk of leakage in many cases. Furthermore, it could be harmful when exposed to the human body in certain applications. To address these issues, a different sensing mechanism was proposed: highly stretchable optical sensors that could detect multiple modes of deformation. The method of operation involves a simple waveguide and its housing which are both made of silicone elastomer. The soft waveguide is coated with a thin gold reflective layer to encapsulate light propagating internally, with an light-emitting diode (LED) and a photodiode embedded at each end. When the sensor is stretched, compressed, or bent, micro-cracks within the reflective layer form and allow part of the light to escape, resulting in optical power losses in the light transmission. In this paper, we describe the design and fabrication of the proposed soft sensors. A prototype was created and characterized for pressure, strain, and curvature up to 350 kPa, 90%, and 0.12 mm-1, respectively, showing promising results of reasonable repeatability and linearity in certain ranges.

We present the design of a soft wearable robotic device composed of elastomeric artificial muscle actuators and soft fabric sleeves, for active assistance of knee motions. A key feature of the device is the two-dimensional design of the elastomer muscles that not only allows the compactness of the device, but also significantly simplifies the manufacturing process. In addition, the fabric sleeves make the device lightweight and easily wearable. The elastomer muscles were characterized and demonstrated an initial contraction force of 38N and maximum contraction of 18mm with 104kPa input pressure, approximately. Four elastomer muscles were employed for assisted knee extension and flexion. The robotic device was tested on a 3D printed leg model with an articulated knee joint. Experiments were conducted to examine the relation between systematic change in air pressure and knee extension-flexion. The results showed maximum extension and flexion angles of 95° and 37°, respectively. However, these angles are highly dependent on underlying leg mechanics and positions. The device was also able to generate maximum extension and flexion forces of 3.5N and 7N, respectively.

This study demonstrates real-time scan plane control dependent on three-dimensional needle bending, as measured from magnetic resonance imaging (MRI)-compatible optical strain sensors. A biopsy needle with embedded fiber Bragg grating (FBG) sensors to measure surface strains is used to estimate its full 3-D shape and control the imaging plane of an MR scanner in real-time, based on the needle's estimated profile. The needle and scanner coordinate frames are registered to each other via miniature radio-frequency (RF) tracking coils, and the scan planes autonomously track the needle as it is deflected, keeping its tip in view. A 3-D needle annotation is superimposed over MR-images presented in a 3-D environment with the scanner's frame of reference. Scan planes calculated based on the FBG sensors successfully follow the tip of the needle. Experiments using the FBG sensors and RF coils to track the needle shape and location in real-time had an average root mean square error of 4.2 mm when comparing the estimated shape to the needle profile as seen in high resolution MR images. This positional variance is less than the image artifact caused by the needle in high resolution SPGR (spoiled gradient recalled) images. Optical fiber strain sensors can estimate a needle's profile in real-time and be used for MRI scan plane control to potentially enable faster and more accurate physician response.

Motion sensing has played an important role in the study of human biomechanics as well as the entertainment industry. Although existing technologies, such as optical or inertial based motion capture systems, have relatively high accuracy in detecting body motions, they still have inherent limitations with regards to mobility and wearability. In this paper, we present a soft motion sensing suit for measuring lower extremity joint motion. The sensing suit prototype includes a pair of elastic tights and three hyperelastic strain sensors. The strain sensors are made of silicone elastomer with embedded microchannels filled with conductive liquid. To form a sensing suit, these sensors are attached at the hip, knee, and ankle areas to measure the joint angles in the sagittal plane. The prototype motion sensing suit has significant potential as an autonomous system that can be worn by individuals during many activities outside the laboratory, from running to rock climbing. In this study we characterize the hyperelastic sensors in isolation to determine their mechanical and electrical responses to strain, and then demonstrate the sensing capability of the integrated suit in comparison with a ground truth optical motion capture system. Using simple calibration techniques, we can accurately track joint angles and gait phase. Our efforts result in a calculated trade off: with a maximum error less than 8%, the sensing suit does not track joints as accurately as optical motion capture, but its wearability means that it is not constrained to use only in a lab.

A proposed adaptive soft orthotic device performs motion sensing and production of assistive forces with a modular, pneumatically-driven, hyper-elastic composite. Wrapping the material around a joint will allow simultaneous motion sensing and active force response through shape and rigidity control. This monolithic elastomer sheet contains a series of miniaturized pneumatically-powered McKibben-type actuators that exert tension and enable adaptive rigidity control. The elastomer is embedded with conductive liquid channels that detect strain and bending deformations induced by the pneumatic actuators. In addition, the proposed system is modular and can be configured for a diverse range of motor tasks, joints, and human subjects. This modular functionality is accomplished with a decentralized network of self-configuring nodes that manage the collection of sensory data and the delivery of actuator feedback commands. This paper mainly describes the design of the soft orthotic device as well as actuator and sensor components. The characterization of the individual sensors, actuators, and the integrated device is also presented.

We present the design and fabrication of centimeter scale robots, which use inchworm-like motion and bidirectional claws. Two prototypes for different locomotion goals were built utilizing these two characteristics: Type I is designed particularly for horizontal surfaces utilizing two linear actuators and compliant claws. This robot is capable of steering and straight motion by utilizing directionally anisotropic friction. Type II is a variant of Type I that is designed for locomotion on ferromagnetic ceilings or vertical planes by using permanent magnets. The Smart Composite Microstructures (SCM) technique enables versatile and multi-jointed meso-scale devices suitable for such robots.

We describe the design of an active soft ankle-foot orthotic device powered by pneumatic artificial muscles for treating gait pathologies associated with neuromuscular disorders. The design is inspired by the biological musculoskeletal system of a human foot and a lower leg, and mimics the muscle-tendon-ligament structure. A key feature of the device is that it is fabricated with flexible and soft materials that provide assistance without restricting degrees of freedom at the ankle joint. Three pneumatic artificial muscles assist dorsiflexion as well as inversion and eversion. The prototype is also equipped with various embedded sensors for gait training and gait pattern analysis. The prototype is capable of 12° dorsiflexion from a resting position of an ankle joint and a 20° dorsiflexion from plantarflexion. Results of early feedback control experiments show controllability of ankle joint angles. Ultimately, we envision a system that not only can provide physical support to improve mobility but also can increase safety and stability during walking, while enhancing muscle usage and encouraging rehabilitation.

Force sensing is an essential requirement for dexterous robot manipulation. We describe composite robot end-effectors that incorporate optical fibers for accurate force sensing and estimation of contact locations. The design is inspired by the sensors in arthropod exoskeletons that allow them to detect contacts and loads on their limbs. In this paper, we present a fabrication process that allows us to create hollow multimaterial structures with embedded fibers and the results of experiments to characterize the sensors and controlling contact forces in a system involving an industrial robot and a two-fingered dexterous hand. We also briefly describe the optical-interrogation method used to measure multiple sensors along a single fiber at kilohertz rates for closed-loop force control.

We describe the dynamic testing and control results obtained with an exoskeletal robot finger with embedded fiber optical sensors. The finger is inspired by the designs of arthropod limbs, with integral strain sensilla concentrated near the joints. The use of fiber Bragg gratings (FBGs) allows for embedded sensors with high strain sensitivity and immunity to electromagnetic interference. The embedded sensors are useful for contact detection and for control of forces during fine manipulation. The application to force control requires precise and high-bandwidth measurement of contact forces. We present a nonlinear force control approach that combines signals from an optical interrogator and conventional joint angle sensors to achieve accurate tracking of desired contact forces.

Force sensing is an essential requirement for dexterous robot manipulation. Although strain gages have been widely used, a new sensing approach is desirable for applications that require greater robustness, design flexibility and immunity to electromagnetic noise. An exoskeletal force sensing robot finger was developed by embedding fiber Bragg grating (FBG) sensors into a polymer-based structure. Multiple FBG sensors were embedded into the structure to allow the manipulator to sense and measure both contact forces and grasping forces. In order to fabricate a three-dimensional structure, a new shape deposition manufacturing (SDM) process was explored. The sensorized SDM-fabricated finger was then characterized using an FBG interrogator. A force localization scheme is also described