Deformable object tracking is used in many robotics applications including biomanipulation, vision-based force sensing, and the control of deformable structures. A tracking algorithm that is robust to occlusions and to spurious edges is essential since these situations can arise unexpectedly in the unstructured environments in which robots must operate. This paper presents a deformable object tracking algorithm that is robust to occlusion and to spurious edges. Robust statistical methods are used to handle occlusion and a modification of the Canny edge detector is presented to handle spurious edges. The modification of the Canny edge operator makes use of information about the object being tracked in order to eliminate spurious edges. The deformable object tracking algorithm's performance is evaluated visually and quantitively by tracking a four degree-of-freedom compliant gripper.

A four degree of freedom thermally actuated MEMS microgripper is presented in this paper. Each jaw of the microgripper has independent x and y degrees of freedom. These extra degrees of freedom give the gripper added dexterity for manipulating microparts. The motion of each gripper jaw is provided by a two degree of freedom compliant mechanism which is based on a five bar rigid link mechanism. This gripper also introduces a novel thermal microactuator design that is capable of bi-directional actuation giving it a greater range of motion than previous thermal actuator designs. The actuator provides a total range of motion of 12.7 /spl mu/m and a maximum force of 1.9 mN. Also, since this microgripper is based on a compliant mechanism, deformable object tracking can be used to provide force as well as position feedback for the gripper. This combination of an increased number of degrees of freedom and increased sensory feedback provides a level of dexterity that has not been previously available in microassembly.

This paper presents a deformable object tracking algorithm based on the boundary element method (BEM). BEM differs from the finite element method (FEM) in that only the boundary of the object needs to be meshed for BEM. FEM requires that the interior of the object is meshed in addition to its boundary. This feature of BEM makes it attractive for computer vision problems. We present a deformable template that uses BEM to model deformations. This deformable template is registered to an image using an energy minimization approach. The BEM tracking algorithm presented in this paper constraints the tracking results to satisfy the condition of static equilibrium. This increases the robustness of the tracking results and enhances the usefulness of the forces obtained from the tracking procedure. We demonstrate the tracking performance of this algorithm for objects with linear and non-linear elastic properties. In addition, the results of tracking the deformations of a cell are presented.

This paper presents a microrobotic system integrating microscope vision and microforce feedback for characterizing biomembrane mechanical properties. Robust visual tracking of deformable biomembrane contour using physics-based models is described. A multi-axis MEMS-based force sensor is used to determine applied forces on biomembranes and develop a novel biomembrane mechanical model. By visually extracting geometry changes on a biomembrane during loading, geometry changes can be used to estimate applied forces using the biomembrane mechanical model and the determined elastic modulus. Forces on a biomembrane can be visually observed and controlled, thus creating a framework for vision and force assimilated cell manipulation. The experimental results quantitatively describe mouse zona pellucida (ZP) stiffness increase during ZP hardening and provide an understanding of ZP protein structure development, i.e., an increase in the number of cross links of protein ZP1 between ZP2-ZP3 units that is conjectured to be responsible for ZP stiffness increase. Furthermore, the system, technique, and model presented in this paper can be applied to investigating mechanical properties of other biomembranes and other cell types, which has the potential to facilitate many biological studies, such as cell injury and recovery where biomembrane mechanical property changes need to be monitored.

This paper demonstrates a method to visually measure the force distribution applied to a linearly elastic object using the contour data in an image. The force measurement is accomplished by making use of the result from linear elasticity that the displacement field of the contour of a linearly elastic object is sufficient to completely recover the force distribution applied to the object. This result leads naturally to a deformable template matching approach where the template is deformed according to the governing equations of linear elasticity. An energy minimization method is used to match the template to the contour data in the image. This technique of visually measuring forces we refer to as vision-based force measurement (VBFM). VBFM has the potential to increase the robustness and reliability of micromanipulation and biomanipulation tasks where force sensing is essential for success. The effectiveness of VBFM is demonstrated for both a microcantilever beam and a microgripper. A sensor resolution of less than +/- 3 nN for the microcantilever and +/- 3 mN for the microgripper was achieved using VBFM. Performance optimizations for the energy minimization problem are also discussed that make this algorithm feasible for real-time applications.

This paper presents a method to perform 2D (two-dimensional) deformable object tracking using the boundary element method (BEM). BEM, like the finite element method (FEM), is a technique to model an elastic solid. BEM differs from FEM in that only the contour of an object needs to be meshed for BEM, making this method attractive for computer vision problems. For FEM, the interior of the object must be meshed also. In order to track deformable objects, a deformable template is defined that uses BEM to model displacements. The template is registered to the image by applying a force field that deforms the template to match the image. This force field is found using an energy minimization approach. Even though the deformable template uses a linear elastic model, it can be used to track the deformations of objects with nonlinear material properties or in cases where there are large deformations. We demonstrate the performance of this method on objects with linear and nonlinear elastic properties. In addition, it is discussed how this method can be readily extended to 3D (three-dimensional) deformable object tracking.

This paper presents a method to model the deformation of an elastic object with an artificial neural network. The neural network is trained directly from images of the elastic object deforming under known loads. Using this process, models can be created for objects such as biological tissues that cannot be modeled by existing techniques. The neural network elastic model is used in conjunction with a deformable template matching algorithm to perform vision-based force measurement (VBFM). We demonstrate this learning method on objects with both linear and nonlinear elastic properties.

When assembling MEMS devices or manipulating biological cells it is often beneficial to have information about the force that is being applied to these objects. This force information is difficult to measure at these scales. We demonstrate a method to reliably measure nanonewton scale forces applied to a micro scale cantilever beam using a computer vision approach. A template matching algorithm is used to estimate the beam deflection to sub-pixel resolution in order to determine the force applied to the beam. The template, in addition to containing information about the geometry of the beam, contains information about the elastic properties of the beam. Minimizing the error between this elastic template and the actual image by means of numerical optimization techniques, we are able to measure forces to within /spl plusmn/3 nN. In addition, we also discuss how this method can be generalized to measure forces in elastic configurations other than a simple cantilever beam using a micro-tweezer as an example. This provides the opportunity for this method to be used with specially designed micromanipulators to provide force as well as vision feedback for micromanipulation tasks.