In this paper, we introduce magnetic mobile micro-gripping microrobots with two independent actuation modes. By aligning two magnets with slight variations in magnetic moment orientations, we create a net magnetic moment for precise position and orientation control through external fields, while harnessing opposing torques on the magnets to induce internal stresses needed for gripping. Our microrobot design features a compliant spring-like structure for significant deflection, enabling a gripping motion under specific magnetic field conditions. Magnet rotation allows precise control over gripper actions, returning to a default state (normally open or closed) when the magnetic field diminishes. This work advances magnetic field-controlled microrobotics, bridging the millimeter-to-micrometer gap. It holds promise for applications in microsurgery, micro-assembly, and microscale exploration.

This article presentsSim-Suction, a robust object-aware suction grasp policy for mobile manipulation platforms with dynamic camera viewpoints, designed to pick up unknown objects from cluttered environments. Suction grasp policies typically employ data-driven approaches, necessitating large-scale, accurately-annotated suction grasp datasets. However, the generation of suction grasp datasets in cluttered environments remains underexplored, leaving uncertainties about the relationship between the object of interest and its surroundings. To address this, we propose a benchmark synthetic dataset,Sim-Suction-Dataset, comprising 500 cluttered environments with 3.2 million annotated suction grasp poses. The efficientSim-Suction-Datasetgeneration process provides novel insights by combining analytical models with dynamic physical simulations to create fast and accurate suction grasp pose annotations. We introduceSim-Suction-Pointnetto generate robust 6-D suction grasp poses by learning point-wise affordances from theSim-Suction-Dataset, leveraging the synergy of zero-shot text-to-segmentation. Real-world experiments for picking up all objects demonstrate thatSim-Suction-Pointnetachieves success rates of 96.76%, 94.23%, and 92.39% on cluttered level 1 objects (prismatic shape), cluttered level 2 objects (more complex geometry), and cluttered mixed objects, respectively.

This paper presents the design and control of a novel microrobot that utilizes two distinct magnetic locomotion methods, a combination of rotating and gradient field control, for precise micro-object manipulation using multiple end-effectors. Rotating magnetic fields induce a tumbling locomotion mode to increase the movement speed and decrease issues associated with stiction and locomotion over rough surfaces. The gradient field control allows for precise manipulation using the end-effectors, which include a pointed tip for splitting groups of objects and a blunt end for pushing or capturing objects. The microrobot is fabricated using a two-photon polymerization 3D printer, allowing for the precise reproduction of complex geometries and designs. The potential applications of this technology in the medical field are discussed, highlighting the potential for in vitro cellular manipulation.

Automating leaf detection and physical leaf sample collection using Internet of Things (IoT) technologies is a crucial task in precision agriculture. In this paper, we present a deep learning-based approach for detecting and segmenting crop leaves for robotic physical sampling. We discuss a method for generating a physical dataset of agricultural crops. Our proposed pipeline incorporates using an RGB-D camera for dataset collection, fusing the depth frame along with RGB images to train Mask R-CNN and YOLOv5 models. We also propose our novel leaf pose estimating algorithm for physical sampling and maximizing leaf sample area while using a robotic arm integrated to the P-AgBot platform. The proposed approach has been experimentally validated on corn and sorghum, in both indoor and outdoor environments. Our method has achieved a best-case detection rate of 90.6%, a 9% smaller error compared to our previous method, and approximately 80% smaller error compared to other state-of-the-art methods in estimating the leaf position.

This paper describes the development of a modular end-effector system (MEES) for autonomous robotic maintenance and repair tasks. The design consists of the following major components: Robot Side Mating Socket Module (RSMS), End-Effector Side Mating Socket Module (EEMS), the Modular Camera System (MCS), and Tool Holder/Changer unit. Multiple prototypes for each component have been manufactured, tested, and evaluated resulting in the final concept. Existing robotic tool-changer systems on the market were evaluated and features were built into the Modular End-Effector System to overcome the current limitations of those systems. A notable advantage to the MEES is that it is a robot agnostic system and simply uses an ISO standard bolt mounting pattern to physically attached to the robot of choice along with an ethernet connection. No external cables are required that could restrain the workspace of the robot manipulator. Additionally, it is compatible with customized wrist-mounted sensors and end-effectors without any modification of the actual robot circuitry. The MEES is demonstrated working with three different end-effectors and two different robots.

Adaptive locomotion of microrobots can be achieved by using a smart polymer such as a hydrogel. For hydrogel-based bilayer helical microrobots, the change of environment such as temperature and pH can result in shape deformation into helical shapes differing from their initial state and hence swimming performance. In this work, we proposed a model for studying the parameters that affect the shape deformation of a hydrogel-based bilayer helical microrobot. Moreover, the dynamics of some examples of responsive helical swimming are compared before and after stimulation.

In this paper, we present a method for pose-estimate-based target tracking (PBTT) that enables the performance of autonomous aerial manipulation operations in unstructured environments using fully on-board computation for both UAV localization and target tracking. The PBTT method does not depend on extracting traditional visual features (e.g. using SIFT, SURF, ORB, etc.) on or near the target. Instead, the algorithm combines input from an RGB-D camera and the UAV's position estimator (which utilizes a downward-facing optical flow camera for horizontal localization) to track a target point selected by a human operator. The effectiveness of the PBTT method is evaluated through several autonomous flight tests performed with the Interacting-Boomcopter (I-BC) UAV platform in unstructured environments and in the presence of light wind disturbances.

This paper presents a microforce-sensing mobile microrobot (μFSMM) for use in automated micromanipulation tasks. The design consists of a planar vision-based microforce sensor end-effector, while the microrobot body is made of chemically etched nickel that is driven by an exterior magnetic field. With a known stiffness, the manipulation forces can be determined from observing the deformation of the end-effector through a camera attached to an optical microscope. After analyzing and calibrating the stiffness of a micromachined prototype, the mobility and in situ force-sensing capabilities are verified through real-time, closed loop, force controlled manipulation tests with automated path planning and navigation. The calibrated stiffness of the microforce sensor end-effectors fabricated is on the order of 10-3N/m. The online (real time) force-sensing resolution is approximately 1.5 μN. The sensing range is 0-20 μN along the two planar directions. In automated micromanipulation experiments with a microcomponent, the μFSMM utilizes realtime force control to apply a prescribed force of 6 μN to a desired location on a fixed microobject. Similarly, in another automated micromanipulation experiment, the μFSMM demonstrates the use of real-time force control to limit the manipulation forces experienced by the microobject to remain below a threshold of 12 μN.

This paper presents a novel approach for autonomously mounting a sensor package on a vertical surface with an unmanned aerial vehicle (UAV). The Interacting-BoomCopter (I-BC) UAV uses an on-board webcam and computer along with a horizontally-mounted reversible propeller on its front boom to autonomously perform the aerial manipulation task. An overview of the vehicle design is presented along with the image processing algorithms used for target tracking, and the implementation of an extended finite state machine (EFSM) for carrying out the high-level autonomous control. The effectiveness of the autonomous control strategy and I-BC platform are examined through the performance of several autonomous sensor mounting flight tests.

In this paper, we present a control framework that allows magnetic microrobot teams to accomplish complex micromanipulation tasks captured by global linear temporal logic (LTL) formulas. To address this problem, we propose an optimal control synthesis method that constructs discrete plans for the robots that satisfy both the assigned tasks as well as proximity constraints between the robots due to the physics of the problem. The proposed algorithm relies on an existing optimal control synthesis approach combined with a novel sampling-based technique to reduce the state-space of the product automaton that is associated with the LTL specifications. The synthesized discrete plans are executed by the microrobots independently using local magnetic fields. Simulation studies show that the proposed algorithm can address large-scale planning problems that cannot be solved using existing optimal control synthesis approaches. Moreover, we present experimental results that also illustrate the potential of the method in practice. To the best of our knowledge, this is the first control framework that allows independent control of teams of magnetic microrobots for temporal micromanipulation tasks.

This paper presents the design of the Interacting-BoomCopter (I-BoomCopter) unmanned aerial vehicle (UAV), designed specifically for environmental interactions. The novel design consists of a horizontally mounted four-blade reversible propeller on a front boom that is attached to a standard tri-rotor UAV configuration. A custom end-effector and force sensor is placed at the end of the front boom for autonomous pushing and pulling interaction tasks. The modeling of the new platform is presented and two prototype versions built, flight tested, and characterized. Finally, the efficacy of the new platform for environmental interactions is evaluated with an autonomous door opening task and a teleoperated door opening and door closing task.

This paper presents the preliminary design of a micro force sensing mobile microrobot. The design consists of a planar, vision-based micro force sensor end-effector, while the microrobot body is made from a nickel magnetic layer driven by an exterior magnetic field. With a known stiffness, the manipulation forces can be determined from observing the deformation of the end-effector through a CCD camera attached to an optical microscope. After analyzing and calibrating the stiffness of a micromachined prototype, manipulation tests are conducted to verify this microrobot prototype is indeed capable of in situ force sensing while performing a manipulation task. This concept can be scaled down further for next generation designs targeting real biomedical applications on microscale.

In this paper, we present a novel approach to micromanipulation and assembly tasks using caging micro-manipulation motion primitives for 2D and 3D autonomous micromanipulation and assembly tasks. A flexible, multi-scale test-bed, developed for use in conjunction with these primitives, is described first. The approach and motion primitives are then discussed. An autonomous micromanipulation and assembly task is described and manipulation plans using the methodology are generated and experimentally executed. The autonomous task shows promising results with a success rate of 93% for 27 individual micro-assembly experiments.

This paper presents a magnetic tumbling microrobot design at the micro-scale with flexible operating modes. The microrobot has a dumb-bell shape whose largest dimension is 400 μm. When subjected to an exterior predefined magnetic field, the magnetic microagent performs a tumbling motion driven by the interacting magnetic forces and momentums. By switching the magnetic field during the motion cycle the agent is also able to perform a sliding locomotion that is useful for micromanipulation. The magnetic field providing the drive force is generated by a portable coil system consisting of five electromagnetic coils. Under the available driven field, the prototype has shown adaptable mobility through tumbling mechanism on various types of surface in both dry and fluid environments, and also shown pushing manipulation in viscous fluid. This manipulation force has been experimentally evaluated through testing with AFM tip and a micro force sensor and shown to be on the order of several μNs.

Traditional vertical take-off and landing micro aerial vehicles (VTOL MAVs) are generally underactuated, i.e., equipped with fewer actuators than degrees-of-freedom (DOF). As a consequence, they possess a limited mobility because of the inherent underactuation (e.g., they can neither translate laterally with a zero attitude nor hover at a spot with a nonzero attitude). In this paper, we present the design of a novel MAV, the Omnicopter, with two central counter-rotating coaxial propellers for thrust and yaw control and three perimeter-mounted variable angle ducted fans to control roll and pitch and provide lateral forces. It can work under two configurations, a fixed 90° ducted fan angle configuration and a variable angle ducted fan configuration. The variable angle configuration provides full actuation to the Omnicopter position/attitude. After a brief introduction of the Omnicopter platform, we discuss the control design, allocation and implementation for the two configurations separately. Simulations and experimental results verify the performance of the Omnicopter.

This paper presents an original configuration of a micro aerial vehicle (MAV), the Omnicopter. Two central counter-rotating coaxial propellers provide a major part of lift force, and three perimeter-mounted tiltable ducted fans are used to supplement the lift force, provide lateral forces and adjust its attitude. Different from traditional underactuated MAVs, the presence of the tilt-rotor mechanism, composed of three ducted fans and three servo motors, on the Omnicopter makes it over-actuated. The characteristic of over-actuation enables the Omnicopter's position dynamics to be decoupled from its attitude dynamics. Based on a complete description of its dynamic model derived using the Newton-Euler motion equations, we propose attitude and position controllers and control allocation for the Omnicopter MAV. Simulation and experimental results are shown to demonstrate its performance.

This paper introduces the concept of caging micromanipulation for use in automated open loop microassembly tasks. Utilizing a caging transport motion primitive along with rotational and translation primitives, we demonstrate full control of the state of the part. Additionally, a framework for planar microassembly task planning is provided based on the A* algorithm. It is used to determine the optimal assembly sequences and part starting locations in the workspace. We also describe a test-bed suitable for planar micro, meso-scale, and nano-scale manipulation and assembly tasks and present simulation and experimental results of this work.

Magnetic principles have proved successful for untethered submillimeter microrobotics, although challenges still exist in areas of propulsion and control. This paper presents the design, analysis, and performance results for a bimorph thin film magnetic microrobot utilizing the magnetostrictive principle as a secondary oscillating operation mode. The microrobot is no larger than 580 μm in its planar dimension and its total thickness is less than 5 μm. As a robot with magnetic material, it can be operated in a pushing/pulling mode in orthogonal directions for movement in a plane, while it's powered with an external magnetic field as low as 1 mT. For the secondary oscillating operation mode utilizing the magnetostrictive principle, in-plane strain is induced, resulting in bending and blocking forces on the robot. These forces are theoretically calculated to prove enough drive force can be generated in this mode. The design is further abstracted and translated into a piezoelectric cantilever FEM model to confirm the theorectical results. Microrobot fabrication and test-bed development based on this analysis is shown, which enabled us to participate in the final competition in the 2010 NIST Mobile Microrobot Challenge, with good performance in the dash and freestyle events. Finally, we discuss the testing results in various dry and fluid environments along with recommendations for future investigation and improvements. Keywords: microrobot, magnetostrictive, bimorph.

In this paper, we demonstrate a systematic way to determine configurations for up to four coordinated micromanipulators to form caging grasps for transporting micro-scale planar, polygonal parts. We exploit the geometry of the part, noting the presence and location of convex corners and non-convex corners, and form opposing force equivalents with the micromanipulator probe tips around the micro-scale parts that define a caging polygon. We perform an error bound analysis for caging grasps derived in this manner and provide theoretical values of this bound for four micro-parts of interest. We demonstrate experimental results of the caging micromanipulation transport primitive using these feature-defined grasps and compare them with the expected error bounds. Finally, we combine this caging transport primitive along with rotational and one-sided-pushing motion primitives, to carry out a representative microassembly task.

This paper describes a test-bed for planar micro and mesoscale manipulation tasks and a framework for planning based on quasi-static models of mechanical systems with intermittent frictional contacts. We show how planar peg-in-the-hole assembly tasks can be designed using randomized motion planning techniques with Mason's models for quasi-static manipulation. Simulation and experimental results are presented in support of our methodology. We develop this further into a systematic approach to incorporating uncertainty into planning manipulation tasks with frictional contacts. We again consider the canonical problem of assembling a peg into a hole at the mesoscale using probes with minimal actuation but with visual feedback from an optical microscope. We consider three sources of uncertainty. First, because of errors in sensing position and orientation of the parts to be assembled, we must consider uncertainty in the sensed configuration of the system. Second, there is uncertainty because of errors in actuation. Third, there are geometric and physical parameters characterizing the environment that are unknown. We discuss the synthesis of robust planning primitives using a single degree-of-freedom probe and the automated generation of plans for mesoscale manipulation. We show simulation and experimental results of our work.

We present a two-dimensional, vision-based force sensor, capable of sensing muN level forces. There are currently no reliable, off-the-shelf, commercially-available force sensors to measure forces at this scale, that can be easily integrated into standard microrobotic test-beds. Our design consists of a planar, elastic mechanism with known force-deflection characteristics. A CCD camera is used to track the deformation of the mechanism as it is used to manipulate objects in a micro/meso-scale robotic manipulation test-bed. By observing the displacements of select points in the mechanism, the manipulation forces can be estimated. The modeling, design, microfabrication, calibration and experimental validation of the force sensor are presented with a brief discussion of an application to a robotic manipulation tasks such as microassembly.

Manipulation and assembly tasks are typically characterized by many nominally rigid bodies coming into frictional contacts, possibly involving impacts. Manipulation tasks are difficult to model because uncertainties associated with friction and assembly tasks are particularly hard to analyze because of the interplay between process tolerance and geometric uncertainties due to manufacturing errors. Manipulation at the meso (hundred microns to millimeters) and micro (several microns to tens of microns) scale is even harder for several reasons. It is difficult to measure forces at the micro-netwon level reliably using off-the-shelf force sensors and good force-feedback control schemes have not proved successful. It is hard to manufacture general-purpose end effectors at this scale and it is even more difficult to grasp and manipulate parts at the micro and meso level than it is at the macro level. Finally, the lack of good models of the mechanics of contact interactions at this scale means that model-based approaches to planning and control are difficult.

This paper describes a test-bed for planar micro manipulation tasks and a framework for planning based on quasi-static models of mechanical systems with frictional contacts. We show how planar peg-in-the-hole assembly tasks can be designed using randomized motion planning techniques with Mason's models for quasi-static manipulation. Finally, we present simulation and experimental results in support of our methodology