Large-scale metal additive manufacturing has become increasingly popular in aerospace and petroleum industries alike for sustainable fabrication of thin-shelled structural components. For example, wire-arc additive manufacturing (WAAM) offers high-deposition rates on large printing areas by robot-assisted welding of thick layers of material. However, WAAM technologies suffer from significant layer displacement and resultant part-scale distortion due to unstable high temperature deposition processes and lack of economically viable support structures. Therefore, geometric accuracy qualification at layer level is critical to process optimization and control. However, layer quality assessment relies on layer identification from large point clouds. Manual layer segmentation is experience-dependent and time-consuming due to high surface roughness, excessive layer remelting, and severe out-of-plane layer displacement. To enable automated layer segmentation for quality assessment, we computationally model a human operator’s intuition utilized in the process of finding layer boundaries, that is, locating nearby regions with a large number of possible boundary points, and finetuning boundaries by learning boundaries functions. In our proposed approach, geometrical features of the boundaries between printed layers are exploited to identify candidate boundary points. Cooperative multi-learning-agents efficiently process the large point clouds to locate sets of nearby regions with a high density of boundary points. Learning agents then sample promising boundary points. Gaussian process regression is employed to fine-tune layer boundaries through learning mean boundary functions and their uncertainties from the sampled points. Simulation studies demonstrate the accuracy and robustness of the procedure under severe surface roughness conditions. WAAM experimental studies illustrate the applicability of the methodology in practice.

Many manufacturing companies are facing an acute shortage of qualified workers. Deploying robotic cells is a potential solution to address this challenge. Historically robots have been deployed only in mass production applications in manufacturing. A large fraction of manufacturing is classified as high-mix manufacturing where a large variety of products are produced. Manually programming robots is not a viable solution in high-mix manufacturing applications. Robotic cells need to be powered by embodied AI to make them useful in high-mix manufacturing applications. This paper aims to build a bridge between smart manufacturing and AI communities to enable AI researchers to develop methods and tool that can be successfully deployed to realize smart robotic cells for high-mix manufacturing applications. This paper highlights key requirements for developing embodied AI for powering robotic cells for high-mix manufacturing applications. It also makes the case for approaches that combine model-based and data-driven methods to meet the needs of embodied AI in manufacturing applications and describes the role of generative AI approaches in smart manufacturing applications. Finally, it describes how AI can be used to enhance digital twins and augment human-machine interfaces in manufacturing applications.

Disassembly is a critical challenge in maintenance and service tasks, particularly in high-precision operations such as electric vehicle (EV) battery recycling. Tasks like prying-open sealed battery covers require precise manipulation and controlled force application. In our approach we collect human demonstrations using a motion capture system, enabling the robot to learn from human-expert disassembly strategies. These demonstrations train a bimanual robotic system in which one arm exerts force with a specialized tool while the other manipulates and removes sealed components. Our method builds on a diffusion-based policy and integrates real-time force sensing to adapt its actions as contact conditions change. We decompose the demonstrations into distinct sub-tasks and apply data augmentation, thereby reducing the number of demonstrations needed and mitigating potential task failures. Our results show that the proposed method, even with a small dataset, achieves a high task success rate and efficiency compared to a standard diffusion technique. We demonstrate in a real-world application that the bimanual system effectively executes chiseling and peeling actions to separate bonded sheet from a substrate.

Uncrewed ground vehicle (UGV) applications, such as warehouse operations, assembly-line production, infrastructure inspection, surveillance, precision farming, and search & rescue, can benefit from shared control, in which a human can semi-automatically control the UGV when needed and let it operate fully-automatically, when desired. Many algorithms have been developed to permit a UGV to semi-autonomously conduct tasks, either individually, or in a group. However, a complete semi-autonomous system that works wherever, and whenever, needed is far from being implemented. Here, we develop a human-robot shared controller for the supervisory control of one or more UGVs by a single person. The shared controller blends an automatic control input with a human control input. The automatic control input consists of a trajectory tracking controller and a control barrier function based input term for collision avoidance. A joystick is used to provide the human control input. Human intent is measured employing a Lyapunov-like storage function, which is used in a convex function based blending law that continuously varies the magnitude of the control inputs coming from the human and the machine. The approach permits us to theoretically prove the asymptotic stability of the closed-loop system. The shared controller is validated using both a physical robot in a cluttered indoor environment, and a hardware-in-the-loop simulated robot operating in virtual warehouse environment.

Multi-robot systems are becoming more common in various real-world applications, such as manufacturing and warehouse logistics. However, task allocation and scheduling for a multi-agent team face complex challenges due to the need to simultaneously consider time-extended tasks, task constraints, and uncertainties in execution. Potential task failures or contingencies can add additional tasks to recover from the failures, and reactively addressing contingencies can decrease teaming efficiency. To efficiently and proactively consider contingencies, this paper proposes treating the problem as a multi-robot task allocation under uncertainty problem. We suggest a hierarchical approach that divides the problem into two layers. We use mathematical program formulation for the lower layer to find the optimal solution for a deterministic multi-robot task allocation problem with known task outcomes. The higher-layer search intelligently generates more likely combinations of contingency scenarios and calls the inner-level search repeatedly to find the optimal task allocation sequence for the given scenario. We validate our results in simulation for manufacturing applications and demonstrate that our method can reduce the effect of potential delays from contingencies.Note to Practitioners—Automation engineers interested in deploying robotic cells in low-volume applications need to consider contingency handling. When the occurrence of contingencies can be characterized as probability distributions, it is often useful to consider using a proactive approach for task allocation and scheduling. To implement our algorithm, automation engineers will need to develop a hierarchical task network specified by domain experts that models task constraints and a task-agent duration model, which may be generated from simulation environments. Furthermore, they must identify tasks that can result in contingencies and describe them with a probabilistic model. This model can be generated from historical data and/or real-world experiments. Lastly, for addressing the contingency, the practitioner will need to specify a task procedure to recover from a specific contingency type. To run the algorithm, we found that repeatedly approximating the best proactive task allocation for a fixed computation budget and dispatching the best tasks worked well. The computation budget required to approximate the best task allocation is directly affected by the number of contingency scenarios that can be sampled. Therefore, the practitioner must determine a suitable computational budget empirically based on the number of contingencies that can occur.

Augmenting capabilities of human operators with multi-robot cells offers substantial advantages for increasing productivity in manufacturing applications. This synergy effectively combines the strengths of both robots and humans, maximizing operational efficiency and leveraging human capabilities. However, achieving these benefits requires real-time, reactive coordination of multi-robot motion generation in response to human motion. Current approaches face significant challenges, particularly in dealing with uncertainties in human motions. To address these issues, this paper introduces the Deep Reinforcement Learning (DRL) approach for end-to-end safe motion generation in human multi-robot collaborative workspaces. First, the DRL approach is augmented by adopting mixed-reality (MR) features to facilitate efficient state perception and representation of tasks, humans, robots, and scenes for enabling effective learning motion generation policy. Moreover, to better promote high-dimensional action generation of the multi-robot systems involving human, an advanced DRL approach is developed. The approach leverages memory-enhanced representation learning, intrinsic reward-guided exploration, and action space pruning to better address the motion generation challenges. Empirical testing demonstrates the effectiveness of the proposed system, with experiments showing high success rates across tasks with varying team sizes and difficulty levels, thereby demonstrating applicability in human-robot collaborative manufacturing tasks.

Multi-robot teams can be very useful in a wide variety of search and rescue missions in challenging environments. In a mission with considerable uncertainty due to intermittent communications, degraded information flow, and failures, humans need to assess both the current and expected future states and update task assignments in human-robot teams as quickly as possible. We have developed an alert generation framework that can perform risk assessment and robot tasking suggestions to assist human supervisors. Our approach for task assignment suggestion generation combines heuristics-based task selection with forward simulation-based probabilistic assessment. As the characteristics of decision aids can largely vary human performance, an alert system may or may not improve decision-making. We aim to configure our framework with a goal to improve human decision-making performance. Towards that, we present some preliminary user studies and design reasoning, which informed our final comprehensive human subject study. We demonstrate in the study that supervisors can improve their decision-making abilities, make faster decisions, and increase mission performance by using our alert generation framework. Our empirical findings also show that our framework does not require significant training and that people with a higher level of trust in automation perform better when provided with alerts. We also find that people with certain personality traits such as high agreeableness and conscientiousness are the most benefited by alerts.

Many applications require the deployment of legged-robot teams to effectively and efficiently carry out missions. The use of multiple robots allows tasks to be executed concurrently, expediting mission completion. It also enhances resilience by enabling task transfer in case of a robot failure. This paper presents a formulation based on Mixed Integer Linear Programming (MILP) for allocating tasks to robots by taking into account travel time and ensuring efficient execution of collaborative tasks. We extended the MILP formulation to account for complexities with legged robot teams. Our results demonstrate that this approach leads to improved performance in terms of the makespan of the mission. We demonstrate the usefulness of this approach using a case study involving the disinfection of a building consisting of multiple rooms.

Multi-robot systems are becoming increasingly prevalent in various real-world applications, such as manufacturing and warehouse logistics. These systems face complex challenges in 1) task allocation due to factors like time-extended tasks, and agent specialization, and 2) uncertainties in task execution. Potential task failures can add further contingency tasks to recover from the failure, thereby causing delays. This paper addresses the problem of Multi-Robot Task Allocation under Uncertainty by proposing a hierarchical approach that decouples the problem into two levels. We use a low-level optimization formulation to find the optimal solution for a deterministic multi-robot task allocation problem with known task outcomes. The higher-level search intelligently generates more likely combinations of failures and calls the inner-level search repeatedly to find the optimal task allocation sequence, given the known outcomes. We validate our results in simulation for a manufacturing domain and demonstrate that our method can reduce the effect of potential delays from contingencies. We show that our algorithm is computationally efficient while improving average makespan compared to other baselines.

In manufacturing, minimizing operational delays is crucial for efficiency and resilience. Therefore, efficiently handling contingencies is essential in human-robot teams working on assembly (i.e., collaborative assembly) applications. This paper introduces a novel approach to generating contingency handling procedures by leveraging recent advances in Large Language Models (LLMs). Our approach uses LLMs to update the required tasks in hierarchical task networks (HTNs) to handle contingencies. The results demonstrate that our approach can handle various contingencies in assembly applications and minimize the impact on the assembly completion time.

In recent years, the field of legged robotics has seen growing interest in enhancing the capabilities of these robots through the integration of articulated robotic arms. However, achieving successful loco-manipulation, especially involving interaction with heavy objects, is far from straightforward, as object manipulation can introduce substantial disturbances that impact the robot’s locomotion. This paper presents a novel framework for legged loco-manipulation that considers whole-body coordination through a hierarchical optimization-based control framework. First, an online manipulation planner computes the manipulation forces and manipulated object task-based reference trajectory. Then, pose optimization aligns the robot’s trajectory with kinematic constraints. The resultant robot reference trajectory is executed via a linear MPC controller incorporating the desired manipulation forces into its prediction model. Our approach has been validated in simulation and hardware experiments, highlighting the necessity of whole-body optimization compared to the baseline locomotion MPC when interacting with heavy objects. Experimental results with Unitree Aliengo, equipped with a custom-made robotic arm, showcase its ability to lift and carry an 8kg payload and manipulate doors.

Bin-packing is an important problem in the robotic warehouse domain. Traditionally, this problem has been studied only for rigid packages (e.g., boxes or rigid objects). In this work, we tackle the problem of bin-packing with deformable packages that have become a popular choice for fulfillment needs. We present a system that incorporates a dual robot arm bimanual setup, uniquely combining suction and sweeping motions to stably and reliably pack deformable packages in a bin. Additionally, we propose a comprehensive action prediction framework to optimize for bin-packing efficiency by predicting optimal actions for both robots involved. Our methodology leverages a two-pronged learning strategy, where initially, we train a model in a self-supervised manner to predict a scoring metric indicative of bin-packing efficiency and then leverage an online optimization scheme to compute optimal actions in real time. The model is pre-trained in simulation in MuJoCo and fine-tuned on small-scale data from a real-world laboratory setting. Our packing score prediction model predicts bin-packing score ∈ [0, 1] with an MSE of 0.003. Real-world experiments validate our method’s adaptability to novel scenarios and its effectiveness in packing operations. Project Website: https://sites.google.com/usc.edu/bimanual-binpacking/

Suction cups are popular for picking and transporting packages in warehouse applications. To maximize throughput, high transport speeds are desired. Many packages are deformable and may detach from the suction cups due to inertial loading if trajectories use excessive velocities. This paper introduces a novel methodology that analyzes package deformation through its curvature at the package-suction cup contact interface to generate a Factor-of-Safety (FOS) score for each waypoint in a given trajectory. By maintaining the FOS above a predetermined threshold, the trajectory planner is able to generate transport trajectories that are both safe and time-optimized. Experimental results show the method’s efficacy, demonstrating a 21.92% reduction in transport times compared to a conservative trajectory generation. Our FOS predictor identified trajectories that ensured safe package transport with 100% accuracy across all 627 real-world experiments.

In a mission with significant uncertainty due to intermittent communications, delayed information flow, and robotic failures, the role of human supervisors is extremely challenging. As and when any new information arrives, humans must infer both the existing and predicted future states, identify potential contingencies, and update task assignments to robots rapidly. We propose methodologies for automated generation of task reallocation suggestions to humans to assist in the decision-making process. Our generated robot retasking plan minimizes a modified makespan of the mission, which incorporates task criticality and penalty for incomplete tasks. The plan considers the effects of potential mission contingencies on the tasks, the robots, and the future performance of the robots operating based on the previous task plans. Our method includes the incorporation of two optional tasks, i.e., relay and robot rescue, for performance improvement. The rescue task has probabilistic outcomes affecting the team size. One or more rescues are incorporated in a way that can minimize the expected value of the modified makespan overall possibilities of rescue outcomes. We have conducted performance evaluation using simulation, demonstrating the value of the optional tasks and performance enhancement using our method of incorporating them. Note to Practitioners—The work reported in this paper will be useful in applications where a team of agents is deployed to carry on a large-scale mission with communication constraints where the number of functional agents can change probabilistically. Typically, such uncertainty is encountered in applications that are challenging or dangerous in nature. Agents can have non-zero probabilities to fail while doing certain risky tasks and to get recovered by other agents. The proposed centralized multi-agent task reallocation method can help in proactively addressing potential contingencies in surveillance, search and rescue, or disaster relief to support resilient operations while having a supervisor in the higher chain of command.

We focus on enabling robots to proactively assist humans in assembly tasks by adapting to their preferred sequence of actions. Much work on robot adaptation requires human demonstrations of the task. However, human demonstrations of real-world assemblies can be tedious and time-consuming. Thus, we propose learning human preferences from demonstrations in a shorter, canonical task to predict user actions in the actual assembly task. The proposed system uses the preference model learned from the canonical task as a prior and updates the model through interaction when predictions are inaccurate. We evaluate the proposed system in simulated assembly tasks and in a real-world human-robot assembly study and we show that both transferring the preference model from the canonical task, as well as updating the model online, contribute to improved accuracy in human action prediction. This enables the robot to proactively assist users, significantly reduce their idle time, and improve their experience working with the robot, compared to a reactive robot.

We consider the problem of task assignment and scheduling for human-robot teams to enable the efficient completion of complex problems, such as satellite assembly. In high-mix, low volume settings, we must enable the human-robot team to handle uncertainty due to changing task requirements, potential failures, and delays to maintain task completion efficiency. We make two contributions: (1) we account for the complex interaction of uncertainty that stems from the tasks and the agents using a multi-agent concurrent MDP framework, and (2) we use Mixed Integer Linear Programs and contingency sampling to approximate action values for task assignment. Our results show that our online algorithm is computationally efficient while making optimal task assignments compared to a value iteration baseline. We evaluate our method on a 24-task representative assembly and a real-world 60-task satellite assembly, and we show that we can find an assignment that results in a near-optimal makespan.

In this work, we present an inverse reinforcement learning approach for solving the problem of task sequencing for robots in complex manufacturing processes. Our proposed framework is adaptable to variations in process and can perform sequencing for entirely new parts. We prescribe an approach to capture feature interactions in a demonstration dataset based on a metric that computes feature interaction coverage. We then actively learn the expert's policy by keeping the expert in the loop. Our training and testing results reveal that our model can successfully learn the expert's policy. We demonstrate the performance of our method on a real-world manufacturing application where we transfer the policy for task sequencing to a manipulator. Our experiments show that the robot can perform these tasks to produce human-competitive performance. Code and video can be found at: https://sites.google.com/usc.edu/irlfortasksequencing

Recent studies on quadruped robots have focused on either locomotion or mobile manipulation using a robotic arm. However, legged robots can manipulate large objects using non-prehensile manipulation primitives, such as planar pushing, to drive the object to the desired location. This paper presents a novel hierarchical model predictive control (MPC) for contact optimization of the manipulation task. Using two cascading MPCs, we split the loco-manipulation problem into two parts: the first to optimize both contact force and contact location between the robot and the object, and the second to regulate the desired interaction force through the robot locomotion. Our method is successfully validated in both simulation and hardware experiments. While the baseline locomotion MPC fails to follow the desired trajectory of the object, our proposed approach can effectively control both object's position and orientation with minimal tracking error. This capability also allows us to perform obstacle avoidance for both the robot and the object during the loco-manipulation task.

Screw-driving is an important operation in numerous applications. In many situations, hole pose cannot be estimated very accurately. Autonomous screw-driving cannot be performed by traditional industrial manipulators in position control mode when the hole pose uncertainty is high. This paper presents a mobile manipulator system for performing autonomous screw-driving in the presence of uncertainties in the hole estimates. It utilizes active compliance in the form of impedance control of the robot and passive compliance in the screwing driving tool to deal with uncertainties. We present a physics-informed machine learning approach to automatically characterize the motion of the screw tip and explain how this motion leads to successful operation in the presence of uncertainty. We also present an approach for detecting failure modes and taking corrective actions. Code and video is available at: https://sites.google.com/usc.edu/physicsinformedscrewdriving

Experimental design of autonomous systems involves defining experimental inputs to maximize the experimenter’s information gained, minimize costs, and balance risk. This effectively leads to improved understanding and trustworthiness, which are necessary for deployment in realworld settings. Since experimental design is inherently a human-in-the-loop, sequential decision making problem, and decisions are being made about complex systems, an investigation into decision-making quality and decision-supporting methods is warranted. In this work, we investigate a decision support system (DSS) to augment the human’s experimental design decision making abilities, and conduct an exploratory user study to investigate the potential for decision support. Our findings show that experimenters, including experienced field roboticists, make suboptimal decisions and mistakes during the experimental design process, which suggests robotics research could benefit from DSSs. Our proposed DSS shows promise in some select aspects of experimental design, including helping to reduce suboptimal decisions, and participants in the user study reported favorable opinions of using such a system, including a sense of usefulness and lack of burden. The broader implication of this work is the identification of decision support in experimental design as one way to help bridge the gap between academia and industry by way of accelerated, informative experimentation and increased system explainability.

Tool manipulation is a crucial skill for robots to perform intricate tasks, and learning from demonstration methods can provide an effective means for robots to learn these skills. However, the process of collecting human demonstration data can be challenging and may lead to information loss, requiring a large number of demonstrations to learn the human's policy. In this work, we propose a novel framework for collecting information-rich human demonstration data for learning complex tool manipulation skills. Our framework can accommodate data collection from multiple modalities such as speech, gesture, motion, video, and 3D depth data. Additionally, the framework actively queries the human expert to improve the information content of the data. We showcase the effectiveness of our method in collecting demonstration data for a complex granular media transport task and performing the task on a real robot.

Serial-link manipulators are required to execute trajectories that enable a robot end-effector or a tool to track a Cartesian path. Practical applications may not require constraining all six degrees of freedom (position and orientation) of the tool resulting in semi-constrained paths. Semi-constrained paths allow improved success rates and better quality trajectories as the robot has more freedom to meet the kinematic and dynamic constraints. Additionally, robotic applications will need to use multiple tool center points (TCPs) on the tool to generate feasible paths for the robot. We present an iterative graph construction method to find trajectories for semi-constrained Cartesian paths that also use multiple TCPs. Our graph-based method finds multiple inverse kinematic solutions for possible Cartesian poses that the robot can take and connects them to build a graph. The algorithm uses cues from the Cartesian space to prioritize poses that produce a better quality solution. A biasing scheme is also developed to selectively sample the starting Cartesian poses from available choices. Our method finds near-optimal solutions with significantly fewer nodes and edges in the graph. The algorithm’s performance results on complex industrial test cases are provided. Note to Practitioners—Industrial applications permit the relaxation of one or more degrees of freedom of the tool while following the Cartesian paths. The relaxation of constraints is introduced by defining tolerances between the tool and the workpiece. Multiple TCPs have to be employed for many tasks to use different surfaces of the tool. In this paper, we present a planning algorithm for semi-constrained Cartesian paths that can incorporate the use of multiple TCPs. The user can define discrete TCPs over the tool contact points or surfaces. The tolerances can be easily defined as angular limits on tool orientation along the Cartesian path. Our planning algorithm can work with others via point constraints in Cartesian space or joint space of the robot. Practitioners from the industry can use the method presented in this paper to develop automated robotic cells that do cutting, sanding, polishing, welding, painting, composite layup, additive manufacturing, and several other common applications.

Wire arc additive manufacturing is a metal additive manufacturing process in which the material is deposited using arc welding technology. It is gaining popularity due to high material deposition rates and faster build time. It is en-abled using robotic manipulators and can build relatively large-scale parts faster when compared with other metal additive manufacturing processes. However, the size of the large-scale parts is limited by the size of the industrial manipulator being used for the process. This limitation is overcome by using a fixed configuration multi-robot cell in which manipulators work cooperatively to build large-scale parts quickly. A fixed multi-robot cell with closely spaced industrial manipulators has high flexibility, but it restricts the part size that can be built. If the manipulators are spread out, the cell loses its flexibility but can build relatively larger parts. This issue can be avoided by using larger size manipulators, which are expensive, or by moving the modest size manipulators based on the part geometries. This paper presents a novel algorithm to generate multi-robot placements for different part geometries to be built using wire arc additive manufacturing. Furthermore, the algorithm hierarchically optimizes the build time and the inverse kinematics consistency in robot paths to improve the process efficiency and part quality. We compare the results with fixed multi-robot cells and provide insights to users to make an informed decision on whether to use a fixed or a flexible multi-robot cell for wire arc additive manufacturing.

Testing and evaluation of field robotic systems requires both experimentation in representative conditions and human supervision to effectively assess components, manage risk, and interpret results. Due to the complexity of robotic sys-tems, we argue this experimentation should be done adaptively by using insights gained from previous trials. Furthermore, we envision an advisory system that could assist experimenters with selecting trial configurations by learning and accounting for human preferences and risk tolerances; however, formal methods for human decision making in the context of field robotic experimentation remains an open question. In this work, we present and analyze a case study for how decisions were made during the testing and evaluation of an off-road, autonomous navigation system. From the perspective of active learning, we find that Bayesian Optimization is a promising mathematical framework for modeling human decision making in adaptive experimental design of field robotics and that a combination of the EI, KG, and PES acquisition functions would likely be useful for realizing an advisory system.

Managing robot workloads in human robot teams is critical for efficient team operation. If robots are overloaded with work, then they will miss deadlines and force humans to take on extra work. This paper presents a framework for a robot to assess its own workload based on an initial goal assignment. The robot does this by generating task and motion plans and computing the probability of missing deadlines due to the possibility of delays in task execution. A branch and bound based search is used to generate task and motion plans by minimizing task execution effort. The robot presents a diverse set of task and motion plans to the humans to offer multiple different options. Humans can either approve a plan or provide guidance to reduce the workload by either relaxing deadlines or removing goal(s) assigned to the robots.

To assist human users according to their individual preference in assembly tasks, robots typically require user demonstrations in the given task. However, providing demonstrations in actual assembly tasks can be tedious and time-consuming. Our thesis is that we can learn the preference of users in actual assembly tasks from their demonstrations in a representative canonical task. Inspired by prior work in economy of human movement, we propose to represent user preferences as a linear reward function over abstract task-agnostic features, such as movement and physical and mental effort required by the user. For each user, we learn the weights of the reward function from their demonstrations in a canonical task and use the learned weights to anticipate their actions in the actual assembly task; without any user demonstrations in the actual task. We evaluate our proposed method in a model-airplane assembly study and show that preferences can be effectively transferred from canonical to actual assembly tasks, enabling robots to anticipate user actions.

Mobile manipulators are being deployed for transporting parts between machines and work stations in warehouses and shop floors. To increase the efficiency of operations, these mobile manipulators are required to complete the tasks as fast as possible. Picking up parts with the manipulator while the mobile base is moving decreases the time required to complete the transportation task and increases the efficiency of operations. However, motions of the manipulator on a moving platform can be risky, and hence, it is desired that the manipulator starts and ends its motions as close as possible to the part being picked up. In this article, we present a bidirectional sampling-based scheme for generating such manipulator trajectories for a given mobile base trajectory for pickup and transportation. Our approach implicitly determines the location of the mobile base where the manipulator motion starts and ends as well as where grasping happens. It also determines which grasping pose to use for picking up the part. Furthermore, we have presented the techniques to reduce the manipulator motion time (span time) and the computation time. Our approach enables us to reduce span time on average by 35% with a$16\times $reduction in the computation time compared to the RRT-based baseline methods.Note to Practitioners—Transportation of objects is a crucial application in industrial and warehouse environments. Conveyor belts and AGVs are typically used for such applications as they provide an efficient mode of transportation of a large number of objects. However, they may not provide the flexibility which mobile manipulators bring in for small-scale operations. The method presented in this article provides a way to increase the efficiency of operation for mobile manipulators for transportation tasks. We attempt to reduce the risks of the manipulator colliding with expensive equipment and moving obstacles by making sure that the manipulator moves only when needed for picking up objects. Moreover, the method can be used to pick up and transport a variety of parts in different ways using a two-fingered gripper. The method can also easily incorporate different types of grippers and mobile platforms.

Robotic manipulators are increasingly being used to perform additive manufacturing. The accuracy of a built part is dependent on the trajectory execution error of the manipulator. For articulated manipulators, the trajectory execution error and achievable build accuracy vary considerably over the workspace. Therefore, the build accuracy depends on where the part is placed in the manipulator workspace. If the part is small compared to the manipulator workspace, its placement can be optimized to improve the accuracy. This paper provides experimental evidence that the placement of the parts changes its build accuracy. We model the trajectory execution error of the manipulator for additive manufacturing. We validate these errors by comparing the predicted errors with the experimental errors. Finally, we present an algorithm to optimize the part placement for improving built part accuracy during robot-based additive manufacturing.

Composites are increasingly becoming a material of choice in the aerospace and automotive industries. Currently, many composite parts are produced by manually laying up sheets on complex molds. Composite sheet layup requires executing two main tasks: (1) grasping a sheet and (2) draping it on the mold. Automating the layup process requires automation of these two tasks. This paper is focused on the automation of the grasping task using robots. This requires an automated generation of grasp plans to enable robots to hold the sheet during the draping process. We present a simulation-based approach for determining robot grasp locations on the composite sheets. We also present an intervention controller that uses a real-time sheet tracking system during plan execution and can prevent failures. We demonstrate the performance of the developed system using a large complex part.

To effectively assist human workers in assembly tasks a robot must proactively offer support by inferring their preferences in sequencing the task actions. Previous work has focused on learning the dominant preferences of human workers for simple tasks largely based on their intended goal. However, people may have preferences at different resolutions: they may share the same high-level preference for the order of the sub-tasks but differ in the sequence of individual actions. We propose a two-stage approach for learning and inferring the preferences of human operators based on the sequence of sub-tasks and actions. We conduct an IKEA assembly study and demonstrate how our approach is able to learn the dominant preferences in a complex task. We show that our approach improves the prediction of human actions through cross-validation. Lastly we show that our two-stage approach improves the efficiency of task execution in an online experiment and demonstrate its applicability in a real-world robot-assisted IKEA assembly.

Multi-arm mobile manipulators can be represented as a combination of multiple robotic agents from the perspective of task-assignment and motion planning. Depending upon the task, agents might collaborate or work independently. Integrating motion planning with task-agent assignment is a computationally slow process as infeasible assignments can only be detected through expensive motion planning queries. We present three speed-up techniques for addressing this problem-(1) spatial constraint checking using conservative surrogates for motion planners, (2) instantiating symbolic conditions for pruning infeasible assignments, and (3) efficiently caching and reusing previously generated motion plans. We show that the developed method is useful for real-world operations that require complex interaction and coordination among high-DOF robotic agents.

High-performance composites are increasingly being used in the industry. Sheet layup is a process of manufacturing composite components using deformable sheets. We have developed a robotic cell to automate the layup process and overcome the limitations of the manual layup. Generating offline trajectories for robots and executing them without online refinement can introduce defects in the process due to uncertainties in the model of the sheet and environmental factors. Our system computes layup and grasping trajectories for the robots and refines them during the layup process based on the sensor data. We use an approach that augments physical experiments with simulations to train a Gaussian process regression model offline. The use of GPR enables us to quickly refine grasp plans and perform a defect-free layup without slowing down the layup process. We present experimental results on two components.

In a mission with considerable uncertainty due to intermittent communications, degraded information flow, and failures, humans need to assess both the current and expected future states, and update task assignments to robots as quickly as possible. We present a forward simulation-based alert system that proactively notifies the human supervisor of possible, negatively-impactful events, which provides an opportunity for the human to retask agents to avoid undesirable scenarios. We propose methods for speeding up mission simulations and extracting alerts from simulation data in order to enable real-time alert generation suitable for time-critical missions. We present the results from a user trial and verify our hypothesis that the decision making performance of human supervisors can be improved by introducing forward simulation-based alerts.

Determining a feasible path for nonholonomic mobile manipulators operating in congested environments is challenging. Sampling-based methods, especially bi-directional tree search-based approaches, are amongst the most promising candidates for quickly finding feasible paths. However, sampling uniformly when using these methods may result in high computation time. This paper introduces two techniques to accelerate the motion planning of such robots. The first one is coordinated focusing of samples for the manipulator and the mobile base based on the information from robot surroundings. The second one is a heuristic for making connections between the two search trees, which is challenging owing to the nonholonomic constraints on the mobile base. Incorporating these two techniques into the bi-directional RRT framework results in about 5x faster and 10x more successful computation of paths as compared to the baseline method.

Complex logistics support missions require the execution of spatially separated information gathering and situational awareness tasks. Mobile robot teams can play an important role in the automated execution of these tasks to reduce mission completion time. Planning strategies for such missions must take into account the formation of effective coalitions among available robots and assignment of tasks to robots with the goal of minimizing the expected mission completion time. The occurrence of unexpected situations that adversely interfere with the execution of the mission may require the execution of contingency tasks so that the originally planned tasks may proceed with minimal disruption. Initially reported potential contingency tasks may not always affect mission tasks due to the uncertainty in the mission environment. When potential contingency tasks are reported, the planner updates its existing plan to minimize the expected mission completion time based on the probability of these contingency tasks impacting the mission, their impact on the mission, and other task characteristics. We describe various heuristic-based strategies to compute task allocations for robots for mission execution. We perform simulation experiments to compare them and analyze the computational performance of the best performing strategy. We show that the proactive approach to contingency task management outperforms both the conservative and reactive approaches. Note to Practitioners-The work reported in this article will be useful for deploying multirobot teams to support complex logistics missions spread over a large area where the robots must be prepared to handle contingencies that can adversely impact the mission. The proposed proactive approach can be used to handle contingencies in information gathering, surveillance, guarding, and situational awareness tasks to support safe and secure transportation of important assets through crowded areas. We use port operation as an illustrative example where unmanned surface and aerial vehicles can be useful in ensuring the safety and security of the ports. This is a computationally challenging problem. This article proposes heuristic algorithms to solve the task allocation problem among many different agents efficiently. The approach presented in this article integrates the available information regarding mission and contingencies, along with the resource constraints to plan the mission execution.

Extrusion-based additive manufacturing (AM), also known as fused deposition modeling (FDM) extrudes filaments through a heated nozzle and builds a part layer-by-layer. Using a smaller diameter nozzle can achieve better surface finish. However, there is a trade-off between surface finish and build times as using a small diameter nozzle leads to smaller layer thickness and long build times. Traditional FDM printers create a part with planar layers, and this restricts control over fiber orientations. This paper presents a robotic cell for multi-resolution AM. The cell consists of two 6 degrees of freedom (DOF) robot manipulators capable of printing non-planar and/or planar layers. We describe algorithms for decomposing parts into multi-resolution layers and generating collision-free trajectories for the robot manipulators. We validate our approach by printing five parts with multi-resolution.

Currently, automatically generated trajectories cannot be directly used on tasks that require high execution accuracies due to errors accused by inaccuracies in the robot model, actuator errors, and controller limitations. These trajectories often need manual refinement. This is not economically viable on low production volume applications. Unfortunately, execution errors are dependent on the nature of the trajectory and end-effector loads, and therefore devising a general purpose automated compensation scheme for reducing trajectory errors is not possible. This paper presents a method for analyzing the given trajectory, executing an exploratory physical run for a small portion of the given trajectory, and learning a compensation scheme based on the measured data. The learned compensation scheme is context-dependent and can be used to reduce the execution error. We have demonstrated the feasibility of this approach by conducting physical experiments.

We pose the problem of path-constrained trajectory generation for the synchronous motion of multi-robot systems as a non-linear optimization problem. Our method determines appropriate parametric representation for the configuration variables, generates an approximate solution as a starting point for the optimization method, and uses successive refinement techniques to solve the problem in a computationally efficient manner. We have demonstrated the effectiveness of the proposed method on challenging simulation and physical experiments with high degrees of freedom robotic systems.

Successfully completing a complex manufacturing task requires finding a feasible placement of the workpiece in the robot workspace. The workpiece placement should be such that the task surfaces on the workpiece are reachable by the robot, the robot can apply the required forces, and the end-effector/tool can move with the desired velocity. This paper formulates the problem of identifying a feasible placement as a non-linear optimization problem over the constraint violation functions. This is a computationally challenging problem. We show that this problem can be solved by successively searching for the solution by incrementally applying different constraints. We demonstrate the feasibility of our approach using several complex workpieces.

To minimize the operation time, mobile manipulators need to pick-up parts while the mobile base and the gripper are moving. The gripper speed needs to be selected to ensure that the pick-up operation does not fail due to uncertainties in part pose estimation. This, in turn, affects the mobile base trajectory. This paper presents an active learning based approach to construct a meta-model to estimate the probability of successful part pick-up for a given level of uncertainty in the part pose estimate. Using this model, we present an optimization-based framework to generate time-optimal trajectories that satisfy the given level of success probability threshold for picking-up the part.

We present a context-dependent bi-directional tree-search framework for point-to-point path planning for manipulators. Conceptually, our framework is composed of six modules: tree selection, focus selection, node selection, target selection, extend selection and connection type selection. Each module consists of a set of interchangeable strategies. By exploiting synergistic interaction between these strategies and selecting appropriate strategies based the contextual cues from the search state, we show an instance of our framework that computes high-quality solutions in a variety of complex scenarios with a low failure rate. We also show that some popular path planning methods in the literature can be easily represented in our framework. We compare our approach with these popular methods in a diverse set of test scenarios. We report a 15-fold reduction in failure rate coupled with at least a 26% drop in solution suboptimality when compared to the best of the alternative methods.

In this paper, we investigate the use of surrogate agents to accelerate test scenario generation for autonomous vehicles. Our goal is to train the surrogate to replicate the true performance modes of the system. We create these surrogates by utilizing imitation learning with deep neural networks. By using imitator surrogates in place of the true agent, we are capable of predicting mission performance more quickly, gaining greater throughput for simulation-based testing. We demonstrate that using on-line imitation learning with Dataset Aggregation (DAgger) can not only correctly encode a policy that executes a complex mission, but can also encode multiple different behavioral modes. To improve performance for the target vehicle and mission, we manipulate the training set during each iteration to remove samples which do not contribute to the final policy. We call this approach Quantile-DAgger (Q-DAgger) and demonstrate its ability to replicate the behaviors of an autonomous vehicle in a collision avoidance scenario.

In most complex missions, unexpected situations arise that may interfere with the planned execution of mission tasks. These situations result in the generation of contingency tasks that need to be executed before the originally planned tasks are completed. Potential contingency tasks may not always affect mission tasks due to the inherent uncertainty in the environment. Deferring action on a potential contingency task may incur a penalty in terms of wasted time due to idle robots if the contingency task becomes a bottleneck in the future. On the other hand, immediate action on a potential contingency task may incur a penalty in terms of wasted time if the contingency task did not actually impact the mission. When a contingency task is reported, the planner generates an updated plan that minimizes expected mission completion time by taking into account the probability of the contingency task impacting mission tasks, its effect on the mission, and its spatial location. We have characterized the performance of the algorithm through simulation experiments. We show that the proactive approach to contingency task management outperforms a conservative approach.

We propose a scalable, parallelizable policy synthesis framework intended for a robot presented with the decision of exploration or rescue, given some time-varying, stochastic mission conditions, referred to as context. We demonstrate the feasibility of such a solution using physics-based simulations to synthesize a policy in a computationally-efficient manner and exhibit superior performance with regards to the minimization of probability of mission failure when compared to two feasible baseline approaches. Furthermore, we present preliminary results that suggest our approach is robust to errors in the state estimation used to build mission context, which further supports the notion of real-world applicability.

In this work, we present a Stochastic Gradient Ascent (SGA) algorithm for multi-vehicle information gathering that accounts for limitations on a vehicle's control authority caused by external forces. By representing vehicle paths using a novel action space representation, rather than a state space representation, we remove the need to perform feasibility calculations on the vehicle's path. Our algorithm uses a stochastic optimization scheme by sampling perturbed action sequences around the current best known sequence to estimate the gradient of a state space information function with respect to the action sequence. Additionally, we use sequential greedy allocation to plan for multiple vehicles. Results are shown using a Navy Coastal Ocean Model (NCOM) for the Gulf of Mexico (GoM). SGA shows improvement in the amount of information gained over a greedy baseline. Additionally, we compare to Monte Carlo Tree Search (MCTS) Method, which is able to gather competitive amounts of information but is more computationally intensive than our approach.

Use of robots is rising in process applications where robots need to interact with parts using tools. Representative examples can be cleaning, polishing, grinding, etc. These tasks can be non-repetitive in nature and the physics-based models of the task performances are unknown for new materials and tools. In order to reduce operation cost and time, the robot needs to identify and optimize the trajectory parameters. The trajectory parameters that influence the performance can be speed, force, torque, stiffness, etc. Building physics-based models may not be feasible for every new task, material, and tool profile as it will require conducting a large number of experiments. We have developed a method that identifies the right set of parameters to optimize the task objective and meet performance constraints. The algorithm makes decisions based on uncertainty in the surrogate model of the task performance. It intelligently samples the parameter space and selects a point for experimentation from the sampled set by determining its probability to be optimum among the set. The iterative process leads to rapid convergence to the optimal point with a small number of experiments. We benchmarked our method against other optimization methods on synthetic problems. The method has been validated by conducting physical experiments on a robotic cleaning problem. The algorithm is general enough to be applied to any optimization problem involving black box constraints.

Motion planning for multi-target autonomous search requires efficiently gathering as much information over an area as possible with an imperfect sensor. In disaster scenarios and contested environments the spatial connectivity may unexpectedly change (due to aftershock, avalanche, flood, building collapse, adversary movements, etc.) and the flight envelope may evolve as a known function of time to ensure rescue worker safety or to facilitate other mission goals. Algorithms designed to handle both expected and unexpected changes must: (1) reason over a sufficiently long time horizon to respect expected changes, and (2) replan quickly in response to unexpected changes. These ambitions are hindered by the submodularity property of mutual information, which makes optimal solutions NP-hard to compute. We present an algorithm for autonomous search in changing environments that uses a variety of techniques to improve both the speed and time horizon, including using e-admissible heuristics to speed up the search.

We propose a novel method for generating test scenarios for a black box autonomous system that demonstrate critical transitions in its performance modes. In complex environments it is possible for an autonomous system to fail at its assigned mission even if it complies with requirements for all subsystems and throws no faults. This is particularly true when the autonomous system may have to choose between multiple exclusive objectives. The standard approach of testing robustness through fault detection is directly stimulating the system and detecting violations of the system requirements. Our approach differs by instead running the autonomous system through full missions in a simulated environment and measuring performance based on high-level mission criteria. The result is a method of searching for challenging scenarios for an autonomous system under test that exercise a variety of performance modes. We utilize adaptive sampling to intelligently search the state space for test scenarios which exist on the boundary between distinct performance modes. Additionally, using unsupervised clustering techniques we can group scenarios by their performance modes and sort them by those which are most effective at diagnosing changes in the autonomous system's behavior.

This paper presents planning algorithms for robotic cleaning of stains on nonplanar surfaces. Access to different portions of the stain may require frequent repositioning and reorienting of the object. Some portions with prominent stain may require multiple passes to remove the stain completely. Two robotic arms have been used in the experiments. The object is immobilized with one arm and the cleaning tool is manipulated with the other. The algorithm generates a sequence of reorientation and repositioning moves required to clean the part after analyzing the stain. The plan is generated by accounting for the kinematic constraints of the robot. Our algorithm uses a depth-first branch-and-bound search to generate setup plans. Cleaning trajectories are generated and optimal cleaning parameters are selected by the algorithm. We have validated our approach through numerical simulations and robotic cleaning experiments with two KUKA robots.

Quadruped locomotion offers significant advantages over wheeled locomotion for small mobile robots operating in challenging terrain. Central pattern generators (CPGs), as found in the neural circuitry of many animals, may be used to generate joint trajectories for quadruped robots. However, basic CPG-based trajectories do not explicitly consider ground contact constraints, a particular concern during turning maneuvers when foot slip is most likely to occur. An alternative approach proposed here is to use task-based CPGs such that ground contact constraints are enforced and foot velocities are explicitly controlled, resulting in stable omnidirectional locomotion. Further, incorporating Virtual Model Control with the task-based CPG trajectories improves the stability of the quadruped in hardware experiments.

In this paper, we consider the problem of guarding a valuable naval asset from a highly maneuverable threat via the use of autonomous unmanned surface vehicles (USVs) as dynamic obstacles. The objective of the defending agent is to maximize the amount of time it takes an intruder boat to enter the restricted area. Here we introduce a set of active blocking strategies which allow the defender to influence the intruder's actions through exploitation of its obstacle avoidance. By applying a mirror transformation to the intruder's current state, a pursuit policy can be formed that always places the defender between the intruder and the asset. To apply these strategies to delayed or imperfect information environments, we present a model-predictive method of calculating defender motion goals and estimating future intruder motions. This predictive estimator is capable of generating a meta-model for the intruder's behavior and returning a probability distribution of intruder control actions based on a reduced set of observable spatial features.

Unmanned vehicles are emerging as an attractive tool for persistent monitoring tasks of a given area, but need automated planning capabilities for effective unattended deployment. Such an automated planner needs to generate collision-free coverage paths by steering waypoints to locations that both minimize the path length and maximize the amount of information gathered along the path. The approach presented in this paper significantly extends prior work and handles motion uncertainty of an unmanned vehicle and the presence of obstacles by using a Markov Decision Process based approach to generate collision-free paths. Simulation results show that the proposed approach is robust to significant motion uncertainties and reduces the probability of collision with obstacles in the environment.

This paper describes RoboTerp, a quadrupedal amphibious robot that achieves locomotion on land and in water with the same legs by switching gaits to match the terrain. The central idea hinges on a passive compliant mechanism attached to the lower leg that enables it to behave like a valve during movement in water. The direction of this valve-like mechanism is aligned such that rhythmic oscillations of the legs generate a net thrust that propels the robot forward in water. By design, this oscillatory leg movement achieves splash-free swimming, and thereby overcomes the shortcomings of most previous wheel-leg based designs, in which rotational movement causes water splashing that leads to significant turbulence in the robot surroundings. We examined different materials and morphological parameters to select the best flap configuration. A modular design allowed rapid iterations of these experiments. We confirmed the performance of the best few configurations found during the experiments through fluid simulations. Finally, we report successful demonstrations of RoboTerp walking on asphalt land, swimming in a pool, and transitioning between uneven rock surface and water in an outdoor creek.

We introduce a model-predictive trajectory planning algorithm for unmanned surface vehicles (USVs) operating in congested civilian traffic. The planner reasons about the availability of contingency maneuvers needed in case of any of the civilian vessels breaches the International Regulations for the Prevention of Collisions at Sea (COLREGs). Our exploratory study indicated that implementing the envisioned planner requires significant speed up of trajectory planning to cope with the dynamics of the scene, and evaluation of collision risk. We describe a new method for efficiently searching 5D state space for a dynamically feasible trajectory using adaptive control action primitives. The algorithm estimates the congestion of the state space regions to evaluate collision risk, and then dynamically scales action primitives used during the search while preserving their dynamical feasibility. Our simulation experiments demonstrate that this leads to a substantial increase in the search efficiency and a decrease in the number of collisions, especially in complex scenarios with a higher number of civilian vessels.

The capability of noninvasive and precise micromanipulation of sensitive, living cells is necessary for understanding their underlying biological processes. Optical tweezers (OT) is an effective tool that uses highly focused laser beams for accurate manipulation of cells and dielectric beads at microscale. However, direct exposure of the laser beams on the cells can negatively influence their behavior or even cause a photo-damage. In this paper, we introduce a control and planning approach for automated, indirect manipulation of cells using silica beads arranged into gripper formations. The developed approach employs path planning and feedback control for efficient, collision-free transport of a cell between two specified locations. The planning component of the approach computes a path that explicitly respects the nonholonomic constraints of the gripper formations. The feedback control component ensures stable tracking of the path by manipulating the cell using a set of predefined maneuvers. We demonstrate the effectiveness of the approach by transporting a yeast cell using four different types of gripper formations along collision-free paths on our OT setup. We analyzed the performance of the proposed gripper formations with respect to their maximum transport speeds and the laser intensity experienced by the cell that depends on the laser power used.

Studying collective migration of cells is currently of considerable interest in biology and medicine leading to possibility of novel diagnosis and treatments. Some cells are highly sensitive to direct laser exposure, which may influence their behavior or even cause photodamage. In addition, manual manipulation of cells is time consuming making it hard to carry out systematic studies that are properly timed to exhibit the desired motility. This paper presents an automated planning approach for precise, collision-free, indirect manipulation of cells with irregular, dynamically changing shapes using Optical Tweezers (OT). We have evaluated the effectiveness of our manipulation approach using physical experiments. We have also carried out an experimental study to demonstrate the effect of the indirect manipulation approach on cell-viability.

In this paper, we present a heuristic planning approach for guarding a valuable asset by a team of autonomous unmanned surface vehicles (USVs) operating in a continuous state-action space. The team's objective is to maximize the amount of time it takes an intruder boat to reach the asset. The team must cooperatively deal with uncertainty about which boats are actual intruders, employ active blocking to slow down intruders' movement towards the asset, and intelligently distribute themselves around the target to optimize future guarding opportunities. Our planner incorporates a market-based algorithm for allocating tasks to individual USVs by forward-simulating the mission and assigning estimated utilities to candidate task-allocation plans. The planner can be automatically adapted to a specific mission by optimizing the behaviors used to fulfil individual tasks. We present detailed simulation results that demonstrate the effectiveness of our approach.

We present a model-predictive trajectory planning algorithm for following a target boat by an autonomous unmanned surface vehicle (USV) in an environment with static obstacle regions and civilian boats. The planner developed in this work is capable of making a balanced trade-off among the following, possibly conflicting criteria: the risk of losing the target boat, trajectory length, risk of collision with obstacles, violation of the Coast Guard Collision Regulations (COLREGs), also known as “rules of the road”, and execution of avoidance maneuvers against vessels that do not follow the rules. The planner addresses these criteria by combining a search for a dynamically feasible trajectory to a suitable pose behind the target boat in 4D state space, forming a time-extended lattice, and reactive planning that tracks this trajectory using control actions that respect the USV dynamics and are compliant with COLREGs. The reactive part of the planner represents a generalization of the velocity obstacles paradigm by computing obstacles in the control space using a system-identified, dynamic model of the USV as well as worst-case and probabilistic predictive motion models of other vessels. We present simulation and experimental results using an autonomous unmanned surface vehicle platform and a human-driven vessel to demonstrate that the planner is capable of fulfilling the above mentioned criteria.

Manipulation of microscopic objects, especially biological objects and microelectromechanical systems (MEMS) components, has become an important area of robotics research over the past several years. Automation is necessary as it is challenging to manually control the microobjects due to the scaling effect of the surface forces, stochastic motion of objects in fluid media, and uncertainty associated with object state estimation. Automation requires real-time control of the position, orientation, and force applied by each of the operational manipulators, as governed by the system-level objectives of optimizing resource, time, and effort, by planning suitable actions for the manipulated objects. In this paper, we provide a survey of the research in planning and control of such automated micromanipulation operations. We present a broad taxonomy based on the underlying approach, and discuss the salient features and experimental success of each research effort. We also identify the major limitations and common trends across all the approaches, discuss the effectiveness of an approach depending on the operation characteristics, and outline promising future research directions.

In this paper, we present a physics-aware, planning approach for automated transport of cells in an optical tweezers-assisted microfluidic chamber. The approach can be used for making a uniform distribution of cells inside the chamber to allow the study of a variety of biological processes, including cell signaling. Fluid forces inside the chamber, modeled using computational fluid dynamics, are incorporated into the widely used Langevin equation to simulate the motion of cells. The developed simulator was used for building a map that contains probabilities of a cell successfully reaching one of the outlets of the chamber from different locations under the influence of the fluid flow. The developed planner not only generates collision-free paths that exploit the fluid flow inside the chamber but also utilizes the offline generated simulation data to decide suitable locations for releasing the cells. This ensures fast and robust cell transport, while minimizing the required laser power and operational time. The planner is based on the heuristic D* Lite algorithm that employs a specific cost function for searching over a novel state-action space representation. The effectiveness of the planning algorithm is demonstrated using both simulation and physical experiments in a microfluidics-optical tweezers hybrid manipulation setup.

Optical Tweezers (OT) are used for highly accurate manipulations of biological cells. However, the direct exposure of cells to focused laser beam may negatively influence their biological functions. In order to overcome this problem, we generate multiple optical traps to grab and move a 3D ensemble of inert particles such as silica microspheres to act as a reconfigurable gripper for a manipulated cell. The relative positions of the microspheres are important in order for the gripper to be robust against external environmental forces and the exposure of high intensity laser on the cell to be minimized. In this paper, we present results of different gripper configurations, experimentally tested using our OT setup, that provide robust gripping as well as minimize laser intensity experienced by the cell. We developed a computational approach that allowed us to perform preliminary modeling and synthesis of the gripper configurations. The gripper synthesis is cast as a multi-objective optimization problem.

The IEEE RAS Ontologies for Robotics and Automation Working Group is dedicated to developing a methodology for knowledge representation and reasoning in robotics and automation. As part of this working group, the Industrial Robots sub-group is tasked with studying industrial applications of the ontology. One of the first areas of interest for this subgroup is the area of kit building or kitting. This is a process that brings parts that will be used in assembly operations together in a kit and then moves the kit to the assembly area where the parts are used in the final assembly. This paper examines the knowledge representations that have been developed and implemented for the kitting problem.

Automated transport of multiple particles using optical tweezers requires real-time path planning to move them in coordination by avoiding collisions among themselves and with randomly moving obstacles. This paper develops a decoupled and prioritized path planning approach by sequentially applying a partially observable Markov decision process algorithm on every particle that needs to be transported. We use an iterative version of a maximum bipartite graph matching algorithm to assign given goal locations to such particles. We then employ a three-step method consisting of clustering, classification, and branch and bound optimization to determine the final collision-free paths. We demonstrate the effectiveness of the developed approach via experiments using silica beads in a holographic tweezers setup. We also discuss the applicability of our approach and challenges in manipulating biological cells indirectly by using the transported particles as grippers.

This paper describes a new process for fabricating planar, multi-material, compliant mechanisms, intended for use in small scale robotics. The process involves laser cutting the mechanism geometry from a rigid material, and refilling the joint areas with a second, elastomeric material. This method allows for a large set of potential materials, with a wide range of material properties, to be used in combination to create mechanisms with highly tailored mechanical properties. These multi-material compliant mechanisms have minimum feature sizes of approximately 100 µm and have demonstrated long lifetimes, easily surviving 100,000 bending cycles. We also present the first use of these compliant mechanisms in a 2.5cm × 2.5cm × 7.5cm, 6g hexapod. This hexapod has been demonstrated moving at speeds up to 6 cm/s, with a predicted maximum speed of up to 17 cm/s.

We present a look-ahead based trajectory planning algorithm for computation of dynamically feasible trajectories for Unmanned Sea Surface Vehicles (USSV) operating in high seas states. The algorithm combines A* based heuristic search and locally bounded optimal planning under motion uncertainty using a variation of the minimax game-tree search. This allows the algorithm to compute trajectories that explicitly consider the possibility of the vehicle safely deviating from its original course due to the ocean waves within a specified look-ahead region. The algorithm can adapt its search based on the user-specified risk thresholds. Moreover, the algorithm produces a contingency plan as a part of the computed trajectory. We demonstrate the capabilities of the algorithm using simulations.

Automated particle transport using optical tweezers requires the use of motion planning to move the particle while avoiding collisions with randomly moving obstacles. This paper describes a stochastic dynamic programming based motion planning framework developed by modifying the discrete version of an infinite-horizon partially observable Markov decision process algorithm. Sample trajectories generated by this algorithm are presented to highlight effectiveness in crowded scenes and flexibility. The algorithm is tested using silica beads in a holographic tweezer set-up and data obtained from the physical experiments are reported to validate various aspects of the planning simulation framework. This framework is then used to evaluate the performance of the algorithm under a variety of operating conditions.