EDBT 2026 Demo / reviewers in the wild / expert
Kenneth Y. Goldberg
dblp:g/KennethYGoldberg · also Ken Goldberg
· DBLP profile ↗
261ranked-venue papers
26as first author
76since 2021 · last 2025
—ORCID · conflict
Domains — the database's venue-derived domains; a paper can count in several
Artificial intelligence and machine learning · 192 · 11 first-author · 69 since 2021Systems, architecture and hardware · 152 · 9 first-author · 54 since 2021Applied, interdisciplinary, general and emerging computing · 36 · 9 first-author · 7 since 2021Databases, data management, data science and information retrieval · 11 · 1 first-authorGraphics, computer vision, multimedia, augmented reality and games · 9 · 1 first-author · 2 since 2021Theory of computation · 9 · 2 first-authorHuman-computer interaction and ubiquitous computing · 8 · 1 first-authorComputer networks · 1 · 1 first-author
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2025 | OTTER: A Vision-Language-Action Model with Text-Aware Visual Feature ExtractionabstractVision-Language-Action (VLA) models aim to predict robotic actions based on visual observations and language instructions. Existing approaches require fine-tuning pre-trained vision-language models (VLMs) as visual and language features are independently fed into downstream policies, degrading the pre-trained semantic alignments. We propose OTTER, a novel VLA architecture that leverages these existing alignments through explicit, text-aware visual feature extraction. Instead of processing all visual features, OTTER selectively extracts and passes only task-relevant visual features that are semantically aligned with the language instruction to the policy transformer. This allows OTTER to keep the pre-trained vision-language encoders frozen. Thereby, OTTER preserves and utilizes the rich semantic understanding learned from large-scale pre-training, enabling strong zero-shot generalization capabilities. In simulation and real-world experiments, OTTER significantly outperforms existing VLA models, demonstrating strong zero-shot generalization to novel objects and environments. Video, code, checkpoints, and dataset: https://ottervla.github.io/. Fangchen Liu, Letian Fu, Tingfan Wu, Mustafa Mukadam, Jitendra Malik, Kenneth Y. Goldberg, Pieter Abbeel |
ICML | 7 |
| 2025 | Robo-DM: Data Management for Large Robot DatasetsabstractRecent results suggest that very large datasets of teleoperated robot demonstrations can be used to train transformer-based models that have the potential to generalize to new scenes, robots, and tasks. However, curating, distributing, and loading large datasets of robot trajectories, which typically consist of video, textual, and numerical modalities - including streams from multiple cameras - remains challenging. We propose Robo-DM, an efficient open-source cloud-based data management toolkit for collecting, sharing, and learning with robot data. With Robo-DM, robot datasets are stored in a self-contained format with Extensible Binary Meta Language (EBML). Robo-DM can significantly reduce the size of robot trajectory data, transfer costs, and data load time during training. Compared to the RLDS format used in OXE datasets, Robo-DM's compression saves space by up to 70x (lossy) and 3.5x (lossless). Robo-DM also accelerates data retrieval by load-balancing video decoding with memory-mapped decoding caches. Compared to LeRobot, a framework that also uses lossy video compression, Robo-DM is up to 50x faster when decoding sequentially. We physically evaluate a model trained by Robo-DM with lossy compression, a pick-and-place task, and In-Context Robot Transformer. Robo-DM uses 75x compression of the original dataset and does not suffer reduction in downstream task accuracy. Code and evaluation scripts can be found on website https://github.com/BerkeleyAutomation/fog_x. Kaiyuan Chen 0001, Letian Fu, Yunliang Chen 0001, Kush Hari, Ashwin Balakrishna, Ted Xiao, Pannag R. Sanketi, John Kubiatowicz, Kenneth Y. Goldberg |
ICRA | 12 |
| 2025 | FogROS2-PLR: Probabilistic Latency-Reliability for Cloud RoboticsabstractCloud robotics enables robots to offload computationally intensive tasks to cloud servers for performance, cost, and ease of management. However, the network and cloud computing infrastructure are not designed for reliable timing guarantees, due to fluctuating Quality-of-Service (QoS). In this work, we formulate an impossibility triangle theorem for: Latency reliability, Singleton server, and Commodity hardware. The LSC theorem suggests that providing replicated servers with uncorrelated failures can exponentially reduce the probability of missing a deadline. We present FogROS2-Probabilistic Latency Reliability (PLR) that uses multiple independent network interfaces to send requests to replicated cloud servers and uses the first response back. We design routing mechanisms to discover, connect, and route through non-default network interfaces on robots. FogROS2-PLR optimizes the selection of interfaces to servers to minimize the probability of missing a deadline. We conduct a cloud-connected driving experiment with two 5 G service providers, demonstrating FogROS2-PLR effectively provides smooth service quality even if one of the service providers experiences low coverage and base station handover. We use 99 Percentile (P99) latency to evaluate anomalous long-tail latency behavior. In one experiment, FogROS2-PLR improves P99 latency by up to 3.7 x compared to using one service provider. We deploy FogROS2-PLR on a physical Stretch 3 robot performing an indoor human-tracking task. Even in a fully covered$\text{Wi}-\text{Fi}$and 5 G environment, FogROS2-PLR improves the responsiveness of the robot reducing mean latency by 36% and P99 latency by 33%. Code and supplementary can be found on website11https://github.com/data-capsule/rt-fogros2. Kaiyuan Chen 0001, Nan Tian, Christian Juette, Tianshuang Qiu, Liu Ren 0001, John Kubiatowicz, Kenneth Y. Goldberg |
ICRA | 7 |
| 2025 | ICRT: In-Context Imitation Learning via Next-Token PredictionabstractIn-context imitation learning is the capability to perform novel tasks when prompted with task demonstration examples. In-Context Robot Transformer (ICRT) is a causal transformer that performs autoregressive prediction on sensorimotor trajectories, which include images, proprioceptive states, and actions. This approach supports flexible and training-free execution of new tasks at test time. Experiments with a Franka Emika robot demonstrate that ICRT can adapt to new environment configurations that differ from both the prompt and the training data. In a multi-task environment setup, ICRT significantly outperforms current state-of-the-art robot foundation models on generalization to unseen tasks. Code, data, and appendix are available on https://icrt.dev. Max Fu, Gaurav Datta, Yunliang Chen 0001, Will Panitch, Fangchen Liu, Kenneth Y. Goldberg |
ICRA | 8 |
| 2025 | Blox-Net: Generative Design-for-Robot-Assembly Using VLM Supervision, Physics Simulation, and a Robot with ResetabstractGenerative AI systems have shown impressive capabilities in creating text, code, and images. Inspired by the importance of research in industrial Design for Assembly, we introduce a novel problem: Generative Design-for-RobotAssembly (GDfRA). The task is to generate an assembly based on a natural language prompt (e.g., “giraffe”) and an image of available physical components, such as 3D-printed blocks. The output is an assembly, a spatial arrangement of these components, accompanied by instructions for a robot to build it. The output geometry must 1) resemble the requested object and 2) be reliably assembled by a 6 DoF robot arm with a suction gripper. We then present Blox-Net, a GDfRA system that combines generative vision language models with well-established methods in computer vision, simulation, perturbation analysis, motion planning, and physical robot experimentation to solve a class of GDfRA problems without human supervision. Blox-Net achieved a Top-1 accuracy of$\mathbf{6 3. 5 \%}$in the semantic accuracy of its designed assemblies. Six designs, after Blox-Net's automated pertubation redesign, were reliably assembled by a robot, achieving near-perfect success across$\mathbf{1 0}$consecutive assembly iterations with human intervention only during reset prior to assembly. The entire pipeline from the textual word to reliable physical assembly is performed without human intervention. Project Page: https://bloxnet.org/ Andrew Goldberg 0001, Kavish Kondap, Tianshuang Qiu, Zehan Ma, Letian Fu, Justin Kerr, Kaiyuan Chen 0001, Kuan Fang, Kenneth Y. Goldberg |
ICRA | 10 |
| 2025 | SuFIA-BC: Generating High Quality Demonstration Data for Visuomotor Policy Learning in Surgical SubtasksabstractBehavior cloning facilitates the learning of dexterous manipulation skills, yet the complexity of surgical environments, the difficulty and expense of obtaining patient data, and robot calibration errors present unique challenges for surgical robot learning. We provide an enhanced surgical digital twin with photorealistic human anatomical organs, integrated into a comprehensive simulator designed to generate high-quality synthetic data to solve fundamental tasks in surgical autonomy. We present SuFIA-BC: visual Behavior Cloning policies for Surgical First Interactive Autonomy Assistants. We investigate visual observation spaces including multi-view cameras and 3D visual representations extracted from a single endoscopic camera view. Through systematic evaluation, we find that the diverse set of photorealistic surgical tasks introduced in this work enables a comprehensive evaluation of prospective behavior cloning models for the unique challenges posed by surgical environments. We observe that current state-of-the-art behavior cloning techniques struggle to solve the contact-rich and complex tasks evaluated in this work, regardless of their underlying perception or control architectures. These findings highlight the importance of customizing perception pipelines and control architectures, as well as curating larger-scale synthetic datasets that meet the specific demands of surgical tasks. Project website: orbit-surgical.github.io/sufia-bc/ Masoud Moghani, Nigel Nelson, Mohamed Ghanem, Andres Diaz-Pinto, Kush Hari, Mahdi Azizian, Kenneth Y. Goldberg, Sean Huver, Animesh Garg |
ICRA | 7 |
| 2025 | Energy Efficient Planning for Repetitive Heterogeneous Tasks in Precision AgricultureabstractRobotic weed removal in precision agriculture introduces a repetitive heterogeneous task planning (RHTP) challenge for a mobile manipulator. RHTP has two unique characteristics: 1) an observe-first-and-manipulate-later (OFML) temporal constraint that forces a unique ordering of two different tasks for each target and 2) energy savings from efficient task collocation to minimize unnecessary movements. RHTP can be framed as a stochastic renewal process. According to the Renewal Reward Theorem, the expected energy usage per task cycle is the long-run average. Traditional task and motion planning focuses on feasibility rather than optimality due to the unknown object and obstacle position prior to execution. However, the known target/obstacle distribution in precision agriculture allows minimizing the expected energy usage. For each instance in this renewal process, we first compute task space partition, a novel data structure that computes all possibilities of task multiplexing and its probabilities with robot reachability. Then we propose a region-based setcoverage problem to formulate the RHTP as a mixed-integer nonlinear programming. We have implemented and solved RHTP using Branch-and-Bound solver. Compared to a baseline in simulations based on real field data, the results suggest a significant improvement in path length, number of robot stops, overall energy usage, and number of replans. Shuangyu Xie, Kenneth Y. Goldberg, Dezhen Song |
ICRA | 2 |
| 2025 | Persistent Object Gaussian Splat (POGS) for Tracking Human and Robot Manipulation of Irregularly Shaped ObjectsabstractTracking and manipulating irregularly-shaped, previously unseen objects in dynamic environments is important for robotic applications in manufacturing, assembly, and logistics. Recently introduced Gaussian Splats [1] efficiently model object geometry, but lack persistent state estimation for taskoriented manipulation. We present Persistent Object Gaussian Splat (POGS), a system that embeds semantics, self-supervised visual features, and object grouping features into a compact representation that can be continuously updated to estimate the pose of scanned objects. POGS updates object states without requiring expensive rescanning or prior CAD models of objects. After an initial multi-view scene capture and training phase, POGS uses a single stereo camera to integrate depth estimates along with self-supervised vision encoder features for object pose estimation. POGS supports grasping, reorientation, and natural language-driven manipulation by refining object pose estimates, facilitating sequential object reset operations with human-induced object perturbations and tool servoing, where robots recover tool pose despite tool perturbations of up to 30°. POGS achieves up to 12 consecutive successful object resets and recovers from 80% of in-grasp tool perturbations. Justin Yu, Kush Hari, Karim El-Refai, Arnav Dalal, Justin Kerr, Chung Min Kim, Richard Cheng, Muhammad Zubair Irshad, Kenneth Y. Goldberg |
ICRA | 9 |
| 2025 | Botany-Bot: Digital Twin Monitoring of Occluded and Underleaf Plant Structures with Gaussian SplatsabstractCommercial plant phenotyping systems using fixed cameras cannot perceive many plant details due to leaf occlusion. In this paper, we present Botany-Bot, a system for building detailed “annotated digital twins” of living plants using two stereo cameras, a digital turntable inside a lightbox, an industrial robot arm, and 3D segmentated Gaussian Splat models. We also present robot algorithms for manipulating leaves to take high-resolution indexable images of occluded details such as stem buds and the underside/topside of leaves. Results from experiments suggest that Botany-Bot can segment leaves with 90.8% accuracy, detect leaves with 86.2% accuracy, lift/push leaves with 77.9% accuracy, and take detailed overside/underside images with 77.3% accuracy. Code, videos, and datasets are available at https://berkeleyautomation.github.io/Botany-Bot/. Simeon Adebola, Chung Min Kim, Justin Kerr, Shuangyu Xie, Prithvi Akella, Jose Luis Susa Rincon, Eugen Solowjow, Kenneth Y. Goldberg |
IROS | 8 |
| 2025 | Surgical D-Knot: Augmented Dexterity for Tying Double Knots by Monitoring Optical Flow in Monocular Attention WindowsabstractKnot tying is a fundamental dexterous surgical subtask that is a key step in suturing. One challenge to robot augmentation is limited depth perception due to the small baseline of surgical endoscopic cameras. In this work, we present Surgical D-Knot: an augmented dexterity pipeline combining learned perception with model-based methods to perform surgical double knots using only one monocular RGB camera. This pipeline includes 2D grasp point identification, 3D suture thread grasping using local feature servoing, suture thread wrapping using relative motion and 2D re-grasp point identification. Human dexterity is required for initial thread setup and thread cutting after each double knot. Physical experiments with 120 double knot trials result in a success rate of 80.83% for the initial knot and 55.83% for the second knot. Translation of surgical knot tying to chicken skin results in success rates of 73.75% for the initial knot and 40% for the second knot. Each double knot requires on average 70 seconds. This is the first work to our knowledge that augments human dexterity for double knot tying. https://sites.google.com/view/surgicaldknot/ Kush Hari, Tanmayi Dasari, Karen Shieh, Ria Jain, Danyal Fer, Gary Guthart, Kenneth Y. Goldberg |
IROS | 8 |
| 2025 | PyRoki: A Modular Toolkit for Robot Kinematic OptimizationabstractRobot motion can have many goals. Depending on the task, we might optimize for pose error, speed, collision, or similarity to a human demonstration. Motivated by this, we present PyRoki: a modular, extensible, and deviceagnostic toolkit for solving kinematic optimization problems. PyRoki couples an interface for specifying kinematic variables and costs with an efficient nonlinear least squares optimizer. Unlike existing tools, it is also device-agnostic: optimization runs natively on CPU, GPU, and TPU. In this paper, we present (i) the design and implementation of PyRoki, (ii) motion retargeting and planning case studies that highlight the advantages of PyRoki’s modularity, and (iii) optimization benchmarking, where PyRoki can be 1.4-1.7x faster and converges to lower errors than cuRobo, an existing GPU-accelerated inverse kinematics library. The code is open-sourced at https://pyroki-toolkit.github.io. Chung Min Kim, Brent Yi, Hongsuk Choi, Yi Ma 0001, Kenneth Y. Goldberg, Angjoo Kanazawa |
IROS | 5 |
| 2025 | Omni-Scan: Creating Visually-Accurate Digital Twin Object Models Using a Bimanual Robot with Handover and Gaussian Splat Mergingabstract3D Gaussian Splats (3DGSs) are 3D object models derived from multi-view images. Such “digital twins” are useful for simulations, virtual reality, E-commerce, robot policy fine-tuning, and part inspection. 3D object scanning usually requires multi-camera arrays, precise laser scanners, or robot wrist-mounted cameras, which have restricted workspaces. We propose Omni-Scan, a pipeline for producing high-quality 3D Gaussian Splat models using a bi-manual robot that grasps an object with one gripper and rotates the object with respect to one stationary camera. The object is then re-grasped by a second gripper to expose surfaces that were occluded by the first gripper. We present the Omni-Scan robot pipeline using DepthAnything, Segment Anything, as well as RAFT optical flow models to identify and isolate objects held by a robot gripper while removing the gripper and the background. We then modify the 3DGS training pipeline to support concatenated datasets with gripper occlusion, producing an omni-directional (360°) model of the object. We apply Omni-Scan to part defect inspection, finding that it can identify visual or geometric defects in 12 different industrial and household objects with an average accuracy of 83.3%. More details and interactive videos of Omni-Scan 3DGS models can be found at https://berkeleyautomation.github.io/omni-scan/. Tianshuang Qiu, Zehan Ma, Karim El-Refai, Hiya Shah, Chung Min Kim, Justin Kerr, Kenneth Y. Goldberg |
IROS | 7 |
| 2025 | Robo2VLM: Improving Visual Question Answering using Large-Scale Robot Manipulation DataabstractVision-Language Models (VLMs) acquire real-world knowledge and general reasoning ability through Internet-scale image-text corpora. They can augment robotic systems with scene understanding and task planning, and assist visuomotor policies that are trained on robot trajectory data. We explore the reverse paradigm — using rich, real, multi-modal robot trajectory data to enhance and evaluate VLMs. In this paper, we present Robo2VLM, a Visual Question Answering (VQA) dataset generation framework for VLMs. Given a human tele-operated robot demonstration with video and robot data, Robo2VLM derives ground-truth from non-visual and non-descriptive sensory modalities, such as end-effector pose, gripper aperture, and force sensing. Based on these modalities, it segments the robot trajectory into a sequence of manipulation phases. At each phase, Robo2VLM uses scene and interaction understanding to identify 3D properties of the robot, task goal, and the target object. The properties are used to generate representative VQA queries – images with textural multiple-choice questions – based on spatial, goal-conditioned, and interaction reasoning question templates. We use a subset of Open X-Embodiment to generate Robo2VLM-1, a large-scale in-the-wild dataset with 684,710 questions based on 463 distinct scenes and 3,396 robotic manipulation tasks from 176k real robot trajectories. Results suggest that Robo2VLM-1 can benchmark and improve VLM capabilities in spatial and interaction reasoning. Kaiyuan Chen 0001, Shuangyu Xie, Zehan Ma, Pannag R. Sanketi, Kenneth Y. Goldberg |
NeurIPS | 5 |
| 2024 | GARField: Group Anything with Radiance FieldsabstractGrouping is inherently ambiguous due to the multiple levels of granularity in which one can decompose a sceneshould the wheels of an excavator be considered separate or part of the whole? We propose Group Anything with Radiance Fields (GARField), an approach for decomposing 3D scenes into a hierarchy of semantically meaningful groups from posed image inputs. To do this we embrace group ambiguity through physical scale: by optimizing a scaleconditioned 3D affinity feature field, a point in the world can belong to different groups of different sizes. We optimize this field from a set of 2D masks provided by Segment Anything (SAM) in a way that respects coarse-to-fine hierarchy, using scale to consistently fuse conflicting masks from different viewpoints. From this field we can derive a hierarchy of possible groupings via automatic tree construction or user interaction. We evaluate GARField on a variety of in-the-wild scenes and find it effectively extracts groups at many levels: clusters of objects, objects, and various subparts. GARField inherently represents multi-view consistent groupings and produces higher fidelity groups than the input SAM masks. GARField's hierarchical grouping could have exciting downstream applications such as 3D asset extraction or dynamic scene understanding. Project site: https://www.garfield.studio/ Chung Min Kim, Mingxuan Wu, Justin Kerr, Kenneth Y. Goldberg, Matthew Tancik, Angjoo Kanazawa |
CVPR | 4 |
| 2024 | A Touch, Vision, and Language Dataset for Multimodal AlignmentabstractTouch is an important sensing modality for humans, but it has not yet been incorporated into a multimodal generative language model. This is partially due to the difficulty of obtaining natural language labels for tactile data and the complexity of aligning tactile readings with both visual observations and language descriptions. As a step towards bridging that gap, this work introduces a new dataset of 44K in-the-wild visiontouch pairs, with English language labels annotated by humans (10%) and textual pseudo-labels from GPT-4V (90%). We use this dataset to train a vision-language-aligned tactile encoder for open-vocabulary classification and a touch-visionlanguage (TVL) model for text generation using the trained encoder. Results suggest that by incorporating touch, the TVL model improves (+29% classification accuracy) tactile-vision-language alignment over existing models trained on any pair of those modalities. Although only a small fraction of the dataset is human labeled, the TVL model demonstrates improved visual-tactile understanding over GPT-4V (+12%) and open-source vision-language models (+32%) on a new touch-vision understanding benchmark. Code, checkpoints and data are available on https: //tactile-vlm.github.io. Letian Fu, Gaurav Datta, Will Panitch, Jaimyn Drake, Joseph Ortiz, Mustafa Mukadam, Mike Lambeta, Roberto Calandra, Kenneth Y. Goldberg |
ICML | 10 |
| 2024 | FogROS2-Config: A Toolkit for Choosing Server Configurations for Cloud RoboticsabstractCloud service providers provide over 50,000 distinct and dynamically changing set of cloud server options. To help roboticists make cost-effective decisions, we present FogROS2-Config, an open toolkit that takes ROS2 nodes as input and automatically runs relevant benchmarks to quickly return a menu of cloud compute services that tradeoff latency and cost. Because it is infeasible to try every hardware configuration, FogROS2-Config quickly samples tests a small set of edge-case servers. We evaluate FogROS2-Config on three robotics application tasks: visual SLAM, grasp planning. and motion planning. FogROS2-Config can reduce the cost by up to 20x. By comparing with a Pareto frontier for cost and latency by running the application task on feasible server configurations, we evaluate cost and latency models and confirm that FogROS2-Config selects efficient hardware configurations to balance cost and latency. Videos and code are available on the website https://sites.google.com/view/fogros2-config Kaiyuan Chen 0001, Kush Hari, Rohil Khare, Charlotte Le, Trinity Chung, Jaimyn Drake, Jeffrey Ichnowski, John Kubiatowicz, Kenneth Y. Goldberg |
ICRA | 9 |
| 2024 | FogROS2-LS: A Location-Independent Fog Robotics Framework for Latency Sensitive ROS2 ApplicationsabstractIn Cloud Robotics, long system latency due to varying network conditions can cause instability and collisions. However, this can be minimized in the almost univeral case where there are multiple sources available for cloud servers. By extending anycast routing, we introduce FogROS2-Latency-Sensitive, a Fog Robotics framework that offers secure, location-independent connections between robots and latency-sensitive cloud-based servers. FogROS2-LS offloads conventional on-board state estimators and feedback controllers to Cloud and Edge compute hardware without modifying existing applications in ROS2. In the presence of multiple identical services, FogROS2-LS dynamically identifies and transitions to the optimal service deployment that meets latency requirements, thereby empowering robots with limited on-board computing capacity to safely and efficiently navigate dynamic, human-dense environments. We evaluate FogROS2-LS with two latency sensitive case studies: (1) Collision Avoidance: a robot arm guided by visual feedback from consistent distance estimation and collision checking on Cloud and Edge. FogROS2-LS reduces collision failures by up to 8.5x by selecting the best available server, and (2) Target Tracking: FogROS2-LS enables robust and continuous target following and can recover from network failures. Videos and code are available on the website https://sites.google.com/view/fogros2-ls. Kaiyuan Chen 0001, Marcus Gualtieri, Nan Tian, Christian Juette, Liu Ren 0001, Jeffrey Ichnowski, John Kubiatowicz, Kenneth Y. Goldberg |
ICRA | 9 |
| 2024 | Manipulator as a Tail: Promoting Dynamic Stability for Legged LocomotionabstractFor locomotion, is an arm on a legged robot a liability or an asset for locomotion? Biological systems evolved additional limbs beyond legs that facilitates postural control. This work shows how a manipulator can be an asset for legged locomotion at high speeds or under external perturbations, where the arm serves beyond manipulation. Since the system has 15 degrees of freedom (twelve for the legged robot and three for the arm), off-the-shelf reinforcement learning (RL) algorithms struggle to learn effective locomotion policies. Inspired by Bernstein’s neurophysiological theory of animal motor learning, we develop an incremental training procedure that initially freezes some degrees of freedom and gradually releases them, using behaviour cloning (BC) from an early learning procedure to guide optimization in later learning. Simulation experiments show that our policy increases the success rate by up to 61 percentage points over the baselines. Simulation and real robot experiments suggest that our policy learns to use the arm as a "tail" to initiate robot turning at high speeds and to stabilize the quadruped under external perturbations. Quantitatively, in simulation experiments, we cut the failure rate up to 43.6% during high-speed turning and up to 31.8% for quadruped under external forces compared to using a locked arm. Antonio Loquercio, Ashish Kumar 0007, Neerja Thakkar, Kenneth Y. Goldberg, Jitendra Malik |
ICRA | 5 |
| 2024 | Conformal Policy Learning for Sensorimotor Control under Distribution ShiftsabstractThis paper focuses on the problem of detecting and reacting to changes in the distribution of a sensorimotor controller’s observables. The key idea is the design of policies that can take conformal quantiles as input, to detect distribution shifts with formal statistical guarantees, which we define as conformal policy learning. We show how to design such policies by using conformal quantiles to switch between base policies with different characteristics, e.g. safety or speed, or directly augmenting a policy observation with a quantile and training it with reinforcement learning. Theoretically, we show that such policies achieve the formal convergence guarantees in finite time. In addition, we thoroughly evaluate their advantages and limitations on two use cases: simulated autonomous driving and active perception with a physical quadruped. Empirical results demonstrate that our approach outperforms five baselines. It is also the simplest of the baseline strategies besides one ablation. Being easy to use, flexible, and with formal guarantees, our work demonstrates how conformal prediction can be an effective tool for sensorimotor learning under uncertainty. Satvik Sharma, Antonio Loquercio, Anastasios Angelopoulos, Kenneth Y. Goldberg, Jitendra Malik |
ICRA | 5 |
| 2024 | Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment CollaborationabstractLarge, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train "generalist" X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. The project website is robotics-transformer-x.github.io. Abigail O'Neill, Abhiram Maddukuri, Abhishek Gupta 0004, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, Albert Tung, Alex Bewley, Alex Irpan, Alexander Khazatsky, Anant Rai, Anchit Gupta, Andrew E. Wang, Anikait Singh, Animesh Garg, Aniruddha Kembhavi, Annie Xie, Anthony Brohan, Antonin Raffin, Archit Sharma, Arefeh Yavary, Arhan Jain, Ashwin Balakrishna, Ayzaan Wahid, Ben Burgess-Limerick, Bernhard Schölkopf, Blake Wulfe, Brian Ichter, Cewu Lu, Charles Xu 0003, Charlotte Le, Chelsea Finn, Chen Wang 0053, Chenfeng Xu, Cheng Chi 0001, Chenguang Huang, Christine Chan, Christopher Agia, Chuer Pan, Chuyuan Fu, Coline Devin, Danfei Xu, Daniel Morton, Danny Drieß, Daphne Chen, Deepak Pathak, Dhruv Shah, Dieter Büchler, Dinesh Jayaraman, Dmitry Kalashnikov, Dorsa Sadigh, Edward Johns, Ethan Paul Foster, Fangchen Liu, Federico Ceola, Fei Xia 0002, Feiyu Zhao, Freek Stulp, Gaoyue Zhou, Gaurav S. Sukhatme, Gautam Salhotra, Gilbert Feng, Giulio Schiavi, Glen Berseth, Gregory Kahn, Guanzhi Wang, Hao Su 0001, Haoshu Fang, Henghui Bao, Heni Ben Amor, Henrik I. Christensen, Hiroki Furuta, Homer Walke, Hongjie Fang, Huy Ha, Igor Mordatch, Ilija Radosavovic, Isabel Leal, Jacky Liang, Jad Abou-Chakra, Jaehyung Kim 0001, Jaimyn Drake, Jan Peters 0001, Jan Schneider 0007, Jasmine Hsu, Jeannette Bohg, Jeffrey T. Bingham, Jensen Gao, Jiaheng Hu, Jiajun Wu 0001, Jiankai Sun, Jianlan Luo, Jiayuan Gu, Jie Tan 0001, Jihoon Oh, Jimmy Wu, Jingpei Lu, Jitendra Malik, João Silvério, Joey Hejna, Jonathan Booher, Jonathan Tompson, Jonathan Yang, Jordi Salvador, Joseph J. Lim, Junhyek Han, Kanishka Rao, Karl Pertsch, Karol Hausman, Keegan Go, Keerthana Gopalakrishnan, Kenneth Y. Goldberg, Kendra Byrne, Kenneth Oslund, Kento Kawaharazuka, Kevin Black, Kevin Zhang 0002, Kiana Ehsani, Kiran Lekkala, Kirsty Ellis, Krishan Rana, Krishnan Srinivasan, Kuan Fang, Kunal Pratap Singh, Kuo-Hao Zeng, Kyle Hatch, Kyle Hsu, Laurent Itti, Yunliang Chen 0001, Lerrel Pinto, Li Fei-Fei 0001, Liam Tan, Linxi Fan, Lionel Ott, Lisa Lee, Luca Weihs, Magnum Chen, Marion Lepert, Marius Memmel, Masayoshi Tomizuka, Masha Itkina, Mateo Guaman Castro, Max Spero, Maximilian Du, Michael Ahn, Michael C. Yip, Mingtong Zhang 0003, Mingyu Ding, Minho Heo, Mohan Kumar Srirama, Mohit Sharma 0001, Moo Jin Kim, Naoaki Kanazawa, Nicklas Hansen 0001, Nicolas Heess, Nikhil J. Joshi, Niko Sünderhauf, Norman Di Palo, Nur Muhammad Shafiullah, Oier Mees, Oliver Kroemer, Osbert Bastani, Pannag R. Sanketi, Patrick Tree Miller, Patrick Yin, Paul Wohlhart, Peng Xu 0010, Peter David Fagan, Peter Mitrano, Pierre Sermanet, Pieter Abbeel, Priya Sundaresan, Qiuyu Chen, Rafael Rafailov, Ria Doshi, Roberto Martin Martin, Rohan Baijal, Rosario Scalise, Rose Hendrix, Roy Lin, Runjia Qian, Russell Mendonca, Rutav Shah, Ryan Hoque, Ryan Julian, Samuel Bustamante-Gomez, Sean Kirmani, Sergey Levine, Sherry Moore, Shikhar Bahl, Shivin Dass, Shubham D. Sonawani, Shuran Song, Sichun Xu, Siddhant Haldar, Siddharth Karamcheti, Simeon Adebola, Simon Guist, Soroush Nasiriany, Stefan Schaal, Stefan Welker, Stephen Tian, Subramanian Ramamoorthy, Sudeep Dasari, Suneel Belkhale, Sungjae Park, Suraj Nair 0003, Suvir Mirchandani, Takayuki Osa, Tanmay Gupta, Tatsuya Harada, Tatsuya Matsushima, Ted Xiao, Thomas Kollar, Tianhe Yu, Tianli Ding, Todor Davchev, Tony Z. Zhao, Travis Armstrong, Trevor Darrell, Trinity Chung, Vidhi Jain, Vincent Vanhoucke, Wolfram Burgard, Xiaolong Wang 0004, Xinghao Zhu, Xinyang Geng, Liangwei Xu, Yecheng Jason Ma 0001, Yejin Kim 0003, Yevgen Chebotar, Yilin Wu 0003, Yonatan Bisk, Yoonyoung Cho, Youngwoon Lee, Yuchen Cui, Yueh-Hua Wu, Yujin Tang, Yuke Zhu, Yunchu Zhang, Yunfan Jiang 0001, Yunshuang Li, Yunzhu Li, Yusuke Iwasawa, Yutaka Matsuo, Zehan Ma, Zichen Jeff Cui, Zichen Zhang 0016, Zipeng Lin |
ICRA | 124 |
| 2024 | Lifelong LERF: Local 3D Semantic Inventory Monitoring Using FogROS2abstractInventory monitoring in homes, factories, and retail stores relies on maintaining data despite objects being swapped, added, removed, or moved. We introduce Lifelong LERF, a method that allows a mobile robot with minimal compute to jointly optimize a dense language and geometric representation of its surroundings. Lifelong LERF maintains this representation over time by detecting semantic changes and selectively updating these regions of the environment, avoiding the need to exhaustively remap. Human users can query inventory by providing natural language queries and receiving a 3D heatmap of potential object locations. To manage the computational load, we use Fog-ROS2, a cloud robotics platform, to offload resource-intensive tasks. Lifelong LERF obtains poses from a monocular RGBD SLAM backend, and uses these poses to progressively optimize a Language Embedded Radiance Field (LERF) for semantic monitoring. Experiments with 3-5 objects arranged on a tabletop and a Turtlebot with a RealSense camera suggest that Lifelong LERF can persistently adapt to changes in objects with up to 91% accuracy. Adam Rashid, Chung Min Kim, Justin Kerr, Letian Fu, Kush Hari, Ayah Ahmad, Kaiyuan Chen 0001, Marcus Gualtieri, Christian Juette, Nan Tian, Liu Ren 0001, Kenneth Y. Goldberg |
ICRA | 14 |
| 2024 | Orbit-Surgical: An Open-Simulation Framework for Learning Surgical Augmented DexterityabstractPhysics-based simulations have accelerated progress in robot learning for driving, manipulation, and locomotion. Yet, a fast, accurate, and robust surgical simulation environment remains a challenge. In this paper, we present Orbit-Surgical, a physics-based surgical robot simulation framework with photorealistic rendering in NVIDIA Omniverse. We provide 14 benchmark surgical tasks for the da Vinci Research Kit (dVRK) and Smart Tissue Autonomous Robot (STAR) which represent common subtasks in surgical training. Orbit-Surgical leverages GPU parallelization to train reinforcement learning and imitation learning algorithms to facilitate study of robot learning to augment human surgical skills. Orbit-Surgical also facilitates realistic synthetic data generation for active perception tasks. We demonstrate Orbit-Surgical sim-to-real transfer of learned policies onto a physical dVRK robot.Project website: orbit-surgical.github.io Qinxi Yu, Masoud Moghani, Karthik Dharmarajan, Vincent Schorp, Will Panitch, Jingzhou Liu, Kush Hari, Mayank Mittal, Kenneth Y. Goldberg, Animesh Garg |
ICRA | 10 |
| 2024 | FogROS2-FT: Fault Tolerant Cloud RoboticsabstractCloud robotics enables robots to offload complex computational tasks to cloud servers for performance and ease of management. However, cloud compute can be costly, cloud services can suffer occasional downtime, and connectivity between the robot and cloud can be prone to variations in network Quality-of-Service (QoS). We present FogROS2-FT (Fault Tolerant) to mitigate these issues by introducing a multi-cloud extension that automatically replicates independent stateless robotic services, routes requests to these replicas, and directs the first response back. With replication, robots can still benefit from cloud computations even when a cloud service provider is down or there is low QoS. Additionally, many cloud computing providers offer low-cost "spot" computing instances that may shutdown unpredictably. Normally, these low-cost instances would be inappropriate for cloud robotics, but the fault tolerance nature of FogROS2-FT allows them to be used reliably. We demonstrate FogROS2-FT fault tolerance capabilities in 3 cloud-robotics scenarios in simulation (visual object detection, semantic segmentation, motion planning) and 1 physical robot experiment (scan-pick-and-place). Running on the same hardware specification, FogROS2-FT achieves motion planning with up to 2.2x cost reduction and up to a 5.53x reduction on 99 Percentile (P99) long-tail latency. FogROS2-FT reduces the P99 long-tail latency of object detection and semantic segmentation by 2.0x and 2.1x, respectively, under network slowdown and resource contention. Videos and code are available at https://sites.google.com/view/fogros2-ft. Kaiyuan Chen 0001, Kush Hari, Trinity Chung, Nan Tian, Christian Juette, Jeffrey Ichnowski, Liu Ren 0001, John Kubiatowicz, Ion Stoica, Kenneth Y. Goldberg |
IROS | 11 |
| 2024 | IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation LearningabstractImitation learning is a promising paradigm for training robot control policies, but these policies can suffer from distribution shift, where the conditions at evaluation time differ from those in the training data. A popular approach for increasing policy robustness to distribution shift is interactive imitation learning (i.e., DAgger and variants), where a human operator provides corrective interventions during policy rollouts. However, collecting a sufficient amount of interventions to cover the distribution of policy mistakes can be burdensome for human operators. We propose IntervenGen (I-Gen), a novel data augmentation system for robot control that autonomously produces a large set of corrective interventions with rich coverage of the state space from a small number of human interventions. We apply I-Gen to 4 simulated environments and 1 physical environment with object pose estimation error and show that it can increase policy robustness by up to 39× with only 10 human interventions. Videos and more results are available at https://sites.google.com/view/intervengen2024. Ryan Hoque, Ajay Mandlekar, Caelan Reed Garrett, Kenneth Y. Goldberg, Dieter Fox |
IROS | 4 |
| 2024 | SuFIA: Language-Guided Augmented Dexterity for Robotic Surgical AssistantsabstractIn this work, we present SuFIA, the first framework for natural language-guided augmented dexterity for robotic surgical assistants. SuFIA incorporates the strong reasoning capabilities of large language models (LLMs) with perception modules to implement high-level planning and low-level control of a robot for surgical sub-task execution. This enables a learning-free approach to surgical augmented dexterity without any in-context examples or motion primitives. SuFIA uses a human-in-the-loop paradigm by restoring control to the surgeon in the case of insufficient information, mitigating unexpected errors for mission-critical tasks. We evaluate SuFIA on four surgical sub-tasks in a simulation environment and two sub-tasks on a physical surgical robotic platform in the lab, demonstrating its ability to perform common surgical sub-tasks through supervised autonomous operation under challenging physical and workspace conditions.Project website: orbit-surgical.github.io/sufia Masoud Moghani, Lars Doorenbos, Will Panitch, Sean Huver, Mahdi Azizian, Kenneth Y. Goldberg, Animesh Garg |
IROS | 6 |
| 2024 | BOMP: Bin-Optimized Motion PlanningabstractIn logistics, the ability to quickly compute and execute pick-and-place motions from bins is critical to increasing productivity. We present Bin-Optimized Motion Planning (BOMP), a motion planning framework that plans arm motions for a six-axis industrial robot with a long-nosed suction tool to remove boxes from deep bins. BOMP considers robot arm kinematics, actuation limits, the dimensions of a grasped box, and a varying height map of a bin environment to rapidly generate time-optimized, jerk-limited, and collision-free trajectories. The optimization is warm-started using a deep neural network trained offline in simulation with 25,000 scenes and corresponding trajectories. Experiments with 96 simulated and 15 physical environments suggest that BOMP generates collision-free trajectories that are up to 58% faster than baseline sampling-based planners and up to 36% faster than an industry-standard Up-Over-Down algorithm, which has an extremely low 15% success rate in this context. BOMP also generates jerk-limited trajectories while baselines do not. Website: https://sites.google.com/berkeley.edu/bomp. Zachary Tam, Karthik Dharmarajan, Tianshuang Qiu, Yahav Avigal, Jeffrey Ichnowski, Kenneth Y. Goldberg |
IROS | 6 |
| 2024 | Language-Embedded Gaussian Splats (LEGS): Incrementally Building Room-Scale Representations with a Mobile RobotabstractBuilding semantic 3D maps is valuable for searching for objects of interest in offices, warehouses, stores, and homes. We present a mapping system that incrementally builds a Language-Embedded Gaussian Splat (LEGS): a detailed 3D scene representation that encodes both appearance and semantics in a unified representation. LEGS is trained online as a robot traverses its environment to enable localization of open-vocabulary object queries. We evaluate LEGS on 4 room-scale scenes where we query for objects in the scene to assess how LEGS can capture semantic meaning. We compare LEGS to LERF [1] and find that while both systems have comparable object query success rates, LEGS trains over 3.5x faster than LERF. Results suggest that a multi-camera setup and incremental bundle adjustment can boost visual reconstruction quality in constrained robot trajectories, and suggest LEGS can localize open-vocabulary and long-tail object queries with up to 66% accuracy. See project website at: berkeleyautomation.github.io/LEGS Justin Yu, Kush Hari, Kishore Srinivas, Karim El-Refai, Adam Rashid, Chung Min Kim, Justin Kerr, Richard Cheng, Muhammad Zubair Irshad, Ashwin Balakrishna, Thomas Kollar, Kenneth Y. Goldberg |
IROS | 12 |
| 2024 | MANIP: A Modular Architecture for Integrating Interactive Perception for Robot ManipulationabstractWe propose a modular systems architecture, MANIP, that can facilitate the design and development of robot manipulation systems by systematically combining learned subpolicies with well-established procedural algorithmic primitives such as Inverse Kinematics, Kalman Filters, RANSAC outlier rejection, PID modules, etc. (aka "Good Old Fashioned Engineering (GOFE)"). The MANIP architecture grew from our lab’s experience developing robot systems for folding clothes, routing cables, and untangling knots. To address failure modes, MANIP can facilitate inclusion of "interactive perception" subpolicies that execute robot actions to modify system state to bring the system into alignment with the training distribution and / or to disambiguate system state when system state confidence is low. We demonstrate how MANIP can be applied with 3 case studies and then describe a detailed case study in cable tracing with experiments that suggest MANIP can improve performance by up to 88%. Code and details are available at: https://berkeleyautomation.github.io/MANIP/ Justin Yu, Tara Sadjadpour, Abigail O'Neill, Mehdi Khfifi, Yunliang Chen 0001, Richard Cheng, Muhammad Zubair Irshad, Ashwin Balakrishna, Thomas Kollar, Kenneth Y. Goldberg |
IROS | 10 |
| 2024 | Automated Pruning and Irrigation of Polyculture PlantsabstractPolyculture farming has environmental advantages but requires substantially more labor than monoculture farming. We present novel hardware and algorithms for automated pruning and irrigation. Using an overhead camera to collect data from physical$1.5~m^{2}$garden testbeds, the autonomous system utilizes a learned Plant Phenotyping convolutional neural network and a Bounding Disk Tracking algorithm to evaluate the individual plant distribution and estimate the state of the garden each day. From this garden state, AlphaGardenSim selects plants to autonomously prune. A trained neural network detects and targets specific prune points on the plant. Two custom-designed pruning tools, compatible with a FarmBot commercial gantry system, are experimentally evaluated. Irrigation is automated using soil moisture sensors. We present results for four 60-day garden cycles. Results suggest the system can autonomously achieve 94% normalized plant diversity with pruning shears while maintaining an average canopy coverage of 84% by the end of the cycles. For code, videos, and datasets, see https://sites.google.com/berkeley.edu/pruningpolyculturej/home.Note to Practitioners—While polyculture farming is closer to how plants grow in nature, it is considered more labor intensive that monoculture farming. In this paper we present approaches and custom hardware for automation of pruning and irrigation. Physical experiments suggest that automation can yield both high coverage and diversity. Simeon Adebola, Mark Presten, Rishi Parikh, Shrey Aeron, Sandeep Mukherjee, Satvik Sharma, Mark Theis, Walter Teitelbaum, Eugen Solowjow, Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 10 |
| 2024 | Optimal Arrangement and Rearrangement of Objects on Shelves to Minimize Robot Retrieval CostabstractFrom residential homes to commercial outlets and industrial warehouses, shelves are widely used to store a variety of objects. We conjecture that optimizing object arrangements can significantly improve the efficiency of searching and retrieving target objects for robots and automation systems. We formalize the Optimal Shelf Arrangement (OSA) problem, which, given a set of objects’ access frequencies and movement costs, optimizes their placement on a shelf to minimize access time. We present, a mixed-integer program (MIP), show that it finds an optimal solution for OSA under certain conditions, and provide bounds on its suboptimality in general cost settings. We further present -Stack, which extends the setting to environments that allow object stacking. Additionally, we consider improving a given suboptimal object arrangement through efficient rearrangement. Experiment results from over 2,700 simulated shelf trials and 54 physical trials using a Fetch robot equipped with a suction grasping tool and pushing blade suggest that an optimal arrangement can reduce the expected retrieval cost by 60–80% in fully-observed configurations and reduce the expected search cost by 50–70% while increasing the search success rate by up to 2x in partially-observed configurations.Note to Practitioners—While there is a large literature on task and motion planning that studies how to rearrange objects from one arrangement to another, there is little research on what an ideal “target” arrangement should be. This article demonstrates that an optimal arrangement of objects makes searching and retrieving objects on shelves significantly easier. We present the Optimal Shelf Arrangement (OSA) problem and algorithms to solve it. Although we assume access to a robot that can push and use a vacuum suction cup gripper, practitioners can adapt the OSA model to their own hardware and shelf constraints and objectives. Also, the algorithms for improving existing shelf arrangements can be applied to scenarios where the objects are already on the shelves rather than to be arranged from scratch. Yunliang Chen 0001, Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 3 |
| 2023 | LERF: Language Embedded Radiance FieldsabstractHumans describe the physical world using natural language to refer to specific 3D locations based on a vast range of properties: visual appearance, semantics, abstract associations, or actionable affordances. In this work we propose Language Embedded Radiance Fields (LERFs), a method for grounding language embeddings from off-the-shelf models like CLIP into NeRF, which enable these types of open-ended language queries in 3D. LERF learns a dense, multi-scale language field inside NeRF by volume rendering CLIP embeddings along training rays, supervising these embeddings across training views to provide multi-view consistency and smooth the underlying language field. After optimization, LERF can extract 3D relevancy maps for a broad range of language prompts interactively in real-time, which has potential use cases in robotics, understanding vision-language models, and interacting with 3D scenes. LERF enables pixel-aligned, zero-shot queries on the distilled 3D CLIP embeddings without relying on region proposals or masks, supporting long-tail open-vocabulary queries hierarchically across the volume. See the project website at: https://lerf.io. Justin Kerr, Chung Min Kim, Kenneth Y. Goldberg, Angjoo Kanazawa, Matthew Tancik |
ICCV | 3 |
| 2023 | Can Machines Garden? Systematically Comparing the AlphaGarden vs. Professional HorticulturalistsabstractThe AlphaGarden is an automated testbed for indoor polyculture farming which combines a first-order plant simulator, a gantry robot, a seed planting algorithm, plant phenotyping and tracking algorithms, irrigation sensors and algorithms, and custom pruning tools and algorithms. In this paper, we systematically compare the performance of the AlphaGarden to professional horticulturalists on the staff of the UC Berkeley Oxford Tract Greenhouse. The humans and the machine tend side-by-side polyculture gardens with the same seed arrangement. We compare performance in terms of canopy coverage, plant diversity, and water consumption. Results from two 60-day cycles suggest that the automated AlphaGarden performs comparably to professional horticulturalists in terms of coverage and diversity, and reduces water consumption by as much as 44%. Code, videos, and datasets are available at https//sites.google.com/berkeley.edulsystematiccomparison Simeon Adebola, Rishi Parikh, Mark Presten, Satvik Sharma, Shrey Aeron, Ananth Rao, Sandeep Mukherjee, Tomson Qu, Christina Wistrom, Eugen Solowjow, Kenneth Y. Goldberg |
ICRA | 11 |
| 2023 | AutoBag: Learning to Open Plastic Bags and Insert ObjectsabstractThin plastic bags are ubiquitous in retail stores, healthcare, food handling, recycling, homes, and school lunchrooms. They are challenging both for perception (due to specularities and occlusions) and for manipulation (due to the dynamics of their 3D deformable structure). We formulate the task of “bagging:” manipulating common plastic shopping bags with two handles from an unstructured initial state to an open state where at least one solid object can be inserted into the bag and lifted for transport. We propose a self-supervised learning framework where a dual-arm robot learns to recognize the handles and rim of plastic bags using UV-fluorescent markings; at execution time, the robot does not use UV markings or UV light. We propose the AutoBag algorithm, where the robot uses the learned perception model to open a plastic bag through iterative manipulation. We present novel metrics to evaluate the quality of a bag state and new motion primitives for reorienting and opening bags based on visual observations. In physical experiments, a YuMi robot using AutoBag is able to open bags and achieve a success rate of 16/30 for inserting at least one item across a variety of initial bag configurations. Supplementary material is available at https://sites.google.com/view/autobag. Yunliang Chen 0001, Baiyu Shi, Daniel Seita, Richard Cheng, Thomas Kollar, David Held, Kenneth Y. Goldberg |
ICRA | 7 |
| 2023 | Automating Vascular Shunt Insertion with the dVRK Surgical RobotabstractVascular shunt insertion is a fundamental surgical procedure used to temporarily restore blood flow to tissues. It is often performed in the field after major trauma. We formulate a problem of automated vascular shunt insertion and propose a pipeline to perform Automated Vascular Shunt Insertion (AVSI) using a da Vinci Research Kit. The pipeline uses a learned visual model to estimate the locus of the vessel rim, plans a grasp on the rim, and moves to grasp at that point. The first robot gripper then pulls the rim to stretch open the vessel with a dilation motion. The second robot gripper then proceeds to insert a shunt into the vessel phantom (a model of the blood vessel) with a chamfer tilt followed by a screw motion. Results suggest that AVSI achieves a high success rate even with tight tolerances and varying vessel orientations up to 30°. Supplementary material, dataset, videos, and visualizations can be found at https://sites.google.com/berkeley.edu/autolab-avsi. Karthik Dharmarajan, Will Panitch, Muyan Jiang, Kishore Srinivas, Baiyu Shi, Yahav Avigal, Thomas Low, Danyal Fer, Kenneth Y. Goldberg |
ICRA | 10 |
| 2023 | Safe Self-Supervised Learning in Real of Visuo-Tactile Feedback Policies for Industrial InsertionabstractIndustrial insertion tasks are often performed repetitively with parts that are subject to tight tolerances and prone to breakage. Learning an industrial insertion policy in real is challenging as the collision between the parts and the environment can cause slippage or breakage of the part. In this paper, we present a safe self-supervised method to learn a visuo-tactile insertion policy that is robust to grasp pose variations. The method reduces human input and collisions between the part and the receptacle. The method divides the insertion task into two phases. In the first align phase, a tactile-based grasp pose estimation model is learned to align the insertion part with the receptacle. In the second insert phase, a vision-based policy is learned to guide the part into the receptacle. The robot uses force-torque sensing to achieve a safe self-supervised data collection pipeline. Physical experiments on the USB insertion task from the NIST Assembly Taskboard suggest that the resulting policies can achieve 45/45 insertion successes on 45 different initial grasp poses, improving on two baselines: (1) a behavior cloning agent trained on 50 human insertion demonstrations (1/45) and (2) an online RL policy (TD3) trained in real (0/45). Letian Fu, Lars Berscheid, Kenneth Y. Goldberg, Sachin Chitta |
ICRA | 5 |
| 2023 | FogROS2: An Adaptive Platform for Cloud and Fog Robotics Using ROS 2abstractMobility, power, and price points often dictate that robots do not have sufficient computing power on board to run contemporary robot algorithms at desired rates. Cloud computing providers such as AWS, GCP, and Azure offer immense computing power and increasingly low latency on demand, but tapping into that power from a robot is non-trivial. We present FogROS2, an open-source platform to facilitate cloud and fog robotics that is included in the Robot Operating System 2 (ROS 2) distribution. FogROS2 is distinct from its predecessor FogROS1 in 9 ways, including lower latency, overhead, and startup times; improved usability, and additional automation, such as region and computer type selection. Additionally, FogROS2 gains performance, timing, and additional improvements associated with ROS 2. In common robot applications, FogROS2 reduces SLAM latency by 50 %, reduces grasp planning time from 14 s to 1.2 s, and speeds up motion planning 45x. When compared to FogROS1, FogROS2 reduces network utilization by up to 3.8x, improves startup time by 63 %, and network round-trip latency by 97 % for images using video compression. The source code, examples, and documentation for FogROS2 are available at https://github.com/BerkeleyAutomation/FogROS2, and is available through the official ROS 2 repository at https://index.ros.org/p/FogROS2/. Jeffrey Ichnowski, Kaiyuan Chen 0001, Karthik Dharmarajan, Simeon Adebola, Michael Danielczuk, Victor Mayoral Vilches, Nikhil Jha, Hugo Zhan, Edith LLontop, Derek Xu, Camilo Buscaron, John Kubiatowicz, Ion Stoica, Joseph Gonzalez 0001, Kenneth Y. Goldberg |
ICRA | 15 |
| 2023 | SGTM 2.0: Autonomously Untangling Long Cables using Interactive PerceptionabstractCables are commonplace in homes, hospitals, and industrial warehouses and are prone to tangling. This paper extends prior work on autonomously untangling long cables by introducing novel uncertainty quantification metrics and actions that interact with the cable to reduce perception uncertainty. We present Sliding and Grasping for Tangle Manipulation 2.0 (SGTM 2.0), a system that autonomously untangles cables approximately 3 meters in length with a bilateral robot using estimates of uncertainty at each step to inform actions. By interactively reducing uncertainty, SGTM 2.0 significantly reduces run-time. Physical experiments with 84 trials suggest that SGTM$2.0$can achieve 83% untangling success on cables with 1 or 2 overhand and figure-8 knots, and 70% termination detection success across these configurations, outperforming SGTM 1.0 by 43% in untangling accuracy and 200% in completion time. Supplementary material, visualizations, and videos can be found at sites.google.com/view/sgtm2. Kaushik Shivakumar, Vainavi Viswanath, Anrui Gu, Yahav Avigal, Justin Kerr, Jeffrey Ichnowski, Richard Cheng, Thomas Kollar, Kenneth Y. Goldberg |
ICRA | 9 |
| 2023 | Learning to Efficiently Plan Robust Frictional Multi-Object GraspsabstractWe consider a decluttering problem where multiple rigid convex polygonal objects rest in randomly placed positions and orientations on a planar surface and must be efficiently transported to a packing box using both single and multi-object grasps. Prior work considered frictionless multi-object grasping. In this paper, we introduce friction to increase the number of potential grasps for a given group of objects, and thus increase picks per hour. We train a neural network using real examples to plan robust multi-object grasps. In physical experiments, we find a 13.7% increase in success rate, a 1.6x increase in picks per hour, and a 6.3x decrease in grasp planning time compared to prior work on multi-object grasping. Compared to single-object grasping, we find a 3.1x increase in picks per hour. Wisdom C. Agboh, Satvik Sharma, Kishore Srinivas, Mallika Parulekar, Gaurav Datta, Tianshuang Qiu, Jeffrey Ichnowski, Eugen Solowjow, Mehmet Remzi Dogar, Kenneth Y. Goldberg |
IROS | 10 |
| 2023 | FogROS2-SGC: A ROS2 Cloud Robotics Platform for Secure Global ConnectivityabstractThe Robot Operating System (ROS2) is the most widely used software platform for building robotics applications. FogROS2 extends ROS2 to allow robots to access cloud computing on demand. We introduce FogROS2-SGC, an extension of FogROS2 that can effectively connect robot systems across different physical locations, networks, and Data Distribution Services (DDS). With globally unique and location-independent identifiers, FogROS2-SGC can securely and efficiently route data between robotics components around the globe. FogROS2-SGC is agnostic to the ROS2 distribution and configuration, is compatible with non-ROS2 software, and seamlessly extends existing ROS2 applications without any code modification. We evaluate FogROS2-SGC with 4 robots and compute nodes that are 3600 km apart. Experiments suggest FogROS2-SGC is 19x faster than rosbridge (a ROS2 package with comparable features, but lacking security). Videos and code are available on the website https://sites.google.com/view/fogros2-sgc. Kaiyuan Chen 0001, Ryan Hoque, Karthik Dharmarajan, Edith LLontop, Simeon Adebola, Jeffrey Ichnowski, John Kubiatowicz, Kenneth Y. Goldberg |
IROS | 8 |
| 2023 | Bagging by Learning to Singulate Layers Using Interactive PerceptionabstractMany fabric handling and 2D deformable material tasks in homes and industries require singulating layers of material such as opening a bag or arranging garments for sewing. In contrast to methods requiring specialized sensing or end effectors, we use only visual observations with ordinary parallel jaw grippers. We propose SLIP: Singulating Layers using Interactive Perception, and apply SLIP to the task of autonomous bagging. We develop SLIP-Bagging, a bagging algorithm that manipulates a plastic or fabric bag from an unstructured state and uses SLIP to grasp the top layer of the bag to open it for object insertion. In physical experiments, a YuMi robot achieves a success rate of 67% to 81% across bags of a variety of materials, shapes, and sizes, significantly improving in success rate and generality over prior work. Experiments also suggest that SLIP can be applied to tasks such as singulating layers of folded cloth and garments. Supplementary material is available at https://sites.google.com/view/slip-bagging/. Yunliang Chen 0001, Baiyu Shi, Roy Lin, Daniel Seita, Ayah Ahmad, Richard Cheng, Thomas Kollar, David Held, Kenneth Y. Goldberg |
IROS | 9 |
| 2023 | Video Prediction Models as Rewards for Reinforcement LearningabstractSpecifying reward signals that allow agents to learn complex behaviors is a long-standing challenge in reinforcement learning.
A promising approach is to extract preferences for behaviors from unlabeled videos, which are widely available on the internet. We present Video Prediction Rewards (VIPER), an algorithm that leverages pretrained video prediction models as action-free reward signals for reinforcement learning. Specifically, we first train an autoregressive transformer on expert videos and then use the video prediction likelihoods as reward signals for a reinforcement learning agent. VIPER enables expert-level control without programmatic task rewards across a wide range of DMC, Atari, and RLBench tasks. Moreover, generalization of the video prediction model allows us to derive rewards for an out-of-distribution environment where no expert data is available, enabling cross-embodiment generalization for tabletop manipulation. We see our work as starting point for scalable reward specification from unlabeled videos that will benefit from the rapid advances in generative modeling. Source code and datasets are available on the project website: https://ViperRL.com Alejandro Escontrela, Ademi Adeniji, Wilson Yan, Ajay Jain, Xue Bin Peng, Kenneth Y. Goldberg, Youngwoon Lee, Danijar Hafner, Pieter Abbeel |
NeurIPS | 6 |
| 2023 | Automating Surgical Peg Transfer: Calibration With Deep Learning Can Exceed Speed, Accuracy, and Consistency of HumansabstractPeg transfer is a well-known surgical training task in the Fundamentals of Laparoscopic Surgery (FLS). While human surgeons teleoperate robots such as the da Vinci to perform this task with high speed and accuracy, it is challenging to automate. This paper presents a novel system and control method using a da Vinci Research Kit (dVRK) surgical robot and a Zivid depth sensor, and a human subjects study comparing performance on three variants of the peg-transfer task: unilateral, bilateral without handovers, and bilateral with handovers. The system combines 3D printing, depth sensing, and deep learning for calibration with a new analytic inverse kinematics model and time-minimized motion controller. In a controlled study of 3384 peg transfer trials performed by the system, an expert surgical resident, and 9 volunteers, results suggest that the system achieves accuracy on par with the experienced surgical resident and is significantly faster and more consistent than the surgical resident and volunteers. The system also exhibits the highest consistency and lowest collision rate. To our knowledge, this is the first autonomous system to achieve “superhuman” performance on a standardized surgical task. All data is available athttps://sites.google.com/view/surgicalpegtransferNote to Practitioners—This paper presents a new approach to calibrating cable-driven robots based on a combination of 3D printing, depth sensing, inverse kinematics, convex optimization, and deep learning. The approach is applied to calibrating the da Vinci, commercial surgical-assist robot, to automate a standard “pick and place” task. Experiments suggest that the resulting system matches human surgical expert performance in speed and accuracy and significantly outperforms humans in terms of consistency. All details on the system including CAD models, code, and user study data are available online. Minho Hwang, Jeffrey Ichnowski, Brijen Thananjeyan, Daniel Seita, Samuel Paradis, Danyal Fer, Thomas Low, Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 8 |
| 2023 | In MemoriamabstractRecounts the career and contributions of Peter Luh. Frank C. Park 0001, Nukula Viswanadham, Kenneth Y. Goldberg, Michael Yu Wang, Yu Sun 0001, MengChu Zhou, Bengt Lennartson, Fan-Tien Cheng |
IEEE Trans Autom. Sci. Eng. | 3 |
| 2022 | LEGS: Learning Efficient Grasp Sets for Exploratory GraspingabstractWhile deep learning has enabled significant progress in designing general purpose robot grasping systems, there remain objects which still pose challenges for these systems. Recent work on Exploratory Grasping has formalized the problem of systematically exploring grasps on these adversarial objects and explored a multi-armed bandit model for identifying high-quality grasps on each object stable pose. However, these systems are still limited to exploring a small number or grasps on each object. We present Learned Efficient Grasp Sets (LEGS), an algorithm that efficiently explores thousands of possible grasps by maintaining small active sets of promising grasps and determining when it can stop exploring the object with high confidence. Experiments suggest that LEGS can identify a high-quality grasp more efficiently than prior algorithms which do not use active sets. In simulation experiments, we measure the gap between the success probability of the best grasp identified by LEGS, baselines, and the most-robust grasp (verified ground truth). After 3000 exploration steps, LEGS outperforms baseline algorithms on 10/14 and 25/39 objects on the Dex-Net Adversarial and EGAD! datasets respectively. We then evaluate LEGS in physical experiments; trials on 3 challenging objects suggest that LEGS converges to high-performing grasps significantly faster than baselines. See https://sites.google.com/view/LEGS-exp-grasping for supplemental material and videos. Letian Fu, Michael Danielczuk, Ashwin Balakrishna, Daniel S. Brown, Jeffrey Ichnowski, Eugen Solowjow, Kenneth Y. Goldberg |
ICRA | 7 |
| 2022 | Implicit Kinematic Policies: Unifying Joint and Cartesian Action Spaces in End-to-End Robot LearningabstractAction representation is an important yet often overlooked aspect in end-to-end robot learning with deep networks. Choosing one action space over another (e.g. target joint positions, or Cartesian end-effector poses) can result in surprisingly stark performance differences between various downstream tasks - and as a result, considerable research has been devoted to finding the right action space for a given application. However, in this work, we instead investigate how our models can discover and learn for themselves which action space to use. Leveraging recent work on implicit behavioral cloning, which takes both observations and actions as input, we demonstrate that it is possible to present the same action in multiple different spaces to the same policy - allowing it to learn inductive patterns from each space. Specifically, we study the benefits of combining Cartesian and joint action spaces in the context of learning manipulation skills. To this end, we present Implicit Kinematic Policies (IKP), which incorporates the kinematic chain as a differentiable module within the deep network. Quantitative experiments across several simulated continuous control tasks-from scooping piles of small objects, to lifting boxes with elbows, to precise block insertion with miscalibrated robots-suggest IKP not only learns complex prehensile and non-prehensile manipulation from pixels better than baseline alternatives, but also can learn to compensate for small joint encoder offset errors. Finally, we also run qualitative experiments on a real UR5e to demonstrate the feasibility of our algorithm on a physical robotic system with real data. See https://tinyurl.com/4wz3nf86 for code and supplementary material. Aditya Ganapathi, Peter R. Florence, Jake Varley, Kaylee Burns, Kenneth Y. Goldberg, Andy Zeng 0001 |
ICRA | 5 |
| 2022 | Mechanical Search on Shelves using a Novel "Bluction" ToolabstractShelves are common in homes, warehouses, and commercial settings due to their storage efficiency. However, this efficiency comes at the cost of reduced visibility and accessibility. When looking from a side (lateral) view of a shelf, most objects will be fully occluded, resulting in a constrained lateral-access mechanical search problem. To address this problem, we introduce: (1) a novel bluction tool, which combines a thin pushing blade and a suction cup gripper, (2) a simulation pipeline and perception model that combine ray-casting with 2D Minkowski sums to efficiently generate target occupancy distributions, and (3) a novel search policy, which optimally reduces target object distribution support area using the bluction tool. Experimental data from 2000 simulated shelf trials and 18 trials with a physical Fetch robot suggest that a bluction tool can improve the average success rate by 26% in simulation and 67% in physical experiments over the highest-performing push-only policy. Michael Danielczuk, Chung Min Kim, Letian Fu, Zachary Tam, Jeffrey Ichnowski, Anelia Angelova, Brian Ichter, Kenneth Y. Goldberg |
ICRA | 9 |
| 2022 | GOMP-FIT: Grasp-Optimized Motion Planning for Fast Inertial TransportabstractHigh-speed motions in pick-and-place operations are critical to making robots cost-effective in many automation scenarios, from warehouses and manufacturing to hospitals and homes. However, motions can be too fast-such as when the object being transported has an open-top, is fragile, or both. One way to avoid spills or damage, is to move the arm slowly. We propose an alternative: Grasp-Optimized Motion Planning for Fast Inertial Transport (GOMP-FIT), a time-optimizing motion planner based on our prior work, that includes con-straints based on accelerations at the robot end-effector. With GOMP-FIT, a robot can perform high-speed motions that avoid obstacles and use inertial forces to its advantage. In experiments transporting open-top containers with varying tilt tolerances, whereas GOMP computes sub-second motions that spill up to 90 % of the contents during transport, GOMP-FIT generates motions that spill 0 % of contents while being slowed by as little as 0 % when there are few obstacles, 30 % when there are high obstacles and 45-degree tolerances, and 50 % when there 15-degree tolerances and few obstacles. Videos and more at: https://berkeleyautomation.github.io/gomp-fit/. Jeffrey Ichnowski, Yahav Avigal, Kenneth Y. Goldberg |
ICRA | 4 |
| 2022 | IPC-GraspSim: Reducing the Sim2Real Gap for Parallel-Jaw Grasping with the Incremental Potential Contact ModelabstractAccurately simulating whether an object will be lifted securely or dropped during grasping is a longstanding Sim2Real challenge. Soft compliant jaw tips are almost universally used with parallel-jaw robot grippers due to their ability to increase contact area and friction between the jaws and the object to be manipulated. However, interactions between the compliant surfaces and rigid objects are notoriously difficult to model. We introduce IPC-GraspSim, a novel grasp simulator that extends Incremental Potential Contact (IPC) - a highly accurate collision + deformation model developed in 2020 for computer graphics. IPC-GraspSim models both the dynamics and the deformation of compliant jaw tips to reduce Sim2Real gap for robot grasping. We evaluate IPC-GraspSim using a set of 2,000 physical grasps across 16 adversarial objects where analytic models perform poorly. In comparison to both analytic quasistatic contact models (soft point contact, REACH, 6DFC) and dynamic grasp simulators (Isaac Gym with FleX), results suggest IPC-GraspSim can predict robustness with higher precision and recall (F1 = 0.85). IPC-GraspSim increases F1 score by 0.03 to 0.20 over analytic baselines and 0.09 over Isaac Gym, at a cost of 8000x and 1.5x more compute time, respectively. All data, code, videos, and supplementary material are available at https://sites.google.com/berkeley.edu/ipcgraspsim. Chung Min Kim, Michael Danielczuk, Isabella Huang, Kenneth Y. Goldberg |
ICRA | 4 |
| 2022 | Real2Sim2Real: Self-Supervised Learning of Physical Single-Step Dynamic Actions for Planar Robot CastingabstractThis paper introduces the task of Planar Robot Casting (PRC): where one planar motion of a robot arm holding one end of a cable causes the other end to slide across the plane toward a desired target. PRC allows the cable to reach points beyond the robot workspace and has applications for cable management in homes, warehouses, and factories. To efficiently learn a PRC policy for a given cable, we propose Real2Sim2Real, a self-supervised framework that automatically collects physical trajectory examples to tune parameters of a dynamics simulator using Differential Evolution, generates many simulated examples, and then learns a policy using a weighted combination of simulated and physical data. We evaluate Real2Sim2Real with three simulators, Isaac Gym-segmented, Isaac Gym-hybrid, and PyBullet, two function approximators, Gaussian Processes and Neural Networks (NNs), and three cables with differing stiffness, torsion, and friction. Results with 240 physical trials suggest that the PRC policies can attain median error distance (as % of cable length) ranging from 8 % to 14 %, outperforming baselines and policies trained on only real or only simulated examples. Code, data, and videos are available at https://tinyurl.com/robotcast. Vincent Lim, Yunliang Chen 0001, Jeffrey Ichnowski, Daniel Seita, Michael Laskey, Kenneth Y. Goldberg |
ICRA | 8 |
| 2022 | Learning to Localize, Grasp, and Hand Over Unmodified Surgical NeedlesabstractRobotic Surgical Assistants (RSAs) are commonly used to perform minimally invasive surgeries by expert surgeons. However, long procedures filled with tedious and repetitive tasks such as suturing can lead to surgeon fatigue, motivating the automation of suturing. As visual tracking of a thin reflective needle is extremely challenging, prior work has modified the needle with nonreflective contrasting paint. As a step towards automation of a suturing subtask without modifying the needle, we propose HOUSTON: Handover of Unmodified, Surgical, Tool-Obstructed Needles, a problem and algorithm that uses a learned active sensing policy with a stereo camera to iteratively localize and align the needle into a visible and accessible pose for the other gripper. To compensate for robot positioning and needle perception errors, the algorithm then executes a high-precision grasping motion that uses multiple cameras. Physical experiments with the da Vinci Research Kit (dVRK) suggest a success rate of 96.7% on needles used in training, and 75 - 92.9% on needles unseen in training. On sequential handovers, HOUSTON successfully executes 32.4 handovers on average before failure. To our knowledge, this work is the first to study handover of unmodified surgical needles. See https: / /tinyurl. com/houston-surgery for additional materials including details about offline datasets and model architectures. Albert Wilcox, Justin Kerr, Brijen Thananjeyan, Jeffrey Ichnowski, Minho Hwang, Samuel Paradis, Danyal Fer, Kenneth Y. Goldberg |
ICRA | 8 |
| 2022 | SpeedFolding: Learning Efficient Bimanual Folding of GarmentsabstractFolding garments reliably and efficiently is a long standing challenge in robotic manipulation due to the complex dynamics and high dimensional configuration space of garments. An intuitive approach is to initially manipulate the garment to a canonical smooth configuration before folding. In this work, we develop SpeedFolding, a reliable and efficient bimanual system, which given user-defined instructions as folding lines, manipulates an initially crumpled garment to (1) a smoothed and (2) a folded configuration. Our primary contribution is a novel neural network architecture that is able to predict pairs of gripper poses to parameterize a diverse set of bimanual action primitives. After learning from 4300 human- annotated and self-supervised actions, the robot is able to fold garments from a random initial configuration in under 120 s on average with a success rate of 93 %. Real-world experiments show that the system is able to generalize to unseen garments of different color, shape, and stiffness. While prior work achieved 3–6 Folds Per Hour (FPH), SpeedFolding achieves 30–40 FPH. See https://pantor.github.io/speedfolding for code, videos, and datasets. Yahav Avigal, Lars Berscheid, Tamim Asfour, Torsten Kröger, Kenneth Y. Goldberg |
IROS | 5 |
| 2022 | Adversarial Motion Priors Make Good Substitutes for Complex Reward FunctionsabstractTraining a high-dimensional simulated agent with an under-specified reward function often leads the agent to learn physically infeasible strategies that are ineffective when deployed in the real world. To mitigate these unnatural behaviors, reinforcement learning practitioners often utilize complex reward functions that encourage physically plausible behaviors. However, a tedious labor-intensive tuning process is often required to create hand-designed rewards which might not easily generalize across platforms and tasks. We propose substituting complex reward functions with “style rewards” learned from a dataset of motion capture demonstrations. A learned style reward can be combined with an arbitrary task reward to train policies that perform tasks using naturalistic strategies. These natural strategies can also facilitate transfer to the real world. We build upon Adversarial Motion Priors - an approach from the computer graphics domain that encodes a style reward from a dataset of reference motions - to demonstrate that an adversarial approach to training policies can produce behaviors that transfer to a real quadrupedal robot without requiring complex reward functions. We also demonstrate that an effective style reward can be learned from a few seconds of motion capture data gathered from a German Shepherd and leads to energy-efficient locomotion strategies with natural gait transitions. Alejandro Escontrela, Xue Bin Peng, Wenhao Yu 0003, Tingnan Zhang, Atil Iscen, Kenneth Y. Goldberg, Pieter Abbeel |
IROS | 6 |
| 2022 | Learning to Fold Real Garments with One Arm: A Case Study in Cloud-Based Robotics ResearchabstractAutonomous fabric manipulation is a longstanding challenge in robotics, but evaluating progress is difficult due to the cost and diversity of robot hardware. Using Reach, a cloud robotics platform that enables low-latency remote execution of control policies on physical robots, we present the first systematic benchmarking of fabric manipulation al-gorithms on physical hardware. We develop 4 novel learning-based algorithms that model expert actions, keypoints, reward functions, and dynamic motions, and we compare these against 4 learning-free and inverse dynamics algorithms on the task of folding a crumpled T-shirt with a single robot arm. The entire lifecycle of data collection, model training, and policy evaluation was performed remotely without physical access to the robot workcell. Results suggest a new algorithm combining imitation learning with analytic methods achieves human-level performance on the flattening task and 93% of human-level performance on the folding task. See https://sites.google.com/berkeley.edu/ cloudfolding for all data, code, models, and supplemental material. Ryan Hoque, Kaushik Shivakumar, Shrey Aeron, Gabriel Deza, Aditya Ganapathi, Adrian Wong, Johnny Lee, Andy Zeng 0001, Vincent Vanhoucke, Kenneth Y. Goldberg |
IROS | 10 |
| 2022 | All You Need is LUV: Unsupervised Collection of Labeled Images Using UV-Fluorescent MarkingsabstractLearning-based perception systems in robotics often requires large-scale image segmentation annotation. Current approaches rely on human labelers, which can be expensive, or simulation data, which can visually differ from real data. This paper proposes Labels from UltraViolet (LUV), a novel framework that enables rapid, automated, inexpensive, high quality data collection in real. LUV uses transparent, UV-fluorescent paint with programmable UV LEDs to collect paired images of a scene in standard and UV lighting. This makes it possible to autonomously extract segmentation masks and keypoints via color thresholding. We apply LUV to a suite of diverse robot perception tasks: locating fabric keypoints, cable segmentation, and surgical needle detection to evaluate its labeling quality, flexibility, and data collection rate. Results suggest that LUV is 180–2500 times faster than a human labeler across the tasks while retaining accuracy and strong task performance. Code, datasets, visualizations, and supplementary material can be found at https://sites.google.com/berkeley.edu/luv. Brijen Thananjeyan, Justin Kerr, Joseph Gonzalez 0001, Kenneth Y. Goldberg |
IROS | 5 |
| 2022 | Multi-object Grasping in the Plane
Wisdom C. Agboh, Jeffrey Ichnowski, Kenneth Y. Goldberg, Mehmet Remzi Dogar |
ISRR | 3 |
| 2022 | Efficiently Learning Single-Arm Fling Motions to Smooth Garments
Yunliang Chen 0001, Ellen R. Novoseller, Daniel Seita, Jeffrey Ichnowski, Michael Laskey, Richard Cheng, Thomas Kollar, Kenneth Y. Goldberg |
ISRR | 9 |
| 2022 | Mechanical Search on Shelves with Efficient Stacking and Destacking of Objects
Letian Fu, Michael Danielczuk, Chung Min Kim, Zachary Tam, Jeffrey Ichnowski, Anelia Angelova, Brian Ichter, Kenneth Y. Goldberg |
ISRR | 9 |
| 2022 | Monte Carlo Augmented Actor-Critic for Sparse Reward Deep Reinforcement Learning from Suboptimal DemonstrationsabstractProviding densely shaped reward functions for RL algorithms is often exceedingly challenging, motivating the development of RL algorithms that can learn from easier-to-specify sparse reward functions. This sparsity poses new exploration challenges. One common way to address this problem is using demonstrations to provide initial signal about regions of the state space with high rewards. However, prior RL from demonstrations algorithms introduce significant complexity and many hyperparameters, making them hard to implement and tune. We introduce Monte Carlo Actor-Critic (MCAC), a parameter free modification to standard actor-critic algorithms which initializes the replay buffer with demonstrations and computes a modified $Q$-value by taking the maximum of the standard temporal distance (TD) target and a Monte Carlo estimate of the reward-to-go. This encourages exploration in the neighborhood of high-performing trajectories by encouraging high $Q$-values in corresponding regions of the state space. Experiments across $5$ continuous control domains suggest that MCAC can be used to significantly increase learning efficiency across $6$ commonly used RL and RL-from-demonstrations algorithms. See https://sites.google.com/view/mcac-rl for code and supplementary material. Albert Wilcox, Ashwin Balakrishna, Jules Dedieu, Wyame Benslimane, Daniel S. Brown, Kenneth Y. Goldberg |
NeurIPS | 6 |
| 2022 | GOMP-ST: Grasp Optimized Motion Planning for Suction Transport
Yahav Avigal, Jeffrey Ichnowski, Max Yiye Cao, Kenneth Y. Goldberg |
WAFR | 4 |
| 2022 | Concepts and Trends in Autonomy for Robot-Assisted SurgeryabstractSurgical robots have been widely adopted with over 4000 robots being used in practice daily. However, these are telerobots that are fully controlled by skilled human surgeons. Introducing "surgeon-assist"-some forms of autonomy-has the potential to reduce tedium and increase consistency, analogous to driver-assist functions for lanekeeping, cruise control, and parking. This article examines the scientific and technical backgrounds of robotic autonomy in surgery and some ethical, social, and legal implications. We describe several autonomous surgical tasks that have been automated in laboratory settings, and research concepts and trends. Paolo Fiorini, Kenneth Y. Goldberg, Yun-Hui Liu 0001, Russell H. Taylor |
Proc. IEEE | 2 |
| 2022 | Simulating Polyculture Farming to Learn Automation Policies for Plant Diversity and Precision IrrigationabstractPolyculture farming, where multiple crop species are grown simultaneously, has potential to reduce pesticide and water usage while improving the utilization of soil nutrients. However, it is much harder to automate polyculture than monoculture. To facilitate research, we present AlphaGardenSim, a fast, first order, open-access polyculture farming simulator with single plant growth and irrigation models tuned using real world measurements. AlphaGardenSim can be used for policy learning as it simulates inter-plant dynamics, including light and water competition between plants in close proximity and approximates growth in a real greenhouse garden at 25,$000\times $the speed of natural growth. This paper extends earlier work with a new action space that includes planting, which dynamically finds new seed locations that increases resources utilization, and an adaptive sampling technique to reduce the number of actions taken at each timestep without affecting performance. We also evaluate other automation policies using a novel metric that combines plant diversity and canopy coverage. Code and supplementary material can be found athttps://github.com/BerkeleyAutomation/AlphaGarden.Note to Practitioners—Monoculture farming is often characterized by heavy agrichemical inputs, such as chemical fertilizers and pesticides, and increased vulnerability to disease and pestilence. This paper is motivated by the lack of long-term sustainability of industrial agriculture, and its implications for human food security. Although polyculture is a sustainable alternative to monoculture farming, it requires more human labor and is more challenging to automate. In this paper we propose a fast, first order simulator that simulates the growth of plants in a polyculture setting. Simulation experiments suggest that the simulator can be used to learn a planting, watering and pruning plan a robot can follow to produce maximal yield from a diverse set of plants with limited irrigation, however it has not yet been tested on a physical garden. In future research we will develop a fully automated controller that will operate planting, irrigation and pruning tools in a physical garden over multiple plant growth cycles. Yahav Avigal, Mark Presten, Mark Theis, Shrey Aeron, Anna Deza, Satvik Sharma, Rishi Parikh, Sebastian Oehme, Stefano Carpin, Joshua Viers, Stavros G. Vougioukas, Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 13 |
| 2021 | Policy Gradient Bayesian Robust Optimization for Imitation LearningabstractThe difficulty in specifying rewards for many real-world problems has led to an increased focus on learning rewards from human feedback, such as demonstrations. However, there are often many different reward functions that explain the human feedback, leaving agents with uncertainty over what the true reward function is. While most policy optimization approaches handle this uncertainty by optimizing for expected performance, many applications demand risk-averse behavior. We derive a novel policy gradient-style robust optimization approach, PG-BROIL, that optimizes a soft-robust objective that balances expected performance and risk. To the best of our knowledge, PG-BROIL is the first policy optimization algorithm robust to a distribution of reward hypotheses which can scale to continuous MDPs. Results suggest that PG-BROIL can produce a family of behaviors ranging from risk-neutral to risk-averse and outperforms state-of-the-art imitation learning algorithms when learning from ambiguous demonstrations by hedging against uncertainty, rather than seeking to uniquely identify the demonstrator’s reward function. Zaynah Javed, Daniel S. Brown, Satvik Sharma, Jerry Zhu, Ashwin Balakrishna, Marek Petrik, Anca D. Dragan, Kenneth Y. Goldberg |
ICML | 8 |
| 2021 | Resource Allocation in Multi-armed Bandit Exploration: Overcoming Sublinear Scaling with Adaptive ParallelismabstractWe study exploration in stochastic multi-armed bandits when we have access to a divisible resource that can be allocated in varying amounts to arm pulls. We focus in particular on the allocation of distributed computing resources, where we may obtain results faster by allocating more resources per pull, but might have reduced throughput due to nonlinear scaling. For example, in simulation-based scientific studies, an expensive simulation can be sped up by running it on multiple cores. This speed-up however, is partly offset by the communication among cores, which results in lower throughput than if fewer cores were allocated to run more trials in parallel. In this paper, we explore these trade-offs in two settings. First, in a fixed confidence setting, we need to find the best arm with a given target success probability as quickly as possible. We propose an algorithm which trades off between information accumulation and throughput and show that the time taken can be upper bounded by the solution of a dynamic program whose inputs are the gaps between the sub-optimal and optimal arms. We also prove a matching hardness result. Second, we present an algorithm for a fixed deadline setting, where we are given a time deadline and need to maximize the probability of finding the best arm. We corroborate our theoretical insights with simulation experiments that show that the algorithms consistently match or outperform baseline algorithms on a variety of problem instances. Brijen Thananjeyan, Kirthevasan Kandasamy, Ion Stoica, Michael I. Jordan, Kenneth Y. Goldberg, Joseph Gonzalez 0001 |
ICML | 5 |
| 2021 | Serverless Multi-Query Motion Planning for Fog RoboticsabstractRobots in semi-structured environments such as homes and warehouses sporadically require computation of high-dimensional motion plans. Cloud and fog-based parallelization of motion planning can speed up planning. This can be further made efficient by the use of "serverless" on-demand computing as opposed to always-on high end computers. This paper explores parallelizing the computation of a sampling-based multi-query motion planner based on asymptotically-optimal Probabilistic Road Maps (PRM*) using the simultaneous execution of 100s of cloud-based serverless functions. We propose an algorithm to overcome the communication and bandwidth limitations of serverless computing and use different work-sharing techniques to further optimize the cost and run time. Additionally, we provide proofs of probabilistic completeness and asymptotic optimality. In experiments on synthetic benchmarks and on a physical Fetch robot performing a sequence of decluttering motions, we observe up to a 50x speedup relative to a 4 core edge computer with only a marginally higher cost. Raghav Anand, Jeffrey Ichnowski, Chenggang Wu 0001, Joseph M. Hellerstein, Joseph Gonzalez 0001, Kenneth Y. Goldberg |
ICRA | 6 |
| 2021 | Learning Seed Placements and Automation Policies for Polyculture Farming with Companion PlantsabstractPolyculture farming is a sustainable farming technique based on synergistic interactions between differing plant types that make them more resistant to diseases and pests and better able to retain water. Reduced uniformity can reduce use of pesticides, fertilizer, and water, but is more labor intensive and more challenging to automate. We describe a scaled physical testbed (1.5m×3.0m) that uses a high resolution camera and soil sensors to monitor polyculture plants to facilitate tuning of plant growth, companion effects, and irrigation parameters for a first-order garden simulator. We use this simulator to develop a novel seed placement algorithm that increases coverage and diversity, and a learned pruning policy. In simulation experiments, the seed placement algorithm yields 60% more coverage and 10% more diversity than random seed placement and the learned pruning policy runs 1000X faster than a procedural lookahead policy to achieve high leaf coverage and plant diversity on adversarial gardens that include plant species with diverse growth rates. These models and policies provide the groundwork for a fully-automated system under development. Code, datasets and supplementary material can be found at https://github.com/BerkeleyAutomation/AlphaGarden/. Yahav Avigal, Anna Deza, Sebastian Oehme, Mark Presten, Mark Theis, Jackson Chui, Paul Shao, Atsunobu Kotani, Satvik Sharma, Rishi Parikh, Michael Luo, Sandeep Mukherjee, Stefano Carpin, Joshua Viers, Stavros G. Vougioukas, Kenneth Y. Goldberg |
ICRA | 18 |
| 2021 | Learning Dense Visual Correspondences in Simulation to Smooth and Fold Real FabricsabstractRobotic fabric manipulation is challenging due to the infinite dimensional configuration space, self-occlusion, and complex dynamics of fabrics. There has been significant prior work on learning policies for specific fabric manipulation tasks, but comparatively less focus on algorithms which can perform many different tasks. We take a step towards this goal by learning point-pair correspondences across different fabric configurations in simulation. Then, given a single demonstration of a new task from an initial fabric configuration, these correspondences can be used to compute geometrically equivalent actions in a new fabric configuration. This makes it possible to define policies to robustly imitate a broad set of multi-step fabric smoothing and folding tasks. The resulting policies achieve 80.3% average task success rate across 10 fabric manipulation tasks on two different physical robotic systems. Results also suggest robustness to fabrics of various colors, sizes, and shapes. See https://tinyurl.com/fabric-descriptors for supplementary material and videos. Aditya Ganapathi, Priya Sundaresan, Brijen Thananjeyan, Ashwin Balakrishna, Daniel Seita, Jennifer Grannen, Minho Hwang, Ryan Hoque, Joseph Gonzalez 0001, Nawid Jamali, Katsu Yamane, Soshi Iba, Kenneth Y. Goldberg |
ICRA | 13 |
| 2021 | Semantic and Geometric Modeling with Neural Message Passing in 3D Scene Graphs for Hierarchical Mechanical SearchabstractSearching for objects in indoor organized environments such as homes or offices is part of our everyday activities. When looking for a desired object, we reason about the rooms and containers the object is likely to be in; the same type of container will have a different probability of containing the target depending on which room it is in. We also combine geometric and semantic information to infer what container is best to search, or what other objects are best to move, if the target object is hidden from view. We use a 3D scene graph representation to capture the hierarchical, semantic, and geometric aspects of this problem. To exploit this representation in a search process, we introduce Hierarchical Mechanical Search (HMS), a method that guides an agent’s actions towards finding a target object specified with a natural language description. HMS is based on a novel neural network architecture that uses neural message passing of vectors with visual, geometric, and linguistic information to allow HMS to process data across layers of the graph while combining semantic and geometric cues. HMS is trained on 1000 3D scene graphs and evaluated on a novel dataset of 500 3D scene graphs with dense placements of semantically related objects in storage locations, and is shown to be significantly better than several baselines at finding objects. It is also close to the oracle policy in terms of the median number of actions required. Additional qualitative results can be found at https://ai.stanford.edu/mech-search/hms Andrey Kurenkov, Roberto Martin Martin, Jeffrey Ichnowski, Kenneth Y. Goldberg, Silvio Savarese |
ICRA | 4 |
| 2021 | Intermittent Visual Servoing: Efficiently Learning Policies Robust to Instrument Changes for High-precision Surgical ManipulationabstractAssisting surgeons with automation of surgical subtasks is challenging due to backlash, hysteresis, and variable tensioning in cable-driven robots. These issues are exacerbated as surgical instruments are changed during an operation. In this work, we propose a framework for automation of high- precision surgical subtasks by learning local, sample-efficient, accurate, closed-loop policies that use visual feedback instead of robot encoder estimates. This framework, which we call deep Intermittent Visual Servoing (IVS), switches to a learned visual servo policy for high-precision segments of repetitive surgical tasks while relying on a coarse open-loop policy for the segments where precision is not necessary. We train the policy using only 180 human demonstrations that are roughly 2 seconds each. Results on a da Vinci Research Kit suggest that combining the coarse policy with half a second of corrections from the learned policy during each high-precision segment improves the success rate on the Fundamentals of Laparoscopic Surgery peg transfer task from 72.9% to 99.2%, 31.3% to 99.2%, and 47.2% to 100.0% for 3 instruments with differing cable properties. In the contexts we studied, IVS attains the highest published success rates for automated surgical peg transfer and is significantly more reliable than previous techniques when instruments are changed. Supplementary material is available at https://tinyurl.com/ivs-icra. Samuel Paradis, Minho Hwang, Brijen Thananjeyan, Jeffrey Ichnowski, Daniel Seita, Danyal Fer, Thomas Low, Joseph Gonzalez 0001, Kenneth Y. Goldberg |
ICRA | 9 |
| 2021 | Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter NetworksabstractRearranging and manipulating deformable objects such as cables, fabrics, and bags is a long-standing challenge in robotic manipulation. The complex dynamics and high-dimensional configuration spaces of deformables, compared to rigid objects, make manipulation difficult not only for multi-step planning, but even for goal specification. Goals cannot be as easily specified as rigid object poses, and may involve complex relative spatial relations such as "place the item inside the bag". In this work, we develop a suite of simulated benchmarks with 1D, 2D, and 3D deformable structures, including tasks that involve image-based goal-conditioning and multi-step deformable manipulation. We propose embedding goal-conditioning into Transporter Networks, a recently proposed model architecture for learning robotic manipulation that rearranges deep features to infer displacements that can represent pick and place actions. We demonstrate that goal-conditioned Transporter Networks enable agents to manipulate deformable structures into flexibly specified configurations without test-time visual anchors for target locations. We also significantly extend prior results using Transporter Networks for manipulating deformable objects by testing on tasks with 2D and 3D deformables. Supplementary material is available at https://berkeleyautomation.github.io/bags/. Daniel Seita, Peter R. Florence, Jonathan Tompson, Erwin Coumans, Vikas Sindhwani, Kenneth Y. Goldberg, Andy Zeng 0001 |
ICRA | 6 |
| 2021 | Robots of the Lost Arc: Self-Supervised Learning to Dynamically Manipulate Fixed-Endpoint CablesabstractWe explore how high-speed robot arm motions can dynamically manipulate ropes and cables to vault over obstacles, knock objects from pedestals, and weave between obstacles. In this paper, we propose a self-supervised learning framework that enables a UR5 robot to perform these three tasks. The framework finds a 3D apex point for the robot arm, which, together with a task-specific trajectory function, defines an arcing motion that dynamically manipulates the cable to perform a task with varying obstacle and target locations. The trajectory function computes minimum-jerk motions that are constrained to remain within joint limits and to travel through the 3D apex point by repeatedly solving quadratic programs to find the shortest and fastest feasible motion. We experiment with 5 physical cables with different thickness and mass and compare performance against two baselines in which a human chooses the apex point. Results suggest that a baseline with a fixed apex across the three tasks achieves respective success rates of 51.7 %, 36.7 %, and 15.0 %, and a baseline with human-specified, task-specific apex points achieves 66.7 %, 56.7 %, and 15.0 % success rate respectively, while the robot using the learned apex point can achieve success rates of 81.7 % in vaulting, 65.0 % in knocking, and 60.0 % in weaving. Code, data, and supplementary materials are available at https://sites.google.com/berkeley.edu/dynrope/home. Harry Zhang, Jeffrey Ichnowski, Daniel Seita, Kenneth Y. Goldberg |
ICRA | 6 |
| 2021 | Mechanical Search on Shelves using Lateral Access X-RAYabstractFinding an occluded object in a lateral access environment such as a shelf or cabinet is a problem that arises in many contexts such as warehouses, retail, healthcare, shipping, and homes. While this problem, known as mechanical search, is well-studied in overhead access environments, lateral access environments introduce constraints on the poses of objects and on available grasp actions, and pushing actions are preferred to preserve the environment structure. We propose LAX-RAY (Lateral Access maXimal Reduction in support Area of occupancY distribution): a system that combines target object occupancy distribution prediction with a mechanical search policy that sequentially pushes occluding objects to reveal a given target object. For scenarios with extruded polygonal objects, we introduce two lateral-access search policies that encode a history of predicted target distributions and can plan up to three actions into the future. We introduce a First-Order Shelf Simulator (FOSS) and use it to evaluate these policies in 800 simulated random shelf environments per policy. We also evaluate in 5 physical shelf environments using a Fetch robot with an embedded PrimeSense RGBD Camera and an attached pushing blade. The policies outperform baselines by up to 25% in simulation and up to 60% in physical experiments. Additionally, the two-step prediction policy is the highest performing in simulation for 8 objects with a 69% success rate, suggesting a tradeoff between future information and prediction errors. Code, videos, and supplementary material can be found at https://sites.google.com/berkeley.edu/lax-ray. Marcus Dominguez-Kuhne, Vishal Satish, Michael Danielczuk, Kate Sanders 0002, Jeffrey Ichnowski, Andrew Lee 0002, Anelia Angelova, Vincent Vanhoucke, Kenneth Y. Goldberg |
IROS | 10 |
| 2021 | A Multi-Chamber Smart Suction Cup for Adaptive Gripping and Haptic ExplorationabstractWe present a novel robot end-effector for gripping and haptic exploration. Tactile sensing through suction flow monitoring is achieved with a new suction cup design that contains multiple chambers for air flow. Each chamber connects with its own remote pressure transducer, which enables both absolute and differential pressure measures between chambers. By changing the overall vacuum applied to this smart suction cup, it can perform different functions such as gentle haptic exploration (low pressure) and monitoring breaks in the seal during strong astrictive gripping (high pressure). Haptic exploration of surfaces through sliding and palpation can guide the selection of suction grasp locations and help to identify the local surface geometry. During suction gripping, a trained LSTM network can localize breaks in the suction seal between four quadrants with up to 97% accuracy and detects breaks in the suction seal early enough to avoid total grasp failure. Tae Myung Huh, Kate Sanders 0002, Michael Danielczuk, Monica S. Li, Yunliang Chen 0001, Kenneth Y. Goldberg, Hannah Stuart |
IROS | 6 |
| 2021 | Disentangling Dense Multi-Cable KnotsabstractDisentangling two or more cables often requires many steps to remove crossings between and within cables. We formalize the problem of disentangling multiple cables and present an algorithm, Iterative Reduction Of Non-planar Multiple cAble kNots (IRON-MAN), that outputs robot actions to remove crossings from multi-cable knotted structures. IRON-MAN uses a learned perception system inspired by prior work in single-cable untying to imitate a graph-based supervisor, and operates on RGB image inputs of the workspace. Given a sequence of images as input, the system can disentangle two-cable twists, three-cable braids, and knots of two or three cables, such as overhand, square, carrick bend, sheet bend, crown, and fisherman’s knots. IRON-MAN keeps track of task-relevant keypoints corresponding to cable endpoints and crossings and iteratively disentangles the cables by identifying and undoing crossings that are critical to knot structure. Using a da Vinci surgical robot, we experimentally evaluate the effectiveness of IRON-MAN on disentangling multi-cable knots of types that appear in the training data, as well as generalizing to novel classes of multi-cable knots. Results suggest that IRON-MAN is effective in disentangling knots involving up to three cables with 80.5% success and generalizing to knot types that are not present during training, with cables of identical or distinct colors. Supplementary material and videos can be found at https://tinyurl.com/multi-cable-disentangling. Vainavi Viswanath, Jennifer Grannen, Priya Sundaresan, Brijen Thananjeyan, Ashwin Balakrishna, Ellen R. Novoseller, Jeffrey Ichnowski, Michael Laskey, Joseph Gonzalez 0001, Kenneth Y. Goldberg |
IROS | 10 |
| 2021 | Accelerating Quadratic Optimization with Reinforcement LearningabstractFirst-order methods for quadratic optimization such as OSQP are widely used for large-scale machine learning and embedded optimal control, where many related problems must be rapidly solved. These methods face two persistent challenges: manual hyperparameter tuning and convergence time to high-accuracy solutions. To address these, we explore how Reinforcement Learning (RL) can learn a policy to tune parameters to accelerate convergence. In experiments with well-known QP benchmarks we find that our RL policy, RLQP, significantly outperforms state-of-the-art QP solvers by up to 3x. RLQP generalizes surprisingly well to previously unseen problems with varying dimension and structure from different applications, including the QPLIB, Netlib LP and Maros-M{\'e}sz{\'a}ros problems. Code, models, and videos are available at https://berkeleyautomation.github.io/rlqp/. Jeffrey Ichnowski, Paras Jain 0001, Bartolomeo Stellato, Goran Banjac, Michael Luo, Francesco Borrelli, Joseph Gonzalez 0001, Ion Stoica, Kenneth Y. Goldberg |
NeurIPS | 9 |
| 2021 | ABC-LMPC: Safe Sample-Based Learning MPC for Stochastic Nonlinear Dynamical Systems with Adjustable Boundary Conditions
Brijen Thananjeyan, Ashwin Balakrishna, Ugo Rosolia, Joseph Gonzalez 0001, Aaron D. Ames, Kenneth Y. Goldberg |
WAFR | 6 |
| 2021 | Sim2Real in Robotics and Automation: Applications and ChallengesabstractTo Perform reliably and consistently over sustained periods of time, large-scale automation critically relies on computer simulation. Simulation allows us and supervisory AI to effectively design, validate, and continuously improve complex processes, and helps practitioners to gain insight into the operation and justify future investments. While numerous successful applications of simulation in industry exist, such as circuit simulation, finite element methods, and computeraided design (CAD), state-of-the-art simulators fall short of accurately modeling physical phenomena, such as friction, impact, and deformation. Sebastian Höfer, Kostas E. Bekris, Ankur Handa, Juan Camilo Gamboa, Melissa Mozifian, Florian Golemo, Christopher G. Atkeson, Dieter Fox, Kenneth Y. Goldberg, John J. Leonard, C. Karen Liu, Jan Peters 0001, Shuran Song, Peter Welinder, Martha White |
IEEE Trans Autom. Sci. Eng. | 9 |
| 2020 | Online Learning with Continuous Variations: Dynamic Regret and ReductionsabstractOnline learning is a powerful tool for analyzing iterative algorithms. However, the classic adversarial setup fails to capture regularity that can exist in practice. Motivated by this observation, we establish a new setup, called Continuous Online Learning (COL), where the gradient of online loss function changes continuously across rounds with respect to the learner’s decisions. We show that COL appropriately describes many interesting applications, from general equilibrium problems (EPs) to optimization in episodic MDPs. Using this new setup, we revisit the difficulty of sublinear dynamic regret. We prove a fundamental equivalence between achieving sublinear dynamic regret in COL and solving certain EPs. With this insight, we offer conditions for efficient algorithms that achieve sublinear dynamic regret, even when the losses are chosen adaptively without any a priori variation budget. Furthermore, we show for COL a reduction from dynamic regret to both static regret and convergence in the associated EP, allowing us to analyze the dynamic regret of many existing algorithms. Ching-An Cheng, Jonathan Lee 0002, Kenneth Y. Goldberg, Byron Boots |
AISTATS | 3 |
| 2020 | GOMP: Grasp-Optimized Motion Planning for Bin PickingabstractRapid and reliable robot bin picking is a critical challenge in automating warehouses, often measured in picks-per-hour (PPH). We explore increasing PPH using faster motions based on optimizing over a set of candidate grasps. The source of this set of grasps is two-fold: (1) grasp-analysis tools such as Dex-Net generate multiple candidate grasps, and (2) each of these grasps has a degree of freedom about which a robot gripper can rotate. In this paper, we present Grasp-Optimized Motion Planning (GOMP), an algorithm that speeds up the execution of a bin-picking robot's operations by incorporating robot dynamics and a set of candidate grasps produced by a grasp planner into an optimizing motion planner. We compute motions by optimizing with sequential quadratic programming (SQP) and iteratively updating trust regions to account for the non-convex nature of the problem. In our formulation, we constrain the motion to remain within the mechanical limits of the robot while avoiding obstacles. We further convert the problem to a time-minimization by repeatedly shorting a time horizon of a trajectory until the SQP is infeasible. In experiments with a UR5, GOMP achieves a speedup of 9x over a baseline planner. Jeffrey Ichnowski, Michael Danielczuk, Vishal Satish, Kenneth Y. Goldberg |
ICRA | 5 |
| 2020 | Fog Robotics Algorithms for Distributed Motion Planning Using Lambda Serverless ComputingabstractFor robots using motion planning algorithms such as RRT and RRT*, the computational load can vary by orders of magnitude as the complexity of the local environment changes. To adaptively provide such computation, we propose Fog Robotics algorithms in which cloud-based serverless lambda computing provides parallel computation on demand. To use this parallelism, we propose novel motion planning algorithms that scale effectively with an increasing number of serverless computers. However, given that the allocation of computing is typically bounded by both monetary and time constraints, we show how prior learning can be used to efficiently allocate resources at runtime. We demonstrate the algorithms and application of learned parallel allocation in both simulation and with the Fetch commercial mobile manipulator using Amazon Lambda to complete a sequence of sporadically computationally intensive motion planning tasks. Jeffrey Ichnowski, Victor Murta, Samuel Paradis, Ron Alterovitz, Joseph Gonzalez 0001, Ion Stoica, Kenneth Y. Goldberg |
ICRA | 8 |
| 2020 | Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth DataabstractRobotic manipulation of deformable 1D objects such as ropes, cables, and hoses is challenging due to the lack of high-fidelity analytic models and large configuration spaces. Furthermore, learning end-to-end manipulation policies directly from images and physical interaction requires significant time on a robot and can fail to generalize across tasks. We address these challenges using interpretable deep visual representations for rope, extending recent work on dense object descriptors for robot manipulation. This facilitates the design of interpretable and transferable geometric policies built on top of the learned representations, decoupling visual reasoning and control. We present an approach that learns point-pair correspondences between initial and goal rope configurations, which implicitly encodes geometric structure, entirely in simulation from synthetic depth images. We demonstrate that the learned representation — dense depth object descriptors (DDODs) — can be used to manipulate a real rope into a variety of different arrangements either by learning from demonstrations or using interpretable geometric policies. In 50 trials of a knot-tying task with the ABB YuMi Robot, the system achieves a 66% knot-tying success rate from previously unseen configurations. See https://tinyurl.com/rope-learning for supplementary material and videos. Priya Sundaresan, Jennifer Grannen, Brijen Thananjeyan, Ashwin Balakrishna, Michael Laskey, Kevin Stone, Joseph Gonzalez 0001, Kenneth Y. Goldberg |
ICRA | 8 |
| 2020 | Motion2Vec: Semi-Supervised Representation Learning from Surgical VideosabstractLearning meaningful visual representations in an embedding space can facilitate generalization in downstream tasks such as action segmentation and imitation. In this paper, we learn a motion-centric representation of surgical video demonstrations by grouping them into action segments/subgoals/options in a semi-supervised manner. We present Motion2Vec, an algorithm that learns a deep embedding feature space from video observations by minimizing a metric learning loss in a Siamese network: images from the same action segment are pulled together while pushed away from randomly sampled images of other segments, while respecting the temporal ordering of the images. The embeddings are iteratively segmented with a recurrent neural network for a given parametrization of the embedding space after pre-training the Siamese network. We only use a small set of labeled video segments to semantically align the embedding space and assign pseudo-labels to the remaining unlabeled data by inference on the learned model parameters. We demonstrate the use of this representation to imitate surgical suturing kinematic motions from publicly available videos of the JIGSAWS dataset. Results give 85.5% segmentation accuracy on average suggesting performance improvement over several state-of-the-art baselines, while kinematic pose imitation gives 0.94 centimeter error in position per observation on the test set. Videos, code and data are available at: https://sites.google.com/view/motion2vec. Ajay Kumar Tanwani, Pierre Sermanet, Andy Yan, Raghav Anand, Mariano Phielipp, Kenneth Y. Goldberg |
ICRA | 6 |
| 2020 | Minimal Work: A Grasp Quality Metric for Deformable Hollow ObjectsabstractRobot grasping of deformable hollow objects such as plastic bottles and cups is challenging, as the grasp should resist disturbances while minimally deforming the object so as not to damage it or dislodge liquids. We propose minimal work as a novel grasp quality metric that combines wrench resistance and object deformation. We introduce an efficient algorithm to compute the work required to resist an external wrench for a manipulation task by solving a linear program. The algorithm first computes the minimum required grasp force and an estimation of the gripper jaw displacements based on the object's empirical stiffness at different locations. The work done by the jaws is the product of the grasp force and the displacements. Grasps requiring minimal work are considered to be of high quality. We collect 460 physical grasps with a UR5 robot and a Robotiq gripper. We consider a grasp to be successful if it completes the task without damaging the object or dislodging the content. Physical experiments suggest that the minimal work quality metric reaches 74.2% balanced accuracy, a metric that is the raw accuracy normalized by the number of successful and failed real-world grasps, and is up to 24.2% higher than classical wrench-based quality metrics. Michael Danielczuk, Jeffrey Ichnowski, Jeffrey Mahler, Eckehard G. Steinbach, Kenneth Y. Goldberg |
ICRA | 6 |
| 2020 | 6DFC: Efficiently Planning Soft Non-Planar Area Contact Grasps using 6D Friction ConesabstractAnalytic grasp planning algorithms typically approximate compliant contacts with soft point contact models to compute grasp quality, but these models are overly conservative and do not capture the full range of grasps available. While area contact models can reduce the number of false negatives predicted by point contact models, they have been restricted to a 3D analysis of the wrench applied at the contact and so are still overly conservative. We extend traditional 3D friction cones and present an efficient algorithm for calculating the 6D friction cone (6DFC) for a non-planar area contact between a compliant gripper and a rigid object. We introduce a novel sampling algorithm to find the 6D friction limit surface for a non-planar area contact and a linearization method for these ellipsoids that reduces the computation of 6DFC constraints to a quadratic program. We show that constraining the wrench applied at the contact in this way increases recall, a metric inversely related to the number of false negative predictions, by 17% and precision, a metric inversely related to the number of false positive predictions, by 2% over soft point contact models on results from 1500 physical grasps on 12 3D printed nonplanar objects with an ABB YuMi robot. The 6DFC algorithm also achieves 6% higher recall with similar precision and 85x faster runtime than a previously proposed area contact model. Michael Danielczuk, Eckehard G. Steinbach, Kenneth Y. Goldberg |
ICRA | 4 |
| 2020 | Dex-Net AR: Distributed Deep Grasp Planning Using a Commodity Cellphone and Augmented Reality AppabstractConsumer demand for augmented reality (AR) in mobile phone applications, such as the Apple ARKit. Such applications have potential to expand access to robot grasp planning systems such as Dex-Net. AR apps use structure from motion methods to compute a point cloud from a sequence of RGB images taken by the camera as it is moved around an object. However, the resulting point clouds are often noisy due to estimation errors. We present a distributed pipeline, Dex-Net AR, that allows point clouds to be uploaded to a server in our lab, cleaned, and evaluated by Dex-Net grasp planner to generate a grasp axis that is returned and displayed as an overlay on the object. We implement Dex-Net AR using the iPhone and ARKit and compare results with those generated with high-performance depth sensors. The success rates with AR on harder adversarial objects are higher than traditional depth images. The server URL is https://sites.google.com/berkeley.edu/dex-net-ar/home. Harry Zhang, Jeffrey Ichnowski, Yahav Avigal, Joseph Gonzalez 0001, Ion Stoica, Kenneth Y. Goldberg |
ICRA | 6 |
| 2020 | X-Ray: Mechanical Search for an Occluded Object by Minimizing Support of Learned Occupancy DistributionsabstractFor applications in e-commerce, warehouses, healthcare, and home service, robots are often required to search through heaps of objects to grasp a specific target object. For mechanical search, we introduce X-Ray, an algorithm based on learned occupancy distributions. We train a neural network using a synthetic dataset of RGBD heap images labeled for a set of standard bounding box targets with varying aspect ratios. X-Ray minimizes support of the learned distribution as part of a mechanical search policy in both simulated and real environments. We benchmark these policies against two baseline policies on 1,000 heaps of 15 objects in simulation where the target object is partially or fully occluded. Results suggest that X-Ray is significantly more efficient, as it succeeds in extracting the target object 82% of the time, 15% more often than the best-performing baseline. Experiments on an ABB YuMi robot with 20 heaps of 25 household objects suggest that the learned policy transfers easily to a physical system, where it outperforms baseline policies by 15% in success rate with 17% fewer actions. Datasets, videos, and experiments are available at https://sites.google.com/berkeley.edu/x-ray. Michael Danielczuk, Anelia Angelova, Vincent Vanhoucke, Kenneth Y. Goldberg |
IROS | 4 |
| 2020 | Deep Imitation Learning of Sequential Fabric Smoothing From an Algorithmic SupervisorabstractSequential pulling policies to flatten and smooth fabrics have applications from surgery to manufacturing to home tasks such as bed making and folding clothes. Due to the complexity of fabric states and dynamics, we apply deep imitation learning to learn policies that, given color (RGB), depth (D), or combined color-depth (RGBD) images of a rectangular fabric sample, estimate pick points and pull vectors to spread the fabric to maximize coverage. To generate data, we develop a fabric simulator and an algorithmic supervisor that has access to complete state information. We train policies in simulation using domain randomization and dataset aggregation (DAgger) on three tiers of difficulty in the initial randomized configuration. We present results comparing five baseline policies to learned policies and report systematic comparisons of RGB vs D vs RGBD images as inputs. In simulation, learned policies achieve comparable or superior performance to analytic baselines. In 180 physical experiments with the da Vinci Research Kit (dVRK) surgical robot, RGBD policies trained in simulation attain coverage of 83% to 95% depending on difficulty tier, suggesting that effective fabric smoothing policies can be learned from an algorithmic supervisor and that depth sensing is a valuable addition to color alone. Supplementary material is available at https://sites.google.com/view/fabric-smoothing. Daniel Seita, Aditya Ganapathi, Ryan Hoque, Minho Hwang, Edward Cen, Ajay Kumar Tanwani, Ashwin Balakrishna, Brijen Thananjeyan, Jeffrey Ichnowski, Nawid Jamali, Katsu Yamane, Soshi Iba, John F. Canny, Kenneth Y. Goldberg |
IROS | 14 |
| 2020 | Multirobot Routing Algorithms for Robots Operating in VineyardsabstractWe consider the problem of multirobot routing in vineyards, a task motivated by our ongoing project aiming at creating a corobotic system to implement precision irrigation on large-scale commercial vineyards. The problem is related to a combinatorial optimization problem on graphs called “team orienteering.” Team orienteering is known to be NP-hard, thus motivating the development of heuristic solutions that can scale to large problem instances. We propose three different parameter-free approaches informed by the domain we consider and compare them against a general purpose heuristic developed previously. In numerous benchmarks derived from data gathered in a commercial vineyard, we demonstrate that our solutions outperform the general purpose heuristic and are scalable, thus allowing us to solve instances with tens of thousands of vertices in the graphs. Note to Practitioners-Routing problems with budget and motion constraints are pervasive to many applications. In particular, the structural constraints considered in this problem are found not only in agricultural environments but also in warehouse logistics and other domains where goods are arranged along regular linear structures. This article proposes and analyzes algorithms that can be applied when multiple agents must be coordinated in these environments. In particular, by utilizing domain-specific knowledge, the solutions proposed in this article outperform general purpose approaches that poorly scale with the size of the environment. The algorithms we present also ensure that no collisions occur between robots-an aspect normally neglected in algorithms previously proposed to solve the team orienteering problem. Thomas C. Thayer, Stavros G. Vougioukas, Kenneth Y. Goldberg, Stefano Carpin |
IEEE Trans Autom. Sci. Eng. | 3 |
| 2019 | Screen Smashing: Iconoclasty in an Age of Illusionary Intelligence (Sheridan See C&C'19 id# cckn07dis for correct DOI & pdf)abstract"Late twentieth-century machines have made thoroughly ambiguous the difference between natural and artificial, mind and body, self-developing and externally designed, and many other distinctions that used to apply to organisms and machines. Our machines are disturbingly lively, and we ourselves are frighteningly inert."" - Donna Haraway. Simians, Cyborgs, and Women: The Reinvention of Nature. The boundaries that Haraway lamented in 1990 were further eroded by the World Wide Web, wireless networks, smartphones, and today by social media, fake news, claims about Artificial Intelligence and an impending "Singularity". Are all boundaries truly illusionary? Or can we question illusions of illusion by actively asserting boundaries between ourselves and our machines? Artistic collaborators Shlain and Goldberg will describe how their art projects and experiences with technology are leading them to rediscover old barricades. Tiffany Shlain, Kenneth Y. Goldberg |
Conference on Designing Interactive Systems | 2 |
| 2019 | Screen Smashing: Iconoclasty in an Age of Illusionary Intelligenceabstract"Late twentieth-century machines have made thoroughly ambiguous the difference between natural and artificial, mind and body, self-developing and externally designed, and many other distinctions that used to apply to organisms and machines. Our machines are disturbingly lively, and we ourselves are frighteningly inert." - Donna Haraway. Simians, Cyborgs, and Women: The Reinvention of Nature. Tiffany Shlain, Kenneth Y. Goldberg |
Creativity & Cognition | 2 |
| 2019 | Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by ClutterabstractWhen operating in unstructured environments such as warehouses, homes, and retail centers, robots are frequently required to interactively search for and retrieve specific objects from cluttered bins, shelves, or tables. Mechanical Search describes the class of tasks where the goal is to locate and extract a known target object. In this paper, we formalize Mechanical Search and study a version where distractor objects are heaped over the target object in a bin. The robot uses an RGBD perception system and control policies to iteratively select, parameterize, and perform one of 3 actions - push, suction, grasp - until the target object is extracted, or either a time limit is exceeded, or no high confidence push or grasp is available. We present a study of 5 algorithmic policies for mechanical search, with 15,000 simulated trials and 300 physical trials for heaps ranging from 10 to 20 objects. Results suggest that success can be achieved in this long-horizon task with algorithmic policies in over 95% of instances and that the number of actions required scales approximately linearly with the size of the heap. Code and supplementary material can be found at http://ai.stanford.edu/mech-search. Michael Danielczuk, Andrey Kurenkov, Ashwin Balakrishna, Matthew Matl, Roberto Martin Martin, Animesh Garg, Silvio Savarese, Kenneth Y. Goldberg |
ICRA | 9 |
| 2019 | Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic DataabstractThe ability to segment unknown objects in depth images has potential to enhance robot skills in grasping and object tracking. Recent computer vision research has demonstrated that Mask R-CNN can be trained to segment specific categories of objects in RGB images when massive hand-labeled datasets are available. As generating these datasets is time-consuming, we instead train with synthetic depth images. Many robots now use depth sensors, and recent results suggest training on synthetic depth data can transfer successfully to the real world. We present a method for automated dataset generation and rapidly generate a synthetic training dataset of 50,000 depth images and 320,000 object masks using simulated heaps of 3D CAD models. We train a variant of Mask R-CNN with domain randomization on the generated dataset to perform category-agnostic instance segmentation without any hand-labeled data and we evaluate the trained network, which we refer to as Synthetic Depth (SD) Mask R-CNN, on a set of real, high-resolution depth images of challenging, densely-cluttered bins containing objects with highly-varied geometry. SD Mask R-CNN outperforms point cloud clustering baselines by an absolute 15% in Average Precision and 20% in Average Recall on COCO benchmarks, and achieves performance levels similar to a Mask R-CNN trained on a massive, hand-labeled RGB dataset and fine-tuned on real images from the experimental setup. We deploy the model in an instance-specific grasping pipeline to demonstrate its usefulness in a robotics application. Code, the synthetic training dataset, and supplementary material are available at https://bit.ly/2letCuE. Michael Danielczuk, Matthew Matl, Saurabh Gupta 0001, Andrew Li, Andrew Lee 0002, Jeffrey Mahler, Kenneth Y. Goldberg |
ICRA | 7 |
| 2019 | A Fog Robotics Approach to Deep Robot Learning: Application to Object Recognition and Grasp Planning in Surface DeclutteringabstractThe growing demand of industrial, automotive and service robots presents a challenge to the centralized Cloud Robotics model in terms of privacy, security, latency, bandwidth, and reliability. In this paper, we present a `Fog Robotics' approach to deep robot learning that distributes compute, storage and networking resources between the Cloud and the Edge in a federated manner. Deep models are trained on non-private (public) synthetic images in the Cloud; the models are adapted to the private real images of the environment at the Edge within a trusted network and subsequently, deployed as a service for low-latency and secure inference/prediction for other robots in the network. We apply this approach to surface decluttering, where a mobile robot picks and sorts objects from a cluttered floor by learning a deep object recognition and a grasp planning model. Experiments suggest that Fog Robotics can improve performance by sim-to-real domain adaptation in comparison to exclusively using Cloud or Edge resources, while reducing the inference cycle time by 4× to successfully declutter 86% of objects over 213 attempts. Ajay Kumar Tanwani, Nitesh Mor, John Kubiatowicz, Joseph Gonzalez 0001, Kenneth Y. Goldberg |
ICRA | 5 |
| 2019 | A Fog Robotic System for Dynamic Visual ServoingabstractCloud Robotics is a paradigm where multiple robots are connected to cloud services via Internet to access “unlimited” computation power, at the cost of network communication. However, due to limitations such as network latency and variability, it is difficult to control dynamic, human compliant service robots directly from the cloud. In this work, we combine cloud robotics with an agile edge device to build a Fog Robotic system by leveraging an asynchronous protocol with a “heartbeat” signal. We use the system to enable robust teleoperation of a dynamic self-balancing robot from the cloud. We use the system to pick up boxes from static locations, a task commonly performed in warehouse logistics. To make cloud teleoperation more intuitive and efficient, we program a cloud-based image based visual servoing (IBVS) module to automatically assist the cloud teleoperator during the object pickups. Visual feedbacks, including apriltag recognition and tracking, are performed in the cloud to emulate a Fog Robotic object recognition system for IBVS. We demonstrate the feasibility of a dynamic real-time automation system using this cloud-edge hybrid design, which opens up possibilities of deploying dynamic robotic control with deep-learning recognition systems in Fog Robotics. Finally, we show that Fog Robotics enables the self-balancing service robot to pick up a box automatically from a person under unstructured environments. Nan Tian, Ajay Kumar Tanwani, Jinfa Chen, Mas Ma, Robert Zhang 0001, Bill Huang, Kenneth Y. Goldberg, Somayeh Sojoudi |
ICRA | 7 |
| 2019 | Partial Caging: A Clearance-Based Definition and Deep LearningabstractCaging grasps limit the mobility of an object to a bounded component of configuration space. We introduce a notion of partial cage quality based on maximal clearance of an escaping path. As this is a computationally demanding task even in a two-dimensional scenario, we propose a deep learning approach. We design two convolutional neural networks and construct a pipeline for real-time partial cage quality estimation directly from 2D images of object models and planar caging tools. One neural network, CageMaskNN, is used to identify caging tool locations that can support partial cages, while a second network that we call CageClearanceNN is trained to predict the quality of those configurations. A dataset of 3811 images of objects and more than 19 million caging tool configurations is used to train and evaluate these networks on previously unseen objects and caging tool configurations. Furthermore, the networks are trained jointly on configurations for both 3 and 4 caging tool configurations whose shape varies along a 1-parameter family of increasing elongation. In experiments, we study how the networks' performance depends on the size of the training dataset, as well as how to efficiently deal with unevenly distributed training data. In further analysis, we show that the evaluation pipeline can approximately identify connected regions of successful caging tool placements and we evaluate the continuity of the cage quality score evaluation along caging tool trajectories. Experiments show that evaluation of a given configuration on a GeForce GTX 1080 GPU takes less than 6 ms. Anastasiia Varava, Michael C. Welle, Jeffrey Mahler, Kenneth Y. Goldberg, Danica Kragic, Florian T. Pokomy |
IROS | 4 |
| 2019 | REACH: Reducing False Negatives in Robot Grasp Planning with a Robust Efficient Area Contact Hypothesis Model
Michael Danielczuk, Jeffrey Mahler, Matthew Matl, Nuttapong Chentanez, Kenneth Y. Goldberg |
ISRR | 6 |
| 2019 | Deep Transfer Learning of Pick Points on Fabric for Robot Bed-Making
Daniel Seita, Nawid Jamali, Michael Laskey, Ajay Kumar Tanwani, Ron Berenstein, Prakash Baskaran, Soshi Iba, John F. Canny, Kenneth Y. Goldberg |
ISRR | 9 |
| 2019 | Mitigating Network Latency in Cloud-Based Teleoperation Using Motion Segmentation and Synthesis
Nan Tian, Ajay Kumar Tanwani, Kenneth Y. Goldberg, Somayeh Sojoudi |
ISRR | 3 |
| 2018 | Parametrized Hierarchical Procedures for Neural Programming
Roy Fox, Richard Shin, Sanjay Krishnan, Kenneth Y. Goldberg, Dawn Song, Ion Stoica |
ICLR (Poster) | 4 |
| 2018 | RLlib: Abstractions for Distributed Reinforcement LearningabstractReinforcement learning (RL) algorithms involve the deep nesting of highly irregular computation patterns, each of which typically exhibits opportunities for distributed computation. We argue for distributing RL components in a composable way by adapting algorithms for top-down hierarchical control, thereby encapsulating parallelism and resource requirements within short-running compute tasks. We demonstrate the benefits of this principle through RLlib: a library that provides scalable software primitives for RL. These primitives enable a broad range of algorithms to be implemented with high performance, scalability, and substantial code reuse. RLlib is available as part of the open source Ray project at http://rllib.io/. Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Kenneth Y. Goldberg, Joseph Gonzalez 0001, Michael I. Jordan, Ion Stoica |
ICML | 6 |
| 2018 | Robustly Adjusting Indoor Drip Irrigation Emitters with the Toyota HSR RobotabstractIndoor plants in homes and commercial buildings such as malls, offices, airports, and hotels, can benefit from precision irrigation to maintain healthy growth and reduce water consumption. As active valves are too costly, and ongoing precise manual adjustment of drip emitters is impractical, we explore how the Toyota HSR mobile manipulator robot can autonomously adjust low-cost passive emitters. To provide sufficient accuracy for gripper alignment, we designed a lightweight, modular Emitter Localization Device (ELD) with cameras and LEDs that can be non-invasively mounted on the arm. This paper presents details of the design, algorithms, and experiments with adjusting emitters using a two-phase procedure: (1) aligning the robot base using the build-in hand camera, and (2) aligning the gripper axis with the emitter axis using the ELD. We report success rates and sensitivity analysis to tune computer vision parameters and joint motor gains. Experiments suggest that emitters can be adjusted with 95 % success rate in approximately 20 seconds. Ron Berenstein, Roy Fox, Stephen McKinley, Stefano Carpin, Kenneth Y. Goldberg |
ICRA | 5 |
| 2018 | Dex-Net 3.0: Computing Robust Vacuum Suction Grasp Targets in Point Clouds Using a New Analytic Model and Deep LearningabstractVacuum-based end effectors are widely used in industry and are often preferred over parallel-jaw and multifinger grippers due to their ability to lift objects with a single point of contact. Suction grasp planners often target planar surfaces on point clouds near the estimated centroid of an object. In this paper, we propose a compliant suction contact model that computes the quality of the seal between the suction cup and local target surface and a measure of the ability of the suction grasp to resist an external gravity wrench. To characterize grasps, we estimate robustness to perturbations in end-effector and object pose, material properties, and external wrenches. We analyze grasps across 1,500 3D object models to generate Dex-Net 3.0, a dataset of 2.8 million point clouds, suction grasps, and grasp robustness labels. We use Dex-Net 3.0 to train a Grasp Quality Convolutional Neural Network (GQ-CNN) to classify robust suction targets in point clouds containing a single object. We evaluate the resulting system in 350 physical trials on an ABB YuMi fitted with a pneumatic suction gripper. When evaluated on novel objects that we categorize as Basic (prismatic or cylindrical), Typical (more complex geometry), and Adversarial (with few available suction-grasp points) Dex-Net 3.0 achieves success rates of 98%, 82%, and 58% respectively, improving to 81% in the latter case when the training set includes only adversarial objects. Code, datasets, and supplemental material can be found at http://berkeleyautomation.github.io/dex-net. Jeffrey Mahler, Matthew Matl, Xinyu Liu 0014, Albert Li, David V. Gealy, Kenneth Y. Goldberg |
ICRA | 6 |
| 2018 | Fast and Reliable Autonomous Surgical Debridement with Cable-Driven Robots Using a Two-Phase Calibration ProcedureabstractAutomating precision subtasks such as debridement (removing dead or diseased tissue fragments) with Robotic Surgical Assistants (RSAs) such as the da Vinci Research Kit (dVRK) is challenging due to inherent nOnlinearities in cable-driven systems. We propose and evaluate a novel two-phase coarse-to-fine calibration method. In Phase I (coarse), we place a red calibration marker on the end effector and let it randomly move through a set of open-loop trajectories to obtain a large sample set of camera pixels and internal robot end-effector configurations. This coarse data is then used to train a Deep Neural Network (DNN) to learn the coarse transformation bias. In Phase II (fine), the bias from Phase I is applied to move the end -effector toward a small set of specific target points on a printed sheet. For each target, a human operator manually adjusts the end -effector position by direct contact (not through teleoperation) and the residual compensation bias is recorded. This fine data is then used to train a Random Forest (RF) to learn the fine transformation bias. Subsequent experiments suggest that without calibration, position errors average 4.55mm. Phase I can reduce average error to 2.14mm and the combination of Phase I and Phase II can reduces average error to 1.08mm. We apply these results to debridement of raisins and pumpkin seeds as fragment phantoms. Using an endoscopic stereo camera with standard edge detection, experiments with 120 trials achieved average success rates of 94.5 %, exceeding prior results with much larger fragments (89.4%) and achieving a speedup of 2.1x, decreasing time per fragment from 15.8 seconds to 7.3 seconds. Source code, data, and videos are available at https://sites.google.com/view/calib-icra/. Daniel Seita, Sanjay Krishnan, Roy Fox, Stephen McKinley, John F. Canny, Kenneth Y. Goldberg |
ICRA | 6 |
| 2018 | Routing Algorithms for Robot Assisted Precision IrrigationabstractWhen robots navigate through vineyards to perform irrigation adjustments, an optimization problem emerges whereby robots are tasked with performing adjustments having the highest cumulative outcome within a given temporal budget due to limited battery charge. To this end, the robot needs to reach a set of spatially distributed sites, and the specific structure of the vineyard imposes various constraints on possible motions. In this paper we first demonstrate that this type of orienteering problem remains NP-hard even for the restricted class of graphs associated with precision irrigation. Then, we devise and analyze two greedy heuristics informed by the problem we consider. Finally, these algorithms are evaluated on settings associated with a commercial vineyard and we show that our methods favorably compare to solutions proposed in the past. Thomas C. Thayer, Stavros G. Vougioukas, Kenneth Y. Goldberg, Stefano Carpin |
ICRA | 3 |
| 2018 | Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality TeleoperationabstractImitation learning is a powerful paradigm for robot skill acquisition. However, obtaining demonstrations suitable for learning a policy that maps from raw pixels to actions can be challenging. In this paper we describe how consumer-grade Virtual Reality headsets and hand tracking hardware can be used to naturally teleoperate robots to perform complex tasks. We also describe how imitation learning can learn deep neural network policies (mapping from pixels to actions) that can acquire the demonstrated skills. Our experiments showcase the effectiveness of our approach for learning visuomotor skills. Tianhao Zhang 0001, Zoe McCarthy, Owen Jow, Dennis Lee 0003, Xi Chen 0022, Kenneth Y. Goldberg, Pieter Abbeel |
ICRA | 6 |
| 2018 | Towards a Soft Fingertip with Integrated Sensing and ActuationabstractSoft material robots are attractive for safe interaction with humans and unstructured environments due to their compliance and low intrinsic stiffness and mass. These properties enable new capabilities such as the ability to conform to environmental geometry for tactile sensing and to undergo large shape changes for actuation. Due to the complex coupling between sensing and actuation in high-dimensional nonlinear soft systems, prior work in soft robotics has primarily focused on either sensing or actuation. This paper presents SOFTcell, a novel controllable stiffness tactile device that incorporates both optical sensing and pneumatic actuation. We report details on the device's design and implementation and analyze results from characterization experiments on sensitivity and performance, which show that SOFTcell can controllably increase its effective modulus from 4.4kPa to 46.1kPa. Additionally, we demonstrate the utility of SOFTcell for grasping in a reactive control task in which tactile data is used to detect fingertip shear as a grasped object slips, and cell pressurization is used to prevent the slip without the need to adjust fingertip position. Benjamin W. McInroe, Carolyn L. Chen, Kenneth Y. Goldberg, Ruzena Bajcsy, Ronald S. Fearing |
IROS | 3 |
| 2018 | A Dynamic Regret Analysis and Adaptive Regularization Algorithm for On-Policy Robot Imitation LearningabstractOn-policy imitation learning algorithms such as Dagger evolve a robot control policy by executing it, measuring performance (loss), obtaining corrective feedback from a supervisor, and generating the next policy. As the loss between iterations can vary unpredictably, a fundamental question is under what conditions this process will eventually achieve a converged policy. If one assumes the underlying trajectory distribution is static (stationary), it is possible to prove convergence for Dagger. Cheng and Boots (2018) consider the more realistic model for robotics where the underlying trajectory distribution, which is a function of the policy, is dynamic and show that it is possible to prove convergence when a condition on the rate of change of the trajectory distributions is satisfied. In this paper, we reframe that result using dynamic regret theory from the field of Online Optimization to prove convergence to locally optimal policies for Dagger, Imitation Gradient, and Multiple Imitation Gradient. These results inspire a new algorithm, Adaptive On-Policy Regularization (AOR), that ensures the conditions for convergence. We present simulation results with cart-pole balancing and walker locomotion benchmarks that suggest AOR can significantly decrease dynamic regret and chattering. To our knowledge, this the first application of dynamic regret theory to imitation learning. Jonathan Lee 0002, Michael Laskey, Ajay Kumar Tanwani, Anil Aswani, Kenneth Y. Goldberg |
WAFR | 5 |
| 2018 | Generalizing Robot Imitation Learning with Invariant Hidden Semi-Markov ModelsabstractGeneralizing manipulation skills to new situations requires extracting invariant patterns from demonstrations. For example, the robot needs to understand the demonstrations at a higher level while being invariant to the appearance of the objects, geometric aspects of objects such as its position, size, orientation and viewpoint of the observer in the demonstrations. In this paper, we propose an algorithm that learns a joint probability density function of the demonstrations with invariant formulations of hidden semi-Markov models to extract invariant segments (also called sub-goals or options), and smoothly follow the generated sequence of states with a linear quadratic tracking controller. The algorithm takes as input the demonstrations observed with respect to different coordinate systems describing virtual landmarks or objects of interest, and adapts the segments according to the environmental changes in a systematic manner. We present variants of this algorithm in latent space with low-rank covariance decompositions, semi-tied covariances, and non-parametric online estimation of model parameters under small variance asymptotics; yielding considerably low sample and model complexity for acquiring new manipulation skills. The algorithm allows a Baxter robot to learn a pick-and-place task while avoiding a movable obstacle based on only 4 kinesthetic demonstrations. Ajay Kumar Tanwani, Jonathan Lee 0002, Brijen Thananjeyan, Michael Laskey, Sanjay Krishnan, Roy Fox, Kenneth Y. Goldberg, Sylvain Calinon |
WAFR | 7 |
| 2018 | Synthesis of Energy-Bounded Planar Caging Grasps Using Persistent HomologyabstractFor applications such as manufacturing, caging grasps restrict object motion without requiring complete immobilization, providing a robust alternative to forceand form-closure grasps. Energy-bounded cages are a new class of caging grasps that relax the requirement of complete caging in the presence of external forces such as gravity or constant velocity pushing in the horizontal plane with Coulomb friction. We address the problem of synthesizing planar energy-bounded cages by identifying gripper and force-direction configurations that maximize the energy required for the object to escape. We present Energy-BoundedCage-Synthesis-2-D (EBCS-2-D), a sampling-based algorithm that uses persistent homology, a recently-developed multiscale approach for topological analysis, to efficiently compute candidate rigid configurations of obstacles that form energy-bounded cages of an object from an α-shape approximation to the configuration space. If a synthesized configuration has infinite escape energy then the object is completely caged. EBCS-2-D runs in O(s3+ sn2) time, where s is the number of samples and n is the number of object and obstacle vertices, where typically n ≪ s. We observe runtimes closer to O(s) for fixed n. We implement EBCS-2-D using the persistent homology algorithms toolbox and study performance on a set of seven planar objects and four gripper types. Experiments suggest that EBCS-2-D takes 2-3 min on a 6 core processor with 200000 pose samples. We also confirm that an rapidly-exploring random tree* motion planner is unable to find escape paths with lower energy. Physical experiments on a five degree of freedom Zymark Zymate and ABB YuMi suggest that push grasps synthesized by EBCS-2-D are robust to perturbations. Data and code are available at http://berkeleyautomation.github.io/caging/. Jeffrey Mahler, Florian T. Pokorny, Sherdil Niyaz, Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 4 |
| 2018 | Guest Editorial Open Discussion of Robot Grasping Benchmarks, Protocols, and MetricsabstractAutomated grasping has a long history of research that is increasing due to interest from industry. One grand challenge for robotics is Universal Picking: the ability to robustly grasp a broad variety of objects in diverse environments for applications from warehouses to assembly lines to homes. Although many researchers now openly share code and data, it is challenging to compare and/or reproduce experimental results to identify which aspects of which approaches work best due to variations in assumptions and experimental protocols, e.g., sensors, lighting, robot arms, grippers, and objects. Jeffrey Mahler, Robert Platt 0001, Alberto Rodriguez 0003, Matei T. Ciocarlie, Aaron M. Dollar, Renaud Detry, Máximo A. Roa, Holly A. Yanco, Adam Norton, Joe Falco, Karl Van Wyk, Elena Messina, Jürgen Leitner, Douglas Morrison, Matthew T. Mason, Oliver Brock, Lael Odhner, Andrey Kurenkov, Matthew Matl, Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 20 |
| 2017 | Design of parallel-jaw gripper tip surfaces for robust graspingabstractParallel-jaw robot grippers can grasp almost any object and are ubiquitous in industry. Although the shape, texture, and compliance of gripper jaw surfaces affect grasp robustness, almost all commercially available grippers provide a pair of rectangular, planar, rigid jaw surfaces. Practitioners often modify these surfaces with a variety of ad-hoc methods such as adding rubber caps and/or wrapping with textured tape. This paper explores data-driven optimization of gripper jaw surfaces over a design space based on shape, texture, and compliance using rapid prototyping. In total, 37 jaw surface design variations were created using 3D printed casting molds and silicon rubber. The designs were evaluated with 1377 physical grasp experiments using a 4-axis robot (with automated reset). These tests evaluate grasp robustness as the probability that the jaws will acquire, lift, and hold a training set of objects at nominal grasp configurations computed by Dex-Net 1.0. Hill-climbing in parameter space yielded a grid pattern of 0.03 inch void depth and 0.0375 inch void width on a silicone polymer with durometer of A30. We then evaluated performance of this design using an ABB YuMi robot grasping a set of eight difficult-to-grasp 3D printed objects in 80 grasps with four gripper surfaces. The factory-provided gripper tips succeeded in 28.7% of the 80 trials, increasing to 68.7% when the tips were wrapped with tape. Gripper tips with gecko-inspired surfaces succeeded in 80.0% of trials, and gripper tips with the designed silicone surfaces succeeded in 93.7% of trials. Menglong Guo, David V. Gealy, Jacky Liang, Jeffrey Mahler, Aimee Goncalves, Stephen McKinley, Juan Aparicio Ojea, Kenneth Y. Goldberg |
ICRA | 8 |
| 2017 | Comparing human-centric and robot-centric sampling for robot deep learning from demonstrationsabstractMotivated by recent advances in Deep Learning for robot control, this paper considers two learning algorithms in terms of how they acquire demonstrations from fallible human supervisors. Human-Centric (HC) sampling is a standard supervised learning algorithm, where a human supervisor demonstrates the task by teleoperating the robot to provide trajectories consisting of state-control pairs. Robot-Centric (RC) sampling is an increasingly popular alternative used in algorithms such as DAgger, where a human supervisor observes the robot execute a learned policy and provides corrective control labels for each state visited. We suggest RC sampling can be challenging for human supervisors and prone to mislabeling. RC sampling can also induce error in policy performance because it repeatedly visits areas of the state space that are harder to learn. Although policies learned with RC sampling can be superior to HC sampling for standard learning models such as linear SVMs, policies learned with HC sampling may be comparable to RC when applied to expressive learning models such as deep learning and hyper-parametric decision trees, which can achieve very low training error provided there is enough data. We compare HC and RC using a grid world environment and a physical robot singulation task. In the latter the input is a binary image of objects on a planar worksurface and the policy generates a motion in the gripper to separate one object from the rest. We observe in simulation that for linear SVMs, policies learned with RC outperformed those learned with HC but that using deep models this advantage disappears. We also find that with RC, the corrective control labels provided by humans can be highly inconsistent. We prove there exists a class of examples in which at the limit, HC is guaranteed to converge to an optimal policy while RC may fail to converge. These results suggest a form of HC sampling may be preferable for highly-expressive learning models and human supervisors. Michael Laskey, Caleb Chuck, Jonathan Lee 0002, Jeffrey Mahler, Sanjay Krishnan, Kevin Jamieson 0001, Anca D. Dragan, Kenneth Y. Goldberg |
ICRA | 8 |
| 2017 | Multilateral surgical pattern cutting in 2D orthotropic gauze with deep reinforcement learning policies for tensioningabstractIn the Fundamentals of Laparoscopic Surgery (FLS) standard medical training regimen, the Pattern Cutting task requires residents to demonstrate proficiency by maneuvering two tools, surgical scissors and tissue gripper, to accurately cut a circular pattern on surgical gauze suspended at the corners. Accuracy of cutting depends on tensioning, wherein the gripper pinches a point on the gauze in R3and pulls to induce and maintain tension in the material as cutting proceeds. An automated tensioning policy maps the current state of the gauze to output a direction of pulling as an action. The optimal tensioning policy depends on both the choice of pinch point and cutting trajectory. We explore the problem of learning a tensioning policy conditioned on specific cutting trajectories. Every timestep, we allow the gripper to react to the deformation of the gauze and progress of the cutting trajectory with a translation unit vector along an allowable set of directions. As deformation is difficult to analytically model and explicitly observe, we leverage deep reinforcement learning with direct policy search methods to learn tensioning policies using a finite-element simulator and then transfer them to a physical system. We compare the Deep RL tensioning policies with fixed and analytic (opposing the error vector with a fixed pinch point) policies on a set of 17 open and closed curved contours in simulation and 4 patterns in physical experiments with the da Vinci Research Kit (dVRK). Our simulation results suggest that learning to tension with Deep RL can significantly improve performance and robustness to noise and external forces. Brijen Thananjeyan, Animesh Garg, Sanjay Krishnan, Carolyn Chen, Lauren Miller, Kenneth Y. Goldberg |
ICRA | 6 |
| 2017 | A cloud robot system using the dexterity network and berkeley robotics and automation as a service (Brass)abstractIn support of Cloud Robotics, Robotics and Automation as a Service (RAaaS) frameworks have the potential to reduce the complexity of software development, simplify software installation and maintenance, and facilitate data sharing for machine learning. In this proof-of-concept paper, we describe Berkeley Robotics and Automation as a Service (Brass), a RAaaS prototype that allows robots to access a remote server that hosts a robust grasp-planning system (Dex-Net 1.0) that maintains data on hundreds of candidate grasps on thousands of 3D object meshes and uses perturbation sampling to estimate and update a stochastic robustness metric for each grasp. Results suggest that such a system can increase grasp reliability over naive locally-computed grasping strategies with network latencies of 30 and 200 msec for servers 500 and 6000 miles away, respectively. We also study how the system can use execution reports from robots in the field to update grasp recommendations over time. Nan Tian, Matthew Matl, Jeffrey Mahler, Yu Xiang Zhou, Samantha Staszak, Christopher Correa, Steven Zheng, Robert Zhang 0001, Kenneth Y. Goldberg |
ICRA | 10 |
| 2016 | SHIV: Reducing supervisor burden in DAgger using support vectors for efficient learning from demonstrations in high dimensional state spacesabstractOnline learning from demonstration algorithms such as DAgger can learn policies for problems where the system dynamics and the cost function are unknown. However they impose a burden on supervisors to respond to queries each time the robot encounters new states while executing its current best policy. The MMD-IL algorithm reduces supervisor burden by filtering queries with insufficient discrepancy in distribution and maintaining multiple policies. We introduce the SHIV algorithm (Svm-based reduction in Human InterVention), which converges to a single policy and reduces supervisor burden in non-stationary high dimensional state distributions. To facilitate scaling and outlier rejection, filtering is based on a measure of risk defined in terms of distance to an approximate level set boundary defined by a One Class support vector machine. We report on experiments in three contexts: 1) a driving simulator with a 27,936 dimensional visual feature space, 2) a push-grasping in clutter simulation with a 22 dimensional state space, and 3) physical surgical needle insertion with a 16 dimensional state space. Results suggest that SHIV can efficiently learn policies with up to 70% fewer queries that DAgger. Michael Laskey, Samantha Staszak, Wesley Yu-Shu Hsieh, Jeffrey Mahler, Florian T. Pokorny, Anca D. Dragan, Kenneth Y. Goldberg |
ICRA | 7 |
| 2016 | Dex-Net 1.0: A cloud-based network of 3D objects for robust grasp planning using a Multi-Armed Bandit model with correlated rewardsabstractThis paper presents the Dexterity Network (Dex-Net) 1.0, a dataset of 3D object models and a sampling-based planning algorithm to explore how Cloud Robotics can be used for robust grasp planning. The algorithm uses a Multi- Armed Bandit model with correlated rewards to leverage prior grasps and 3D object models in a growing dataset that currently includes over 10,000 unique 3D object models and 2.5 million parallel-jaw grasps. Each grasp includes an estimate of the probability of force closure under uncertainty in object and gripper pose and friction. Dex-Net 1.0 uses Multi-View Convolutional Neural Networks (MV-CNNs), a new deep learning method for 3D object classification, to provide a similarity metric between objects, and the Google Cloud Platform to simultaneously run up to 1,500 virtual cores, reducing experiment runtime by up to three orders of magnitude. Experiments suggest that correlated bandit techniques can use a cloud-based network of object models to significantly reduce the number of samples required for robust grasp planning. We report on system sensitivity to variations in similarity metrics and in uncertainty in pose and friction. Code and updated information is available at http://berkeleyautomation.github.io/dex-net/. Jeffrey Mahler, Florian T. Pokorny, Brian Hou, Melrose Roderick, Michael Laskey, Mathieu Aubry, Kai Kohlhoff, Torsten Kröger, James J. Kuffner, Kenneth Y. Goldberg |
ICRA | 10 |
| 2016 | TSC-DL: Unsupervised trajectory segmentation of multi-modal surgical demonstrations with Deep LearningabstractThe growth of robot-assisted minimally invasive surgery has led to sizable datasets of fixed-camera video and kinematic recordings of surgical subtasks. Segmentation of these trajectories into locally-similar contiguous sections can facilitate learning from demonstrations, skill assessment, and salvaging good segments from otherwise inconsistent demonstrations. Manual, or supervised, segmentation can be prone to error and impractical for large datasets. We present Transition State Clustering with Deep Learning (TSC-DL), a new unsupervised algorithm that leverages video and kinematic data for task-level segmentation, and finds regions of the visual feature space that correlate with transition events using features constructed from layers of pre-trained image classification Deep Convolutional Neural Networks (CNNs). We report results on three datasets comparing Deep Learning architectures (AlexNet and VGG), choice of convolutional layer, dimensionality reduction techniques, visual encoding, and the use of Scale Invariant Feature Transforms (SIFT). We find that the deep architectures extract features that result in up-to a 30.4% improvement in Silhouette Score (a measure of cluster tightness) over the traditional “shallow” features from SIFT. We also present cases where TSC-DL discovers human annotator omissions. Supplementary material, data and code is available at: http://berkeleyautomation.github.io/tsc-dl/. Adithyavairavan Murali, Animesh Garg, Sanjay Krishnan, Florian T. Pokorny, Pieter Abbeel, Trevor Darrell, Kenneth Y. Goldberg |
ICRA | 7 |
| 2016 | Topological trajectory clustering with relative persistent homologyabstractCloud Robotics techniques based on Learning from Demonstrations suggest promising alternatives to manual programming of robots and autonomous vehicles. One challenge is that demonstrated trajectories may vary dramatically: it can be very difficult, if not impossible, for a system to learn control policies unless the trajectories are clustered into meaningful consistent subsets. Metric clustering methods, based on a distance measure, require quadratic time to compute a pairwise distance matrix and do not naturally distinguish topologically distinct trajectories. This paper presents an algorithm for topological clustering based on relative persistent homology, which, for a fixed underlying simplicial representation and discretization of trajectories, requires only linear time in the number of trajectories. The algorithm incorporates global constraints formalized in terms of the topology of sublevel or superlevel sets of a function and can be extended to incorporate probabilistic motion models. In experiments with real automobile and ship GPS trajectories as well as pedestrian trajectories extracted from video, the algorithm clusters trajectories into meaningful consistent subsets and, as we show in an experiment with ship trajectories, results in a faster and more efficient clustering than a metric clustering by Fréchet distance. Florian T. Pokorny, Kenneth Y. Goldberg, Danica Kragic |
ICRA | 2 |
| 2016 | High-dimensional Winding-Augmented Motion Planning with 2D topological task projections and persistent homologyabstractRecent progress in motion planning has made it possible to determine homotopy inequivalent trajectories between an initial and terminal configuration in a robot configuration space. Current approaches have however either assumed the knowledge of differential one-forms related to a skeletonization of the collision space, or have relied on a simplicial representation of the free space. Both of these approaches are currently however not yet practical for higher dimensional configuration spaces. We propose 2D topological task projections (TTPs): mappings from the configuration space to 2-dimensional spaces where simplicial complex filtrations and persistent homology can identify topological properties of the high-dimensional free configuration space. Our approach only requires the availability of collision free samples to identify winding centers that can be used to determine homotopy inequivalent trajectories. We propose the Winding Augmented RRT and RRT* (WA-RRT/RRT*) algorithms using which homotopy inequivalent trajectories can be found. We evaluate our approach in experiments with configuration spaces of planar linkages with 2-10 degrees of freedom. Results indicate that our approach can reliably identify suitable topological task projections and our proposed WA-RRT and WA-RRT* algorithms were able to identify a collection of homotopy inequivalent trajectories in each considered configuration space dimension. Florian T. Pokorny, Danica Kragic, Lydia E. Kavraki, Kenneth Y. Goldberg |
ICRA | 4 |
| 2016 | Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimizationabstractFor supervised automation of multi-throw suturing in Robot-Assisted Minimally Invasive Surgery, we present a novel mechanical needle guide and a framework for optimizing needle size, trajectory, and control parameters using sequential convex programming. The Suture Needle Angular Positioner (SNAP) results in a 3x error reduction in the needle pose estimate in comparison with the standard actuator. We evaluate the algorithm and SNAP on a da Vinci Research Kit using tissue phantoms and compare completion time with that of humans from the JIGSAWS dataset [5]. Initial results suggest that the dVRK can perform suturing at 30% of human speed while completing 86% suture throws attempted. Videos and data are available at: berkeleyautomation.github.io/amts. Siddarth Sen, Animesh Garg, David V. Gealy, Stephen McKinley, Yiming Jen, Kenneth Y. Goldberg |
ICRA | 6 |
| 2016 | Occlusion-aware multi-robot 3D trackingabstractWe introduce an optimization-based control approach that enables a team of robots to cooperatively track a target using onboard sensing. In this setting, the robots are required to estimate their own positions as well as concurrently track the target. Our probabilistic method generates controls that minimize the expected uncertainty of the target. Additionally, our method efficiently reasons about occlusions between robots and takes them into account for the control generation. We evaluate our approach in a number of experiments in which we simulate a team of quadrotor robots flying in three-dimensional space to track a moving target on the ground. We compare our method to other state-of-the-art approaches represented by the random sampling technique, lattice planning method, and our previous method. Our experimental results indicate that our method achieves up to 8 times smaller maximum tracking error and up to 2 times smaller average tracking error than the next best approach in the presented scenarios. Karol Hausman, Gregory Kahn, Sachin Patil, Jörg Müller 0004, Kenneth Y. Goldberg, Pieter Abbeel, Gaurav S. Sukhatme |
IROS | 5 |
| 2016 | ActiveClean: An Interactive Data Cleaning Framework For Modern Machine LearningabstractDatabases can be corrupted with various errors such as missing, incorrect, or inconsistent values. Increasingly, modern data analysis pipelines involve Machine Learning, and the effects of dirty data can be difficult to debug.Dirty data is often sparse, and naive sampling solutions are not suited for high-dimensional models. We propose ActiveClean, a progressive framework for training Machine Learning models with data cleaning. Our framework updates a model iteratively as the analyst cleans small batches of data, and includes numerous optimizations such as importance weighting and dirty data detection. We designed a visual interface to wrap around this framework and demonstrate ActiveClean for a video classification problem and a topic modeling problem. Sanjay Krishnan, Michael J. Franklin, Kenneth Y. Goldberg, Jiannan Wang 0001, Eugene Wu 0002 |
SIGMOD Conference | 3 |
| 2016 | PrivateClean: Data Cleaning and Differential PrivacyabstractRecent advances in differential privacy make it possible to guarantee user privacy while preserving the main characteristics of the data. However, most differential privacy mechanisms assume that the underlying dataset is clean. This paper explores the link between data cleaning and differential privacy in a framework we call PrivateClean. PrivateClean includes a technique for creating private datasets of numerical and discrete-valued attributes, a formalism for privacy-preserving data cleaning, and techniques for answering sum, count, and avg queries after cleaning. We show: (1) how the degree of privacy affects subsequent aggregate query accuracy, (2) how privacy potentially amplifies certain types of errors in a dataset, and (3) how this analysis can be used to tune the degree of privacy. The key insight is to maintain a bipartite graph relating dirty values to clean values and use this graph to estimate biases due to the interaction between cleaning and privacy. We validate these results on four datasets with a variety of well-studied cleaning techniques including using functional dependencies, outlier filtering, and resolving inconsistent attributes. Sanjay Krishnan, Jiannan Wang 0001, Michael J. Franklin, Kenneth Y. Goldberg, Tim Kraska |
SIGMOD Conference | 4 |
| 2016 | SWIRL: A SequentialWindowed Inverse Reinforcement Learning Algorithm for Robot Tasks With Delayed Rewards
Sanjay Krishnan, Animesh Garg, Richard Liaw, Brijen Thananjeyan, Lauren Miller, Florian T. Pokorny, Kenneth Y. Goldberg |
WAFR | 7 |
| 2016 | Synthesis of Energy-Bounded Planar Caging Grasps using Persistent Homology
Jeffrey Mahler, Florian T. Pokorny, Sherdil Niyaz, Kenneth Y. Goldberg |
WAFR | 4 |
| 2016 | ActiveClean: Interactive Data Cleaning For Statistical ModelingabstractAnalysts often clean dirty data iteratively--cleaning some data, executing the analysis, and then cleaning more data based on the results. We explore the iterative cleaning process in the context of statistical model training, which is an increasingly popular form of data analytics. We propose ActiveClean, which allows for progressive and iterative cleaning in statistical modeling problems while preserving convergence guarantees. ActiveClean supports an important class of models called convex loss models (e.g., linear regression and SVMs), and prioritizes cleaning those records likely to affect the results. We evaluate ActiveClean on five real-world datasets UCI Adult, UCI EEG, MNIST, IMDB, and Dollars For Docs with both real and synthetic errors. The results show that our proposed optimizations can improve model accuracy by up-to 2.5x for the same amount of data cleaned. Furthermore for a fixed cleaning budget and on all real dirty datasets, ActiveClean returns more accurate models than uniform sampling and Active Learning. Sanjay Krishnan, Jiannan Wang 0001, Eugene Wu 0002, Michael J. Franklin, Kenneth Y. Goldberg |
Proc. VLDB Endow. | 5 |
| 2016 | Editorial Fine-Tuning T-ASEabstractPresents the introductory editorial for this issue of the publication. Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 1 |
| 2016 | Editorial: "One Robot is Robotics, Ten Robots is Automation"abstractAutomation has come of age. As Raja Chatila aptly summarized in the quote above, automation addresses the challenges that arise when robots scale beyond proof-of-concept. Germany approved a $5B sale of Kuka to Midea Corp in China to provide robots for assembly automation. General Electric Corporation is now focusing on automation algorithms and data analytics and predicts this will be a $225 billion market within five years. These developments build on ongoing research in Big Data, Cloud Computing, Deep Learning, Open-Source Software, and Government/Industry initiatives, such as The “Internet of Things,” “Smarter Planet,” “Industrial Internet,” “Industrie 4.0,” and “Made in China 2025.” Automation is playing an increasingly central role in the global economy and in our daily lives. Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 1 |
| 2015 | The Slip-Pad: A haptic display using interleaved belts to simulate lateral and rotational slipabstractWe introduce a novel haptic display designed to reproduce the sensation of both lateral and rotational slip on a user's fingertip. The device simulates three-degrees-of-freedom of slip by actuating four interleaved tactile belts on which the user's finger rests. We present the specifications for the device, the mechanical design considerations, and initial evaluation experiments. We conducted experiments on user discrimination of tangential lateral and rotational slip. Initial results from our preliminary experiments suggest the device design has potential to simulate both tangential lateral and rotational slip. Source files: https://github.com/Slip-Pad. Colin Ho, Jonathan Kim, Sachin Patil, Kenneth Y. Goldberg |
World Haptics | 4 |
| 2015 | Active exploration using trajectory optimization for robotic grasping in the presence of occlusionsabstractWe consider the task of actively exploring unstructured environments to facilitate robotic grasping of occluded objects. Typically, the geometry and locations of these objects are not known a priori. We mount an RGB-D sensor on the robot gripper to maintain a 3D voxel map of the environment during exploration. The objective is to plan the motion of the sensor in order to search for feasible grasp handles that lie within occluded regions of the map. In contrast to prior work that generates exploration trajectories by sampling, we directly optimize the exploration trajectory to find grasp handles. Since it is challenging to optimize over the discrete voxel map, we encode the uncertainty of the positions of the occluded grasp handles as a mixture of Gaussians, one per occluded region. Our trajectory optimization approach encourages exploration by penalizing a measure of the uncertainty. We then plan a collision-free trajectory for the robot arm to the detected grasp handle. We evaluated our approach by actively exploring and attempting 300 grasps. Our experiments suggest that compared to the baseline method of sampling 10 trajectories, which successfully grasped 58% of the objects, our active exploration formulation with trajectory optimization successfully grasped 93% of the objects, was 1.3× faster, and had 3.2× fewer failed grasp attempts. Gregory Kahn, Peter Sujan, Sachin Patil, Shaunak Dattaprasad Bopardikar, Julian Ryde, Kenneth Y. Goldberg, Pieter Abbeel |
ICRA | 6 |
| 2015 | GP-GPIS-OPT: Grasp planning with shape uncertainty using Gaussian process implicit surfaces and Sequential Convex ProgrammingabstractComputing grasps for an object is challenging when the object geometry is not known precisely. In this paper, we explore the use of Gaussian process implicit surfaces (GPISs) to represent shape uncertainty from RGBD point cloud observations of objects. We study the use of GPIS representations to select grasps on previously unknown objects, measuring grasp quality by the probability of force closure. Our main contribution is GP-GPIS-OPT, an algorithm for computing grasps for parallel-jaw grippers on 2D GPIS object representations. Specifically, our method optimizes an approximation to the probability of force closure subject to antipodal constraints on the parallel jaws using Sequential Convex Programming (SCP). We also introduce GPIS-Blur, a method for visualizing 2D GPIS models based on blending shape samples from a GPIS. We test the algorithm on a set of 8 planar objects with transparency, translucency, and specularity. Our experiments suggest that GP-GPIS-OPT computes grasps with higher probability of force closure than a planner that does not consider shape uncertainty on our test objects and may converge to a grasp plan up to 5.7×faster than using Monte-Carlo integration, a common method for grasp planning under shape uncertainty. Furthermore, initial experiments on the Willow Garage PR2 robot suggest that grasps selected with GP-GPIS-OPT are up to 90% more successful than those planned assuming a deterministic shape. Our dataset, code, and videos of our experiments are available at http://rll.berkeley.edu/icra2015grasping/. Jeffrey Mahler, Sachin Patil, Ben Kehoe, Jur P. van den Berg, Matei T. Ciocarlie, Pieter Abbeel, Kenneth Y. Goldberg |
ICRA | 7 |
| 2015 | Learning by observation for surgical subtasks: Multilateral cutting of 3D viscoelastic and 2D Orthotropic Tissue PhantomsabstractAutomating repetitive surgical subtasks such as suturing, cutting and debridement can reduce surgeon fatigue and procedure times and facilitate supervised tele-surgery. Programming is difficult because human tissue is deformable and highly specular. Using the da Vinci Research Kit (DVRK) robotic surgical assistant, we explore a “Learning By Observation” (LBO) approach where we identify, segment, and parameterize motion sequences and sensor conditions to build a finite state machine (FSM) for each subtask. The robot then executes the FSM repeatedly to tune parameters and if necessary update the FSM structure. We evaluate the approach on two surgical subtasks: debridement of 3D Viscoelastic Tissue Phantoms (3d-DVTP), in which small target fragments are removed from a 3D viscoelastic tissue phantom; and Pattern Cutting of 2D Orthotropic Tissue Phantoms (2d-PCOTP), a step in the standard Fundamentals of Laparoscopic Surgery training suite, in which a specified circular area must be cut from a sheet of orthotropic tissue phantom. We describe the approach and physical experiments with repeatability of 96% for 50 trials of the 3d-DVTP subtask and 70% for 20 trials of the 2d-PCOTP subtask. A video is available at: http://j.mp/Robot-Surgery-Video-Oct-2014. Adithyavairavan Murali, Siddarth Sen, Ben Kehoe, Animesh Garg, Seth McFarland, Sachin Patil, W. Douglas Boyd, Susan Lim, Pieter Abbeel, Kenneth Y. Goldberg |
ICRA | 10 |
| 2015 | Models of human-centered automation in a debridement taskabstractIn robot-assisted surgery, manipulation tasks can be achieved through collaboration among robotic and human agents. Collaboration models can potentially include multiple agents working towards a shared objective - a scenario referred to as multilateral manipulation. In this work, we examine multilateral manipulation in the task of debridement: removing dead tissue or damaged tissue fragments to allow remaining healthy tissue to heal. We extended our previously developed multilateral manipulation software framework to the task of debridement and implemented four different collaboration models: (1) fully autonomous debridement, (2) shared control between a human and robotic agent, (3) supervised control where the operator identifies foreign bodies to be excised, and (4) teleoperation. We demonstrate these collaboration models with the RAVEN-II robot, an open-architecture surgical robot with two cable-driven 7-DOF arms. Each collaboration model included methods to remove foreign bodies from the field of view of a stereoscopic camera setup. Results indicate tradeoffs in experiment duration, hardware requirements, and safety implications among the four collaboration models. Kirk A. Nichols, Adithyavairavan Murali, Siddarth Sen, Kenneth Y. Goldberg, Allison M. Okamura |
IROS | 4 |
| 2015 | A paced shared-control teleoperated architecture for supervised automation of multilateral surgical tasksabstractAutomation of repetitive tasks can improve laparoscopic surgical procedures by unloading surgeons and reducing duration, trauma, and expense. However, surgical procedures involve delicate manipulation of deformable tissues in a very dynamic environment, suggesting that automated execution of surgical tasks should be carried out under the supervision of the surgeon. We propose a teleoperated architecture that allows a surgeon to employ and supervise agents that can autonomously perform or assist with surgical tasks. The architecture is independent of the automation method. It includes a dominance factor that allows the surgeon to take control over the slave robot at any time, and an aggressiveness factor that sets the performance pace of the autonomous agent. We tested the architecture during execution of a multilateral tension-and-cut task, where a human operator and an autonomous agent are responsible for tensioning or cutting of a tissue. The architecture allowed for supervised and paced automation of the task. We found that collaboration of the human operator and autonomous agent can lead to shorter completion time compared to performance of only a human. Kamran Shamaei, Yuhang Che, Adithyavairavan Murali, Siddarth Sen, Sachin Patil, Kenneth Y. Goldberg, Allison M. Okamura |
IROS | 6 |
| 2015 | Transition State Clustering: Unsupervised Surgical Trajectory Segmentation for Robot Learning
Sanjay Krishnan, Animesh Garg, Sachin Patil, Colin Lea, Gregory D. Hager, Pieter Abbeel, Kenneth Y. Goldberg |
ISRR (2) | 7 |
| 2015 | M-CAFE: Managing MOOC Student Feedback with Collaborative FilteringabstractOngoing student feedback on course content and assignments can be valuable for MOOC instructors in the absence of face-to-face-interaction. To collect ongoing feedback and scalably identify valuable suggestions, we built the MOOC Collaborative Assessment and Feedback Engine (M-CAFE). This mobile platform allows MOOC students to numerically assess the course, their own performance, and provide textual suggestions about how the course could be improved on a weekly basis. M-CAFE allows students to visualize how they compare with their peers and read and evaluate what others have suggested, providing peer-to-peer collaborative filtering. We evaluate M-CAFE based on data from two EdX MOOCs. Alison Cliff, Allen Huang, Sanjay Krishnan, Brandie Nonnecke, Kanji Uchino, Samuel Joseph, Armando Fox, Kenneth Y. Goldberg |
L@S | 9 |
| 2015 | Stale View Cleaning: Getting Fresh Answers from Stale Materialized ViewsabstractMaterialized views (MVs), stored pre-computed results, are widely used to facilitate fast queries on large datasets. When new records arrive at a high rate, it is infeasible to continuously update (maintain) MVs and a common solution is to defer maintenance by batching updates together. Between batches the MVs become increasingly stale with incorrect, missing, and superfluous rows leading to increasingly inaccurate query results. We propose Stale View Cleaning (SVC) which addresses this problem from a data cleaning perspective. In SVC, we efficiently clean a sample of rows from a stale MV, and use the clean sample to estimate aggregate query results. While approximate, the estimated query results reflect the most recent data. As sampling can be sensitive to long-tailed distributions, we further explore an outlier indexing technique to give increased accuracy when the data distributions are skewed. SVC complements existing deferred maintenance approaches by giving accurate and bounded query answers between maintenance. We evaluate our method on a generated dataset from the TPC-D benchmark and a real video distribution application. Experiments confirm our theoretical results: (1) cleaning an MV sample is more efficient than full view maintenance, (2) the estimated results are more accurate than using the stale MV, and (3) SVC is applicable for a wide variety of MVs. Sanjay Krishnan, Jiannan Wang 0001, Michael J. Franklin, Kenneth Y. Goldberg, Tim Kraska |
Proc. VLDB Endow. | 4 |
| 2015 | Efficient Proximity Probing Algorithms for MetrologyabstractMetrology, the theoretical and practical study of measurement, has applications in automated manufacturing, inspection, robotics, surveying, and healthcare. The geometric probing problem considers how to optimally use a probe to measure geometric properties. In this paper, we consider a proximity probe which, given a point, returns the distance to the boundary of the nearest object. When there is an unknown convex polygon P in the plane, the goal is to minimize the number of probe measurement needed to exactly determine the shape and location of P. We present an algorithm with upper bound of 3.5n + k + 2 probes, where n is the number of vertices and k ≤ 3 is the number of acute angles of P. The algorithm requires constant time per probe, and hence, O(n) time to determine P. We also address the related problem where the unknown polygon is a member of a known finite set Γ and the goal is to efficiently determine which polygon is present. When m is the size of Γ and n' is the maximum number of vertices of any member of Γ, we present an algorithm with an upper bound of 2n + 2 probes with O(1) computations per probe and a O(n'm) preprocessing phase (depending only on Γ). Aviv Adler, Fatemeh Panahi, A. Frank van der Stappen, Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 4 |
| 2015 | Editorial Multiplicity Has More Potential Than SingularityabstractPresents an editorial on the developing concept in artificial intelligence referred to as "muliplicity." The term “Singularity” is used to describe a hypothetical punctuation point sometime in the future where Artificial Intelligence (AI) will surpass human intelligence. The concept has been popularized by science fiction author Vernor Vinge and Ray Kurzweil. “Superintelligence,” a 2014 book by Nick Bostrom, explores a similar theme. Recently, Stephen Hawking, Elon Musk, and Bill Gates publicly expressed fears about the dangers of post-Singularity AI and robotics. Here, it is suggested that "Singularity" is distracting attention from a far more realistic and important development that we might call “Multiplicity.” Multiplicity characterizes an emerging category of systems where diverse groups of humans work together with diverse groups of machines to solve difficult problems. Multiplicity combines the wisdom of crowds with the power of cloud computing. Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 1 |
| 2015 | A Survey of Research on Cloud Robotics and AutomationabstractThe Cloud infrastructure and its extensive set of Internet-accessible resources has potential to provide significant benefits to robots and automation systems. We consider robots and automation systems that rely on data or code from a network to support their operation, i.e., where not all sensing, computation, and memory is integrated into a standalone system. This survey is organized around four potential benefits of the Cloud: 1) Big Data: access to libraries of images, maps, trajectories, and descriptive data; 2) Cloud Computing: access to parallel grid computing on demand for statistical analysis, learning, and motion planning; 3) Collective Robot Learning: robots sharing trajectories, control policies, and outcomes; and 4) Human Computation: use of crowdsourcing to tap human skills for analyzing images and video, classification, learning, and error recovery. The Cloud can also improve robots and automation systems by providing access to: a) datasets, publications, models, benchmarks, and simulation tools; b) open competitions for designs and systems; and c) open-source software. This survey includes over 150 references on results and open challenges. A website with new developments and updates is available at: http://goldberg.berkeley.edu/cloud-robotics/. Ben Kehoe, Sachin Patil, Pieter Abbeel, Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 4 |
| 2015 | Cloud-Based Grasp Analysis and Planning for Toleranced Parts Using Parallelized Monte Carlo SamplingabstractThis paper considers grasp planning in the presence of shape uncertainty and explores how cloud computing can facilitate parallel Monte Carlo sampling of combination actions and shape perturbations to estimate a lower bound on the probability of achieving force closure. We focus on parallel-jaw push grasping for the class of parts that can be modeled as extruded 2-D polygons with statistical tolerancing. We describe an extension to model part slip and experimental results with an adaptive sampling algorithm that can reduce sample size by 90%. We show how the algorithm can also bound part tolerance for a given grasp quality level and report a sensitivity analysis on algorithm parameters. We test a cloud-based implementation with varying numbers of nodes, obtaining a 515 × speedup with 500 nodes in one case, suggesting the algorithm can scale linearly when all nodes are reliable. Code and data are available at: http://automation.berkeley.edu/cloud-based-grasping. Ben Kehoe, Deepak Warrier, Sachin Patil, Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 4 |
| 2015 | Planning Curvature and Torsion Constrained Ribbons in 3D With Application to Intracavitary BrachytherapyabstractWe present an approach for planning ensembles of channels, ribbons, within 3D printed implants for facilitating radiation therapy treatment of cancer. The ribbons are traced out by sweeping a constant width rigid body (cuboid) along spatial curves. We propose a method for planning multiple disjoint and mutually collision-free ribbons of finite thickness along curvature and torsion constrained curves in 3D space. This is equivalent to planning motions for the cross section of the ribbon along a spatial curve such that the cross section is oriented along the unit binormal to the curve defined according to the Frenet-Serret frame. We propose a two-stage planning approach. In the first stage, a customized sampling-based planner uses rapidly exploring random trees (RRTs) to generate feasible curvature and torsion constrained ribbons. In the second stage, the curvature and torsion along each ribbon is locally optimized using sequential quadratic programming (SQP). We use this approach to design curved radiation delivery channels inside custom 3D printed implants that allow temporary insertion of a high-dose radioactive source that is threaded through the channels using a wire and allowed to dwell for specified times to expose cancerous tumors for intracavitary brachytherapy treatment. Constraints on the curvature and torsion are required for 3D printing (to allow flushing of sacrificial material) and for smooth insertion of radioactive sources. In simulation experiments, this approach achieves an improvement of 46% in tumor coverage compared with a greedy approach that generates channels sequentially. Sachin Patil, Jia Pan 0001, Pieter Abbeel, Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 4 |
| 2015 | Two Large Open-Access Datasets for Fitts' Law of Human Motion and a Succinct Derivation of the Square-Root VariantabstractFitts' law specifies a logarithmic relationship between motion duration and the ratio of target distance over target size. This paper introduces two large open-access datasets from experimental user studies: first a controlled (in-lab) study with 46 participants, and second an uncontrolled online study using a Java applet. We present a succinct derivation of the square-root variant of Fitts' law using optimal control theory and compare three models that relate motion duration to the ratio of target distance over target size: LOG (Fitts' original logarithmic function), SQR (square-root), and LOG' (McKenzie's logarithmic plus 1.0). We find that: 1) the data from the controlled and uncontrolled studies are consistent; 2) for homogeneous targets (with fixed size and distance), the SQR model yields a significantly better fit than LOG or LOG', except with the most difficult targets (where the ratio of target distance over target size is large) where the models are not significantly different; and 3) for heterogeneous targets (with varying size and distance), SQR yields a significantly better fit than LOG for easy targets and LOG yields a significantly better fit for targets of medium difficulty, while the LOG' model yields a significantly better fit than both LOG and SQR on very difficult targets. The anonymized datasets including 94 580 human reaching motion timing measurements are, to our knowledge, the largest collected to date. Kenneth Y. Goldberg, Siamak Faridani, Ron Alterovitz |
IEEE Trans. Hum. Mach. Syst. | 1 |
| 2014 | Planning locally optimal, curvature-constrained trajectories in 3D using sequential convex optimizationabstract3D curvature-constrained motion planning finds applications in a wide variety of domains, including motion planning for flexible, bevel-tip medical needles, planning curvature-constrained channels in 3D printed implants for targeted brachytherapy dose delivery or channels for cooling turbine blades, and path planning for unmanned aerial vehicles (UAVs). In this work, we present a motion planning technique using sequential convex optimization for computing locally optimal, curvature-constrained trajectories to desired targets while avoiding obstacles in 3D environments. We report two main contributions in this work: (i) curvature-constrained trajectory optimization in 6D pose (position and orientation) space, and (ii) planning multiple trajectories that are mutually collision-free. We demonstrate the performance of our approach on two clinically motivated applications. Our experiments indicate that our approach can compute high-quality plans for medical needle steering in 1.6 seconds on a commodity PC, enabling re-planning during execution to correct for perturbations. Our approach can also be used for designing optimized channel layouts within 3D printed implants for intracavitary brachytherapy. Yan Duan, Sachin Patil, John Schulman, Kenneth Y. Goldberg, Pieter Abbeel |
ICRA | 4 |
| 2014 | Autonomous multilateral debridement with the Raven surgical robotabstractAutonomous robot execution of surgical sub-tasks has the potential to reduce surgeon fatigue and facilitate supervised tele-surgery. This paper considers the sub-task of surgical debridement: removing dead or damaged tissue fragments to allow the remaining healthy tissue to heal. We present an autonomous multilateral surgical debridement system using the Raven, an open-architecture surgical robot with two cable-driven 7 DOF arms. Our system combines stereo vision for 3D perception with trajopt, an optimization-based motion planner, and model predictive control (MPC). Laboratory experiments involving sensing, grasping, and removal of 120 fragments suggest that an autonomous surgical robot can achieve robustness comparable to human performance. Our robot system demonstrated the advantage of multilateral systems, as the autonomous execution was 1.5× faster with two arms than with one; however, it was two to three times slower than a human. Execution speed could be improved with better state estimation that would allow more travel between MPC steps and fewer MPC replanning cycles. The three primary contributions of this paper are: (1) introducing debridement as a sub-task of interest for surgical robotics, (2) demonstrating the first reliable autonomous robot performance of a surgical sub-task using the Raven, and (3) reporting experiments that highlight the importance of accurate state estimation for future research. Further information including code, photos, and video is available at: http://rll.berkeley.edu/raven. Ben Kehoe, Gregory Kahn, Jeffrey Mahler, Jonathan Kim, Alex X. Lee, Anna Lee, Keisuke Nakagawa, Sachin Patil, W. Douglas Boyd, Pieter Abbeel, Kenneth Y. Goldberg |
ICRA | 11 |
| 2014 | Gaussian belief space planning with discontinuities in sensing domainsabstractDiscontinuities in sensing domains are common when planning for many robotic navigation and manipulation tasks. For cameras and 3D sensors, discontinuities may be inherent in sensor field of view or may change over time due to occlusions that are created by moving obstructions and movements of the sensor. The associated gaps in sensor information due to missing measurements pose a challenge for belief space and related optimization-based planning methods since there is no gradient information when the system state is outside the sensing domain. We address this in a belief space context by considering the signed distance to the sensing region. We smooth out sensing discontinuities by assuming that measurements can be obtained outside the sensing region with noise levels depending on a sigmoid function of the signed distance. We sequentially improve the continuous approximation by increasing the sigmoid slope over an outer loop to find plans that cope with sensor discontinuities. We also incorporate the information contained in not obtaining a measurement about the state during execution by appropriately truncating the Gaussian belief state. We present results in simulation for tasks with uncertainty involving navigation of mobile robots and reaching tasks with planar robot arms. Experiments suggest that the approach can be used to cope with discontinuities in sensing domains by effectively re-planning during execution. Sachin Patil, Yan Duan, John Schulman, Kenneth Y. Goldberg, Pieter Abbeel |
ICRA | 4 |
| 2014 | A methodology for learning, analyzing, and mitigating social influence bias in recommender systemsabstractThe seminal 2003 paper by Cosley, Lab, Albert, Konstan, and Reidl, demonstrated the susceptibility of recommender systems to rating biases. To facilitate browsing and selection, almost all recommender systems display average ratings before accepting ratings from users which has been shown to bias ratings. This effect is called Social Inuence Bias (SIB); the tendency to conform to the perceived \norm" in a community. We propose a methodology to 1) learn, 2) analyze, and 3) mitigate the effect of SIB in recommender systems. In the Learning phase, we build a baseline dataset by allowing users to rate twice: before and after seeing the average rating. In the Analysis phase, we apply a new non-parametric significance test based on the Wilcoxon statistic to test whether the data is consistent with SIB. If significant, we propose a Mitigation phase using polynomial regression and the Bayesian Information Criterion (BIC) to predict unbiased ratings. We evaluate our approach on a dataset of 9390 ratings from the California Report Card (CRC), a rating-based system designed to encourage political engagement. We found statistically significant evidence of SIB. Mitigating models were able to predict changed ratings with a normalized RMSE of 12.8% and reduce bias by 76.3%. The CRC, our data, and experimental code are available at: http://californiareportcard.org/data/ Sanjay Krishnan, Jay Patel 0002, Michael J. Franklin, Kenneth Y. Goldberg |
RecSys | 4 |
| 2014 | A sample-and-clean framework for fast and accurate query processing on dirty dataabstractIn emerging Big Data scenarios, obtaining timely, high-quality answers to aggregate queries is difficult due to the challenges of processing and cleaning large, dirty data sets. To increase the speed of query processing, there has been a resurgence of interest in sampling-based approximate query processing (SAQP). In its usual formulation, however, SAQP does not address data cleaning at all, and in fact, exacerbates answer quality problems by introducing sampling error. In this paper, we explore an intriguing opportunity. That is, we explore the use of sampling to actually improve answer quality. We introduce the Sample-and-Clean framework, which applies data cleaning to a relatively small subset of the data and uses the results of the cleaning process to lessen the impact of dirty data on aggregate query answers. We derive confidence intervals as a function of sample size and show how our approach addresses error bias. We evaluate the Sample-and-Clean framework using data from three sources: the TPC-H benchmark with synthetic noise, a subset of the Microsoft academic citation index and a sensor data set. Our results are consistent with the theoretical confidence intervals and suggest that the Sample-and-Clean framework can produce significant improvements in accuracy compared to query processing without data cleaning and speed compared to data cleaning without sampling. Jiannan Wang 0001, Sanjay Krishnan, Michael J. Franklin, Kenneth Y. Goldberg, Tim Kraska, Tova Milo |
SIGMOD Conference | 4 |
| 2014 | Scaling up Gaussian Belief Space Planning Through Covariance-Free Trajectory Optimization and Automatic Differentiation
Sachin Patil, Gregory Kahn, Michael Laskey, John Schulman, Kenneth Y. Goldberg, Pieter Abbeel |
WAFR | 5 |
| 2014 | Planning Curvature and Torsion Constrained Ribbons in 3D with Application to Intracavitary Brachytherapy
Sachin Patil, Jia Pan 0001, Pieter Abbeel, Kenneth Y. Goldberg |
WAFR | 4 |
| 2014 | Potential-based bounded-cost search and Anytime Non-Parametric A*
Roni Stern, Ariel Felner, Jur P. van den Berg, Rami Puzis, Rajat Shah, Kenneth Y. Goldberg |
Artif. Intell. | 6 |
| 2013 | Putting the turing into manufacturing: recent developments in algorithmic automationabstractAs global labor costs increase and product life cycles decrease, there is renewed interest in research in automated manufacturing systems that can be reliably and rapidly configured. Inspired by Turing's abstractions for computing, Algorithmic Automation explores mathematical abstractions and algorithms that allow the functionality of assembly lines and manufacturing automation systems to be designed independent of their underlying implementations. Abstractions based on minimal sets of geometric primitives can provide the foundation for formal specification, analysis, design, optimization, and verification. Algorithmic Automation is characterized by: (1) formal specification of sets of admissible inputs (eg, polyhedra) and operations (eg, parallel-jaw grasps), (2) complete algorithms that compute all solutions or terminate with a report that no solution exists, and (3) bounds on complexity as a function of input size. This extended abstract summarizes selected results and open problems. Kenneth Y. Goldberg |
SoCG | 1 |
| 2013 | Cloud-based robot grasping with the google object recognition engineabstractRapidly expanding internet resources and wireless networking have potential to liberate robots and automation systems from limited onboard computation, memory, and software. “Cloud Robotics” describes an approach that recognizes the wide availability of networking and incorporates open-source elements to greatly extend earlier concepts of “Online Robots” and “Networked Robots”. In this paper we consider how cloud-based data and computation can facilitate 3D robot grasping. We present a system architecture, implemented prototype, and initial experimental data for a cloud-based robot grasping system that incorporates a Willow Garage PR2 robot with onboard color and depth cameras, Google's proprietary object recognition engine, the Point Cloud Library (PCL) for pose estimation, Columbia University's GraspIt! toolkit and OpenRAVE for 3D grasping and our prior approach to sampling-based grasp analysis to address uncertainty in pose. We report data from experiments in recognition (a recall rate of 80% for the objects in our test set), pose estimation (failure rate under 14%), and grasping (failure rate under 23%) and initial results on recall and false positives in larger data sets using confidence measures. Ben Kehoe, Akihiro Matsukawa, Sal Candido, James J. Kuffner, Kenneth Y. Goldberg |
ICRA | 5 |
| 2013 | Sigma hulls for Gaussian belief space planning for imprecise articulated robots amid obstaclesabstractIn many home and service applications, an emerging class of articulated robots such as the Raven and Baxter trade off precision in actuation and sensing to reduce costs and to reduce the potential for injury to humans in their workspaces. For planning and control of such robots, planning in belief ssigma hullpace, i.e., modeling such problems as POMDPs, has shown great promise but existing belief space planning methods have primarily been applied to cases where robots can be approximated as points or spheres. In this paper, we extend the belief space framework to treat articulated robots where the linkage can be decomposed into convex components. To allow planning and collision avoidance in Gaussian belief spaces, we introduce the concept of sigma hulls: convex hulls of robot links transformed according to the sigma standard deviation boundary points generated by the Unscented Kalman filter (UKF). We characterize the signed distances between sigma hulls and obstacles in the workspace to formulate efficient collision avoidance constraints compatible with the Gilbert-Johnson-Keerthi (GKJ) and Expanding Polytope Algorithms (EPA) within an optimization-based planning framework. We report results in simulation for planning motions for a 4-DOF planar robot and a 7-DOF articulated robot with imprecise actuation and inaccurate sensors. These experiments suggest that the sigma hull framework can significantly reduce the probability of collision and is computationally efficient enough to permit iterative re-planning for model predictive control. Alex X. Lee, Yan Duan, Sachin Patil, John Schulman, Zoe McCarthy, Jur P. van den Berg, Kenneth Y. Goldberg, Pieter Abbeel |
IROS | 7 |
| 2013 | Robot-Guided Open-Loop Insertion of Skew-Line Needle Arrangements for High Dose Rate BrachytherapyabstractWe present a study in human-centered automation that has potential to reduce patient side effects from high dose rate brachytherapy (HDR-BT). To efficiently deliver radiation to the prostate while minimizing trauma to sensitive structures such as the penile bulb, we modified the Acubot-RND 7-axis robot to guide insertion of diamond-tip needles into desired skew-line geometric arrangements. We extend and integrate two algorithms: Needle Planning with Integer Programming (NPIP) and Inverse Planning with Integer Programming (IPIP) to compute skew-line needle and dose plans. We performed three physical experiments with anatomically correct phantom models to study performance: two with the robot and one control experiment with an expert human physician (coauthor Hsu) without the robot. All were able to achieve needle arrangements that meet the RTOG-0321 clinical dose objectives with zero trauma to the penile bulb. We analyze systematic and random errors in needle placement; total RMS error for the robot system operating without feedback ranged from 2.6 to 4.3 mm, which is comparable to the RMS error of 2.7 mm obtained in an earlier study for PPI-BT treatment using a robot with 3D ultrasound feedback. Animesh Garg, Timmy Siauw, Dmitry Berenson, J. Adam M. Cunha, I-Chow Hsu, Jean Pouliot, Dan Stoianovici, Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 8 |
| 2013 | In Memoriam Richard A. (Dick) VolzabstractA brief curriculum vitae for Richard A. (Dick) Volz is provided highlighting his professional achievements. As IEEE Robotics and Automation Society Vice-President of Publications Dick Volz played a fundamental role in the conception of the IEEE Transactions on Automation Science and Engineering. Mr. Volz convened and worked closely with an international Steering Committee that stepped back to consider the past and future trajectory of research in Robotics and Automation, concluding that a new publication devoted to Automation was essential to expand our community and scope. He will be remembered as a co-founder of this publication. Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 1 |
| 2012 | A robot path planning framework that learns from experienceabstractWe propose a framework, called Lightning, for planning paths in high-dimensional spaces that is able to learn from experience, with the aim of reducing computation time. This framework is intended for manipulation tasks that arise in applications ranging from domestic assistance to robot-assisted surgery. Our framework consists of two main modules, which run in parallel: a planning-from-scratch module, and a module that retrieves and repairs paths stored in a path library. After a path is generated for a new query, a library manager decides whether to store the path based on computation time and the generated path's similarity to the retrieved path. To retrieve an appropriate path from the library we use two heuristics that exploit two key aspects of the problem: (i) A correlation between the amount a path violates constraints and the amount of time needed to repair that path, and (ii) the implicit division of constraints into those that vary across environments in which the robot operates and those that do not. We evaluated an implementation of the framework on several tasks for the PR2 mobile manipulator and a minimally-invasive surgery robot in simulation. We found that the retrieve-and-repair module produced paths faster than planning-from-scratch in over 90% of test cases for the PR2 and in 58% of test cases for the minimally-invasive surgery robot. Dmitry Berenson, Pieter Abbeel, Kenneth Y. Goldberg |
ICRA | 3 |
| 2012 | Toward cloud-based grasping with uncertainty in shape: Estimating lower bounds on achieving force closure with zero-slip push graspsabstractThis paper explores how Cloud Computing can facilitate grasping with shape uncertainty. We consider the most common robot gripper: a pair of thin parallel jaws, and a class of objects that can be modeled as extruded polygons. We model a conservative class of push-grasps that can enhance object alignment. The grasp planning algorithm takes as input an approximate object outline and Gaussian uncertainty around each vertex and center of mass. We define a grasp quality metric based on a lower bound on the probability of achieving force closure. We present a highly-parallelizable algorithm to compute this metric using Monte Carlo sampling. The algorithm uses Coulomb frictional grasp mechanics and a fast geometric test for conservative conditions for force closure. We run the algorithm on a set of sample shapes and compare the grasps with those from a planner that does not model shape uncertainty. We report computation times with single and multi-core computers and sensitivity analysis on algorithm parameters. We also describe physical grasp experiments using the Willow Garage PR2 robot. Ben Kehoe, Dmitry Berenson, Kenneth Y. Goldberg |
ICRA | 3 |
| 2012 | What Is Automation?abstractIt is argued that automation has expanded beyond its roots in Manufacturing to include applications in Healthcare, Security, Transportation, Agriculture, Construction, Energy, and many other areas. Both Robotics and Automation explore the frontiers of automated and semi-automated machines. Both fields are increasingly concerned with the role of humans and human interfaces, and with the potential of the Internet and Cloud Computing. So what is the difference between Robotics and Automation? There are many possible distinctions. Here is the summary from our Society's Field of Interest Statement: "...Robotics focuses on systems incorporating sensors and actuators that operate autonomously or semi-autonomously in cooperation with humans. Robotics research emphasizes intelligence and adaptability to cope with unstructured environments. Automation research emphasizes efficiency, productivity, quality, and reliability, focusing on systems that operate autonomously, often in structured environments over extended periods, and on the explicit structuring of such environments." This statement emphasizes how Automation emphasizes structured versus unstructured environments, reliability versus adaptability, and efficiency versus exploratory operations. These are valuable distinctions and the author would like to propose another one. In his view, research in Robotics emphasizes Feasibility. Feasibility focuses on proof-of-concept, demonstrating how a new functionality can be achieved. On the other hand, he feels, research in Automation emphasizes Quality. The author wishes to dispel the myth of the excluded middle: Robotics and Automation are not disjoint. Feasibility and Quality are closely related. Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 1 |
| 2012 | Editorial Tracking T-ASE Inventory by Methodology and Application
Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 1 |
| 2012 | Editorial [intro. to Guest Editorial by Prof. Raff D'Andrea]abstractSummary form only given, as follows. The Editor-in-Chief presents a Guest Editorial by Prof. Raff D'Andrea of ETH Zurich. Dr. D'Andrea played a key role in designing and developing the innovative technology behind Kiva Systems, which was acquired by Amazon.com in May 2012 for US$775M. Kiva Systems invented a radical new approach to warehouse automation and logistics based on large numbers of small mobile platforms that move between stations for order fulfillment, where human workers perform recognition, grasping, loading, and unloading of items from the platforms. The systems required many advances in design and engineering to achieve the necessary Quality (with a capital Q: reliability, efficiency, precision) for deployment in major warehouses (if you haven't seen them, there are cool videos showing scores of bright orange platforms in action on the Internet). In the Guest Editorial, Raff summarizes key insights, such as using feedback from low-cost sensing and computation, combined with data analytics and machine learning, to achieve high accuracy and precision from inexpensive mechanical components. Raff also proposes several exciting challenges for future research, such as new algorithms for high-level planning and scheduling that can optimize the number of mobile platforms, and a call for a reliable indoor equivalent to GPS. He argues for new system design tools that apply innovations in formal verification and model uncertainty and information flow. Raff closes with an exciting and characteristically bold Grand Challenge for Robotics and Automation. Kiva Systems is an inspiring success story in our field and Raff is the ideal person to describe it. Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 1 |
| 2011 | Anytime Nonparametric AabstractAnytime variants of Dijkstra's and A* shortest path algorithms quickly produce a suboptimal solution and then improve it over time. For example, ARA* introduces a weighting value "epsilon" to rapidly find an initial suboptimal path and then reduces "epsilon" to improve path quality over time. In ARA*, "epsilon" is based on a linear trajectory with ad-hoc parameters chosen by each user. We propose a new Anytime A* algorithm, Anytime Nonparametric A* (ANA*), that does not require ad-hoc parameters, and adaptively reduces varepsilon to expand the most promising node per iteration, adapting the greediness of the search as path quality improves. We prove that each node expanded by ANA* provides an upper bound on the suboptimality of the current-best solution. We evaluate the performance of ANA* with experiments in the domains of robot motion planning, gridworld planning, and multiple sequence alignment. The results suggest that ANA* is as efficient as ARA* and in most cases: (1) ANA* finds an initial solution faster, (2) ANA* spends less time between solution improvements, (3) ANA* decreases the suboptimality bound of the current-best solution more gradually, and (4) ANA* finds the optimal solution faster. ANA* is freely available from Maxim Likhachev's Search-based Planning Library (SBPL). Jur P. van den Berg, Rajat Shah, Arthur Huang, Kenneth Y. Goldberg |
AAAI | 4 |
| 2011 | EG-RRT: Environment-guided random trees for kinodynamic motion planning with uncertainty and obstaclesabstractExisting sampling-based robot motion planning methods are often inefficient at finding trajectories for kinodynamic systems, especially in the presence of narrow passages between obstacles and uncertainty in control and sensing. To address this, we propose EG-RRT, an Environment-Guided variant of RRT designed for kinodynamic robot systems that combines elements from several prior approaches and may incorporate a cost model based on the LQG-MP framework to estimate the probability of collision under uncertainty in control and sensing. We compare the performance of EG-RRT with several prior approaches on challenging sample problems. Results suggest that EG-RRT offers significant improvements in performance. Léonard Jaillet, Judy Hoffman, Jur P. van den Berg, Pieter Abbeel, Josep M. Porta, Kenneth Y. Goldberg |
IROS | 6 |
| 2011 | Grasping and Fixturing as Submodular Coverage Problems
John Schulman, Kenneth Y. Goldberg, Pieter Abbeel |
ISRR | 2 |
| 2010 | Opinion space: a scalable tool for browsing online commentsabstractInternet users are increasingly inclined to contribute comments to online news articles, videos, product reviews, and blogs. The most common interface for comments is a list, sorted by time of entry or by binary ratings. It is widely recognized that such lists do not scale well and can lead to "cyberpolarization," which serves to reinforce extreme opinions. We present Opinion Space: a new online interface incorporating ideas from deliberative polling, dimensionality reduction, and collaborative filtering that allows participants to visualize and navigate through a diversity of comments. This self-organizing system automatically highlights the comments found most insightful by users from a range of perspectives. We report results of a controlled user study. When Opinion Space was compared with a chronological List interface, participants read a similar diversity of comments. However, they were significantly more engaged with the system, and they had significantly higher agreement with and respect for the comments they read. Siamak Faridani, Ephrat Bitton, Kimiko Ryokai, Kenneth Y. Goldberg |
CHI | 4 |
| 2010 | Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrationsabstractIn the future, robotic surgical assistants may assist surgeons by performing specific subtasks such as retraction and suturing to reduce surgeon tedium and reduce the duration of some operations. We propose an apprenticeship learning approach that has potential to allow robotic surgical assistants to autonomously execute specific trajectories with superhuman performance in terms of speed and smoothness. In the first step, we record a set of trajectories using human-guided backdriven motions of the robot. These are then analyzed to extract a smooth reference trajectory, which we execute at gradually increasing speeds using a variant of iterative learning control. We evaluate this approach on two representative tasks using the Berkeley Surgical Robots: a figure eight trajectory and a two handed knot-tie, a tedious suturing sub-task required in many surgical procedures. Results suggest that the approach enables (i) rapid learning of trajectories, (ii) smoother trajectories than the human-guided trajectories, and (iii) trajectories that are 7 to 10 times faster than the best human-guided trajectories. Jur P. van den Berg, Daniel Duckworth, Humphrey Hu, Andrew Wan, Xiao-Yu Fu, Kenneth Y. Goldberg, Pieter Abbeel |
ICRA | 7 |
| 2010 | Gravity-Based Robotic Cloth Folding
Jur P. van den Berg, Kenneth Y. Goldberg, Pieter Abbeel |
WAFR | 3 |
| 2010 | LQG-Based Planning, Sensing, and Control of Steerable Needles
Jur P. van den Berg, Sachin Patil, Ron Alterovitz, Pieter Abbeel, Kenneth Y. Goldberg |
WAFR | 5 |
| 2009 | Promoting social creativity: a component of a national initiative for social participationabstractThis panel will discuss group processes that promote social creativity in science, engineering, arts, and humanities. We will offer positive and negative examples of social creativity projects, while suggesting research directions for dramatically increased social participation. The goal is to develop strategies that would expand resources and opportunities for research and education in social creativity. This requires our community to develop a unified position, then reach out to national science funding agencies, while building the case for the importance of this topic beyond our own community. How can social creativity, collaborative discovery, distributed innovation, and collective intelligence be framed as an international priority to cope with the problems of the 21st century and how can we identify a clear set of research challenges? Ben Shneiderman, Elizabeth F. Churchill, Gerhard Fischer, Kenneth Y. Goldberg |
Creativity & Cognition | 4 |
| 2009 | Guiding medical needles using single-point tissue manipulationabstractThis paper addresses the use of robotic tissue manipulation in medical needle insertion procedures to improve targeting accuracy and to help avoid damaging sensitive tissues. To control these multiple, potentially competing objectives, we present a phased controller that operates one manipulator at a time using closed-loop imaging feedback. We present an automated procedure planning technique that uses tissue geometry to select the needle insertion location, manipulation locations, and controller parameters. The planner uses a stochastic optimization of a cost function that includes tissue stress and robustness to disturbances. We demonstrate the system on 2D tissues simulated with a mass-spring model, including a simulation of a prostate brachytherapy procedure. It can reduce targeting errors from more than 2 cm to less than 1 mm, and can also shift obstacles by over 1 cm to clear them away from the needle path. Meysam Torabi, Kris Hauser, Ron Alterovitz, Vincent Duindam, Kenneth Y. Goldberg |
ICRA | 5 |
| 2009 | Surgical retraction of non-uniform deformable layers of tissue: 2D robot grasping and path planningabstractThis paper considers robotic automation of a common surgical retraction primitive of exposing an underlying area by grasping and lifting a thin, 3D, possibly inhomogeneous layer of tissue. We present an algorithm that computes a set of stable and secure grasp-and-retract trajectories for a point-jaw gripper moving along a plane, and runs a 3D finite element (FEM) simulation to certify and assess the quality of each trajectory. To compute secure candidate grasp locations, we use a continuous spring model of thin, inhomogeneous deformable objects with linear energy potential. Experiments show that this method produces many of the same grasps as an exhaustive optimization with an FEM mesh, but is orders of magnitude cheaper: our method runs in O(v log v) time, where v is the number of veins, while the FEM computation takes O(pn3) time, where n is the number of nodes in the FEM mesh and p is the number of nodes on its perimeter. Furthermore, we present a constant tissue curvature (CTC) retraction trajectory that distributes strain uniformly around the medial axis of the tissue. 3D FEM simulations show that the CTC achieves retractions with lower tissue strain than circular and linear trajectories. Overall, our algorithm computes and certifies a high-quality retraction in about one minute on a PC. Rik Jansen, Kris Hauser, Nuttapong Chentanez, A. Frank van der Stappen, Kenneth Y. Goldberg |
IROS | 5 |
| 2009 | Nonparametric belief propagation for distributed tracking of robot networks with noisy inter-distance measurementsabstractWe consider the problem of tracking multiple moving robots using noisy sensing of inter-robot and inter-beacon distances. Sensing is local: there are three fixed beacons at known locations, so distance and position estimates propagate across multiple robots. We show that the technique of Nonparametric Belief Propagation (NBP), a graph-based generalization of particle filtering, can address this problem and model multi-modal and ring-shaped uncertainty distributions. NBP provides the basis for distributed algorithms in which messages are exchanged between local neighbors. Generalizing previous approaches to localization in static sensor networks, we improve efficiency and accuracy by using a dynamics model for temporal tracking. We compare the NBP dynamic tracking algorithm with SMCL+R, a sequential Monte Carlo algorithm. Whereas NBP currently requires more computation, it converges in more cases and provides estimates that are 3 to 4 times more accurate. NBP also facilitates probabilistic models of sensor accuracy and network connectivity. Jeremy Schiff, Erik B. Sudderth, Kenneth Y. Goldberg |
IROS | 3 |
| 2009 | Planning fireworks trajectories for steerable medical needles to reduce patient traumaabstractAccurate insertion of needles to targets in 3D anatomy is required for numerous medical procedures. To reduce patient trauma, a "fireworks" needle insertion approach can be used in which multiple needles are inserted from a single small region on the patient's skin to multiple targets in the tissue. In this paper, we explore motion planning for "fireworks" needle insertion in 3D environments by developing an algorithm based on Rapidly-exploring Random Trees (RRTs). Given a set of targets, we propose an algorithm to quickly explore the configuration space by building a forest of RRTs and to find feasible plans for multiple steerable needles from a single entry region. We present two path selection algorithms with different optimality considerations to optimize the final plan among all feasible outputs. Finally, we demonstrate the performance of the proposed algorithm with a simulation based on a prostate cancer treatment environment. Jijie Xu, Vincent Duindam, Ron Alterovitz, Jean Pouliot, J. Adam M. Cunha, I-Chow Hsu, Kenneth Y. Goldberg |
IROS | 7 |
| 2009 | Donation dashboard: a recommender system for donation portfoliosabstractIn this paper we present Donation Dashboard, a system that recommends non-profit organizations to users in the form of a portfolio of donation amounts. Recommendations are made using our Eigentaste 2.0 constant-time collaborative filtering algorithm in combination with a new method for generating a weighted portfolio of recommendations. The key challenge is to generate a customized portfolio that does not necessarily exclude items already rated by the user. Under our method, the weights for items in the portfolio that have not yet been rated by the user are normalized factors of their predicted ratings, and the weights for items previously rated by the user are normalized factors of the actual ratings. Donation Dashboard 1.0 launched in April 2008, and as of May 8 2009 we have collected over 59,000 ratings of 70 nonprofit organizations from over 3,800 users. Tavi Nathanson, Ephrat Bitton, Kenneth Y. Goldberg |
RecSys | 3 |
| 2009 | Sensorless Motion Planning for Medical Needle Insertion in Deformable TissuesabstractMinimally invasive medical procedures such as biopsies, anesthesia drug injections, and brachytherapy cancer treatments require inserting a needle to a specific target inside soft tissues. This is difficult because needle insertion displaces and deforms the surrounding soft tissues causing the target to move during the procedure. To facilitate physician training and preoperative planning for these procedures, we develop a needle insertion motion planning system based on an interactive simulation of needle insertion in deformable tissues and numerical optimization to reduce placement error. We describe a 2-D physically based, dynamic simulation of needle insertion that uses a finite-element model of deformable soft tissues and models needle cutting and frictional forces along the needle shaft. The simulation offers guarantees on simulation stability for mesh modifications and achieves interactive, real-time performance on a standard PC. Using texture mapping, the simulation provides visualization comparable to ultrasound images that the physician would see during the procedure. We use the simulation as a component of a sensorless planning algorithm that uses numerical optimization to compute needle insertion offsets that compensate for tissue deformations. We apply the method to radioactive seed implantation during permanent seed prostate brachytherapy to minimize seed placement error. Ron Alterovitz, Kenneth Y. Goldberg, Jean Pouliot, I-Chow Hsu |
IEEE Trans. Inf. Technol. Biomed. | 2 |
| 2009 | Interactive simulation of surgical needle insertion and steeringabstractWe present algorithms for simulating and visualizing the insertion and steering of needles through deformable tissues for surgical training and planning. Needle insertion is an essential component of many clinical procedures such as biopsies, injections, neurosurgery, and brachytherapy cancer treatment. The success of these procedures depends on accurate guidance of the needle tip to a clinical target while avoiding vital tissues. Needle insertion deforms body tissues, making accurate placement difficult. Our interactive needle insertion simulator models the coupling between a steerable needle and deformable tissue. We introduce (1) a novel algorithm for local remeshing that quickly enforces the conformity of a tetrahedral mesh to a curvilinear needle path, enabling accurate computation of contact forces, (2) an efficient method for coupling a 3D finite element simulation with a 1D inextensible rod with stick-slip friction, and (3) optimizations that reduce the computation time for physically based simulations. We can realistically and interactively simulate needle insertion into a prostate mesh of 13,375 tetrahedra and 2,763 vertices at a 25 Hz frame rate on an 8-core 3.0 GHz Intel Xeon PC. The simulation models prostate brachytherapy with needles of varying stiffness, steering needles around obstacles, and supports motion planning for robotic needle insertion. We evaluate the accuracy of the simulation by comparing against real-world experiments in which flexible, steerable needles were inserted into gel tissue phantoms. Nuttapong Chentanez, Ron Alterovitz, Daniel Ritchie 0001, Lita Cho, Kris Hauser, Kenneth Y. Goldberg, Jonathan Richard Shewchuk, James F. O'Brien |
ACM Trans. Graph. | 6 |
| 2008 | Screw-based motion planning for bevel-tip flexible needles in 3D environments with obstaclesabstractBevel-tip flexible needles have greater mobility than straight rigid needles, and can be used to reach targets behind sensitive or impenetrable areas. Accurately planning and executing the optimal motions for such steerable needles is difficult, however, and requires solving inverse kinematics for a nonholonomic system.This paper presents an approach to 3D motion planning for bevel-tip needles in an environment with obstacles. Instead of discretizing the configuration space as in earlier work, we discretize the control space, such that the trajectory of the needle can be expressed analytically without the need for approximate numerical simulation. This results in a fast optimization routine that finds a locally optimal path in a 3D environment with obstacles, requiring just a few seconds of computation time on a standard PC.We introduce two different discretization strategies that lead to differently structured paths and show that both produce valid trajectories from start to goal. To our knowledge, the presented method is the first to address motion planning for bevel-tip needles in a 3D environment with obstacles. Vincent Duindam, Ron Alterovitz, S. Shankar Sastry, Kenneth Y. Goldberg |
ICRA | 4 |
| 2008 | 3D Motion Planning Algorithms for Steerable Needles Using Inverse Kinematics
Vincent Duindam, Jijie Xu, Ron Alterovitz, S. Shankar Sastry, Kenneth Y. Goldberg |
WAFR | 5 |
| 2007 | Respectful cameras: detecting visual markers in real-time to address privacy concernsabstractTo address privacy concerns with digital video surveillance cameras, we propose a practical, real-time approach that preserves the ability to observe actions while obscuring individual identities. In our proposed respectful cameras system, people who wish to remain anonymous agree to wear colored markers such as a hat or vest. The system automatically tracks these markers using statistical learning and classification to infer the location and size of each face and then inserts elliptical overlays. Our objective is to obscure the face of each individual wearing a marker, while minimizing the overlay area in order to maximize the remaining observable region of the scene. Our approach incorporates a visual color-tracker based on a 9 dimensional color-space by using a probabilistic AdaBoost classifier with axis-aligned hyperplanes as weak-learners. We then use particle filtering to incorporate interframe temporal information. We present experiments illustrating the performance of our system in both indoor and outdoor settings, where occlusions, multiple crossing targets, and lighting changes occur. Results suggest that the respectful camera system can reduce false negative rates to acceptable levels (under 2%). Jeremy Schiff, Marci Meingast, Deirdre K. Mulligan, S. Shankar Sastry, Kenneth Y. Goldberg |
IROS | 5 |
| 2007 | Eigentaste 5.0: constant-time adaptability in a recommender system using item clusteringabstractRecommender systems strive to recommend items that users will appreciate and rate highly, often presenting items in order of highest predicted ratings first. In this working paper we present Eigentaste 5.0, a constant-time recommender system that dynamically adapts the order that items are recommended by integrating user clustering with item clustering and monitoring item portfolio effects. This extends our Eigentaste 2.0 algorithm, which uses principal component analysis to cluster users offline. In preliminary experiments we backtested Eigentaste 5.0 on data collected from Jester, our online joke recommender system. Results suggest that it will perform better than Eigentaste 2.0. The new algorithm also uses item clusters to address the cold-start problem for introducing new items. Tavi Nathanson, Ephrat Bitton, Kenneth Y. Goldberg |
RecSys | 3 |
| 2007 | Approximate Algorithms for a Collaboratively Controlled Robotic CameraabstractDeployed as a natural environment observatory or a surveillance device, a remote networked robotic pan-tilt-zoom camera needs to be controlled by simultaneous frame requests from both online users and in situ sensors such as motion detectors. This paper presents algorithms that are capable of finding a camera frame that optimizes a measure of total satisfaction over all requests, which is a generalized version of the single frame-selection problem proposed by Song et al. in 2006.We present a lattice-based approximation algorithm; given n requests and approximation bound ∈, we analyze the tradeoff between solution quality and the corresponding computation time, and prove that the algorithm runs in O(n/∈3) time. We also develop a branch-and-bound-like implementation that reduces the constant factor of the algorithm by more than 70%. We have implemented the algorithms, and numerical experiment results conform to our analysis. Field experiments of the proposed algorithms have been conducted in the past three years. The proposed algorithms have been deployed successfully in a variety of real world applications including natural environment observation, building construction monitoring, and the surveillance of public space. Dezhen Song, Kenneth Y. Goldberg |
IEEE Trans. Robotics | 2 |
| 2006 | Blades: a New Class of Geometric Primitives for Feeding 3D Parts on Vibratory TracksabstractThe vibratory bowl feeder remains the most common approach to the automated feeding (orienting) of industrial parts. We study the algorithmic design of devices on the bowl feeder track that filter out all but one orientation of a given polyhedral part. In this context, we propose a simple new primitive, consisting of one horizontally mounted convex polygonal metal "blade", that can feed a broad class of three-dimensional parts by reorienting and rejecting all but those in a desired orientation. This powerful new 3D geometric feeding primitive combines the reorientation functionality of fences with the rejection functionality of traps. Due to its simplicity, the proposed primitive allows for the development of methods to automate its design process. We present a complete procedure that takes as input any polyhedral part along with its center of mass. Given this input, the procedure identifies all single blade solutions that feed the part. The output is either the set of all valid blade designs or a notification that the part cannot be fed using a single blade Onno C. Goemans, Kenneth Y. Goldberg, A. Frank van der Stappen |
ICRA | 2 |
| 2006 | Aligning Windows of Live Video From an Imprecise Pan-tilt-zoom Robotic Camera into a Remote Panoramic DisplayabstractA pan-tilt-zoom robotic camera can provide detailed live video of selected areas of interest within a large potential viewing field. To provide spatial context for human observers, it is desirable to insert the resulting live video into a large spherical panoramic display representing the entire viewing field. Accurate alignment of the video stream within the panoramic display is difficult due to small errors in the robot pan-tilt values and image distortion due to nonlinear projection. Existing image alignment algorithms cannot keep up with rapid changes in camera position. In this paper, we present a constant-time image alignment algorithm based on spherical projection and projection-invariant selective sampling that accurately registers paired images at 25 frames per second on a standard PC. Experiments suggest that the new alignment algorithm is faster than previous algorithms by a factor four or more. In a companion paper, we present a new calibration algorithm based on image variance density that optimally estimates camera pan-tilt parameters Ni Qin, Dezhen Song, Kenneth Y. Goldberg |
ICRA | 3 |
| 2006 | A Minimum Variance Calibration Algorithm for Pan-tilt Robotic Cameras in Natural EnvironmentsabstractA new generation of inexpensive robotic pan-tilt cameras can maintain high-resolution panoramic displays of natural environments. However, the pan-tilt mechanisms are imprecise: small errors can produce large errors in the panoramic display. It is thus important to accurately estimate pan-tilt values. We present a new calibration algorithm that does not rely on calibration markers or fixed orthogonal edges which are rarely available in natural scenes. Our calibration algorithm uses image variance density to optimally estimate camera pan and tilt values by incrementally refining image registration using overlapping images from prior frames. Experiments suggest that the new calibration algorithm can reduce calibration error by 81%. In a companion paper, we present a new image registration algorithm based on spherical projection that optimally aligns the resulting frames Dezhen Song, Ni Qin, Kenneth Y. Goldberg |
ICRA | 3 |
| 2006 | Sensitivity analysis: unexpected outcomes in art and engineeringabstractContemporary art and engineering research are both at their best when things don't turn out as planned. I'll present selected examples based on artworks developed with students and other collaborators involving robots and networks over the past 20 years. These projects set out to investigate intersections of technology and nature, such as the Telegarden, a robot installation that allowed online participants to remotely tend a living garden; Ballet Mori, a classical dance performed to sounds triggered by live seismic data; and Demonstrate, where an ultra high-resolution video camera raised eyebrows at the 40th anniversary of the Free Speech Movement. Every project led to unexpected twists and complications...I'll also argue that the languages of contemporary art and engineering research are complex, dynamic, and often frustratingly impenetrable to outsiders. In art, a blue disk can be a cliche, or, in the right place at the right time, profound. In engineering, analogous contexts determine the beauty of a coordinate frame or mathematical equation. In both spheres, aesthetic interpretation is based on knowledge of prior art and contemporary dialogues. Being so similar, it is not surprising that unexpected forces arise when these two spheres are brought together. Kenneth Y. Goldberg |
ACM Multimedia | 1 |
| 2006 | Constant-Curvature Motion Planning Under Uncertainty with Applications in Image-Guided Medical Needle Steering
Ron Alterovitz, Michael S. Branicky, Kenneth Y. Goldberg |
WAFR | 3 |
| 2006 | Exact algorithms for single frame selection on multiaxis SatellitesabstractNew multi-axis satellites allow camera imaging parameters to be set during each time slot based on competing demand for images, specified as rectangular requested viewing zones over the camera's reachable field of view. The single frame selection (SFS) problem is to find the camera frame parameters that maximize reward during each time window. We formalize the SFS problem based on a new reward metric that takes into account area coverage and image resolution. For a set of n client requests and a satellite with m discrete resolution levels, we give an algorithm that solves the SFS problem in time O(n/sup 2/m). For satellites with continuously variable resolution (m=/spl infin/), we give an algorithm that runs in time O(n/sup 3/). We have implemented all algorithms and verify performance using random inputs. Note to Practitioners-This paper is motivated by recent innovations in earth imaging by commercial satellites. In contrast to previous methods that required waits of up to 21 days for desired earth- satellite alignment, new satellites have onboard pan-tilt-zoom cameras that can be remotely directed to provide near real-time response to requests for images of specific areas on the earth's surface. We consider the problem of resolving competing requests for images: Given client demand as a set of rectangles on the earth surface, compute camera settings that optimize the tradeoff between pan, tilt, and zoom parameters to maximize camera revenue during each time slot. We define a new quality metric and algorithms for solving the problem for the cases of discrete and continuous zoom values. These results are a step toward multiple frame selection which will be addressed in future research. The metric and algorithms presented in this paper may also be applied to collaborative teleoperation of ground-based robot cameras for inspection and videoconferencing and for scheduling astronomic telescopes. Dezhen Song, A. Frank van der Stappen, Kenneth Y. Goldberg |
IEEE Trans Autom. Sci. Eng. | 3 |
| 2005 | Planning for Steerable Bevel-tip Needle Insertion Through 2D Soft Tissue with ObstaclesabstractWe explore motion planning for a new class of highly flexible bevel-tip medical needles that can be steered to previously unreachable targets in soft tissue. Planning for these procedures is difficult because the needles bend during insertion and cause the surrounding soft tissues to displace and deform. In this paper, we develop a planning algorithm for insertion of highly flexible bevel-tip needles into soft tissues with obstacles in a 2D imaging plane. Given an initial needle insertion plan specifying location, orientation, bevel rotation, and insertion distance, the planner combines soft tissue modeling and numerical optimization to generate a needle insertion plan that compensates for simulated tissue de formations, locally avoids polygonal obstacles, and minimizes needle insertion distance. The simulator computes soft tissue deformations using a finite element model that incorporates the effects of needle tip and frictional forces using a 2D mesh. We formulate the planning problem as a constrained nonlinear optimization problem that is locally minimized using a penalty method that converts the formulation to a sequence of unconstrained optimization problems. We apply the planner to bevel-right and bevel-left needles and generate plans for targets that are unreachable by rigid needles. Ron Alterovitz, Kenneth Y. Goldberg, Allison M. Okamura |
ICRA | 2 |
| 2005 | Steering flexible needles under Markov motion uncertaintyabstractWhen inserted into soft tissues, flexible needles with bevel tips have been shown experimentally to follow a path of constant curvature in the direction of the bevel. By controlling 2 degrees of freedom at the needle base (bevel direction and insertion distance), these needles can be steered around obstacles to reach targets inaccessible to rigid needles. Motion planning for needle steering is a type of nonholonomic planning for a Dubins car with no reversal. We develop a motion planning algorithm based on dynamic programming where the path of the needle is uncertain due to uncertainty in tissue properties, needle mechanics, and interaction forces. The algorithm computes a discrete control sequence of insertions and direction changes so the needle reaches a target in an imaging plane while minimizing expected cost due to insertion distance, direction changes, and obstacle collisions. We efficiently sample the state space of needle tip positions and orientations and define bounds on the errors due to discretization. We formulate the motion planning problem as a Markov decision process (MDP) and use infinite horizon dynamic programming to compute an optimal control sequence. We first apply the method to the deterministic motion case where the needle precisely follows a path of constant curvature and then to the uncertain motion case where state transitions are defined by a probability distribution. Our implementation generates motion plans for bevel-tip needles that reach targets inaccessible to rigid needles and demonstrates that accounting for uncertainty can lead to significantly different motion plans. Ron Alterovitz, Andrew E. B. Lim, Kenneth Y. Goldberg, Gregory S. Chirikjian, Allison M. Okamura |
IROS | 3 |
| 2005 | Networked Robotic Cameras for Collaborative Observation of Natural Environments
Dezhen Song, Kenneth Y. Goldberg |
ISRR | 2 |
| 2004 | Unsupervised Scoring for Scalable Internet-based Collaborative TeleoperationabstractFor applications in education and entertainment, scalable Internet-based collaborative teleoperation allows many users simultaneously to share control of a single device. Automated numerical methods that can assess and record performance provide an incentive for users to participate and a means to evaluate individual and group performance. In this paper we describe "unsupervised scoring": a numerical approach to assessment based on clustering and response time. Like unsupervised learning, this approach is based on identifying regularities in the input rather than comparing input with desired output specified by an external supervisor. We present an algorithm for rapidly computing user scores that scales linearly with the number of users. We describe an implemented Java-based user interface incorporating this metric, an application based on the classic Twister game, and results where individual scores are compared with group performance. Kenneth Y. Goldberg, Dezhen Song, In Yong Song, Jane McGonigal, Dana Plautz |
ICRA | 1 |
| 2004 | D-space and Deform Closure: a Framework for Holding Deformable PartsabstractWe extend the form closure framework for rigid parts to holding a class of deformable parts. In this class, a part is a linearly elastic, frictionless polygon with a finite element mesh and given stiffness matrix. We define the D-space (deformation-space) of a part as the C-space of all its mesh nodes. We define the free space as the intersection of the set of topology preserving mesh configurations with the complement of the union of all D-obstacles that represent collisions of the part with finger bodies. Consider a given set of finger bodies in frictionless contact with a part. When positive work is needed to release the part, we say that it is in deform closure. We present a numerical example and prove two results: (1) if contact set holds a rigid part in form closure, it will hold the equivalent deformable part in deform closure; and (2) deform closure is frame invariant. K. Gopalakrishnan 0002, Kenneth Y. Goldberg |
ICRA | 2 |
| 2004 | An Exact Algorithm Optimizing Coverage-resolution for Automated Satellite Frame SelectionabstractNear real time satellite imaging provides timely images of the earth for weather prediction, disaster response, search and rescue, surveillance, and defense applications. As the satellite passes over the earth, camera imaging parameters are changed during each time window based on demand for images, specified as user requested zones in the reachable field of view during that time window. The satellite frame selection (SFS) problem is to find the camera frame parameters that maximize reward during each time window. To automate satellite management, we formalize the SFS problem based on a new reward metric that incorporates both image resolution and coverage. For a set of n client requests we give a series of algorithms, the fastest computes optimal results in O(n/sup 3/) for satellites with continuously variable resolution. We have implemented the algorithms and compare computation speed for all algorithms. Dezhen Song, A. Frank van der Stappen, Kenneth Y. Goldberg |
ICRA | 3 |
| 2004 | Computing Deform Closure Grasps
K. Gopalakrishnan 0002, Kenneth Y. Goldberg |
WAFR | 2 |
| 2004 | Networked Robots: Ten Years of Experiments
Kenneth Y. Goldberg |
WAFR | 1 |
| 2004 | Unilateral fixtures for sheet-metal parts with holesabstractIn this paper, we introduce unilateral fixtures , a new class of fixtures for sheet-metal parts with holes. These fixtures use cylindrical jaws with conical grooves that facilitate part alignment; each jaw provides the equivalent of four point contacts. The fixtures are unilateral in the sense that their actuating mechanisms are restricted to one side/surface of the part, facilitating access to the other side/surface for assembly or inspection. We present a two-phase algorithm for computing unilateral fixtures. Phase I is a geometric algorithm that assumes the part is rigid and applies two-dimensional (2-D) and three-dimensional (3-D) kinematic analysis of form closure to identify all candidate locations for pairs of primary jaws. We prove three new grasp properties for 2-D and 3-D grips at concave vertices and define a scale-invariant quality metric based on the sensitivity of part orientation to infinitesimal relaxation of jaw position. Phase II uses a finite element method to compute part deformation and to arrange secondary contacts at part edges and interior surfaces. For a given sheet-metal part, given as a 2-D surface embedded in 3-D with e edges, n concavities and m mesh nodes, Phase I takes O(e+n/sup 4/3/log/sup 1/3/n+glogg) time to compute a list of g pairs of primary jaws ranked by quality. Phase II computes the location of r secondary contacts in O(grm/sup 3/) time. Note to Practitioners-This paper was motivated by the problem of holding sheet-metal parts for automobile bodies but it also applies to other sheet-metal components that have cut or stamped holes. Existing approaches to fixturing such parts generally have contacting mechanisms on both sides of the sheet that restrict access for welding or inspection. This paper suggests a new approach using pairs of grooved cylinders, activated from only one side of the part (hence "unilateral"). These cylinders mate with opposing corners of holes in the sheet and push apart to hold the sheet in tension, thus acting as both locators and clamps. In this paper, we mathematically characterize the mechanics and conditions for a unilateral fixture to hold a given part. We then show how such fixtures can be efficiently computed; this can allow a computer-aided design (CAD) system (with finite element capability) to automatically generate and propose unilateral fixtures for a given part. Preliminary physical experiments suggest that this approach is feasible but it has not yet been incorporated into a CAD system nor tested in production. In future research, we will address the design of unilateral fixtures that hold two or more parts simultaneously for welding. K. Gopalakrishnan 0002, Kenneth Y. Goldberg, Gary M. Bone, Matthew Zaluzec, Rama Koganti, Rich Pearson, Patricia Deneszczuk |
IEEE Trans Autom. Sci. Eng. | 2 |
| 2004 | Editorial
Peter B. Luh, Kenneth Y. Goldberg, Richard A. Volz |
IEEE Trans Autom. Sci. Eng. | 2 |
| 2004 | Guest Editorial
Michael Yu Wang, Edward C. DeMeter, Shreyes N. Melkote, Kenneth Y. Goldberg, Zexiang Li 0001 |
IEEE Trans Autom. Sci. Eng. | 4 |
| 2003 | Efficient algorithms for shared camera controlabstractWe consider a system that allows n networked users to share control over a robotic webcamera. Each user guides the camera pan, tilt and zoom, by drawing a rectangle in the user interface. The server adjusts the camera to best satisfy the user requests, by solving a geometric optimization problem that requires fitting one rectangle to many. We improve upon previous results with an O(n3/2 log3 n) time exact algorithm for this problem. We also present a simple near-linear time e-approximation algorithm. We have implemented the latter and report on experimental results. Sariel Har-Peled, Vladlen Koltun, Dezhen Song, Kenneth Y. Goldberg |
SCG | 4 |
| 2003 | Needle insertion and radioactive seed implantation in human tissues: simulation and sensitivity analysisabstractTo facilitate training and planning for medical procedures such as prostate brachytherapy, we are developing an interactive simulation of needle insertion and radioactive seed implantation in soft tissues. We describe a new 2D dynamic FEM model based on a reduced set of scalar parameters such as needle friction, sharpness, and velocity, where the mesh is updated to maintain element boundaries along the needle shaft and the effects of needle tip and frictional forces are simulated. The computational complexity of our model grows linearly with the number of elements in the mesh and achieves 24 frames per second for 1250 triangular elements on a 750 MHz PC. We use the simulator to characterize the sensitivity of seed placement error to physician-controlled and biological parameters. Results indicate that seed placement error is highly sensitive to physician-controlled parameters such as needle position, sharpness, and friction, and less sensitive to patient-specific parameters such as tissue stiffness and compressibility. Ron Alterovitz, Kenneth Y. Goldberg, Jean Pouliot, Richard Tascherau, I-Chow Hsu |
ICRA | 2 |
| 2003 | "Unilateral" fixturing of sheet metal parts using modular jaws with plane-cone contactsabstractTo fixture sheet metal parts for welding, we propose "unilateral fixtures" consisting of modular fixturing elements that lie almost completely on one side of the part. These are based on cylindrical jaws with conical grooves which provide the equivalent of 4 point contacts. We propose a two-phase procedure for designing unilateral fixtures. The first phase is a geometric algorithm that assumes the part is rigid and computes vg-grips (vertex-groove grips). The vg-grip algorithm uses a fast sufficient test for immobility to generate a list of vg-grips and find bounds on jaw cone angles for each. The second phase is a fast heuristic procedure that uses FEM to arrange secondary contacts to reduce part deformation. For a part described by n concave "virtual vertices", a list of vg-grips and minimum half cone angles for each vg-grip can be generated in O(n/sup 2/) time. We also propose a quality metric based on the sensitivity of the part's orientation to an infinitesimal relaxation of the jaws that can be evaluated in constant time for a given fixture. For an FEM model with m nodes, the second phase takes O(m/sup 3/r) time to arrange r secondary contacts for each vg-grip. K. Gopalakrishnan 0002, Matthew Zaluzec, Rama Koganti, Patricia Deneszczuk, Kenneth Y. Goldberg |
ICRA | 5 |
| 2003 | Sensorless planning for medical needle insertion proceduresabstractMedical procedures such as seed implantation, biopsies, and treatment injections require inserting a needle tip to a specific target location inside the human body. This is difficult because (1) needle insertion causes soft tissues to displace and deform, and (2) it is often difficult or impossible to obtain precise imaging data during insertion. We are developing a sensorless planning system for needle insertion that incorporates numerical optimization with a soft tissue simulation based on a dynamic FEM formulation that models the effects of needle tip and frictional forces using a 2D mesh. In this paper we describe a sensorless planning algorithm for radioactive seed implantation that computes needle insertion offsets that compensate for tissue deformations. We apply the method to seed implantation during permanent seed prostate brachytherapy to minimize seed placement error in simulation without relying on real-time imaging. Ron Alterovitz, Kenneth Y. Goldberg, Jean Pouliot, Richard Tascherau, I-Chow Hsu |
IROS | 2 |
| 2003 | ShareCam part 1: interface, system architecture, and implementation of a collaboratively controlled robotic WebcamabstractShareCam is a robotic pan, tilt, and zoom web-based camera controlled by simultaneous frame requests from online users. Part II describes algorithms. This paper, part I, focuses on the system. Robotic Webcameras are commercially available but currently restrict control only one user at a time. ShareCam introduces a new interface that allows simultaneous control many users. In this Java-based interface, participating users interact desired frames remotely located browsers where users draw desired frames over a fixed panoramic image. User inputs re transmitted back to pair of PC servers that compute optimal camera back to a pair of PC servers that compute optimal camera parameters, servo the camera, and provide a video stream to all users. We describe the system, online experiments, and compare results with two frame selection models based on user "satisfaction", one memoryless and the second based on satisfaction over multiple motion cycles. Dezhen Song, Kenneth Y. Goldberg |
IROS | 2 |
| 2003 | ShareCam part II: approximate and distributed algorithms for a collaboratively controlled robotic WebcamabstractShareCam is a robotic pan, tilt, and zoom Web-based camera controlled by simultaneous frame requests from online users. Part I describes the system. This paper, part II, focuses on algorithms. The ShareCam problem is to find a camera frame that optimizes a measure of total user satisfaction. We present a grid-based approximation algorithm: given camera frame requests from n users, and approximation bound /spl epsi/, we analyze the trade of between solution quality and processing speed and prove that the algorithm runs in O(n//spl epsi//sup 3/) time. The algorithm can be distributed to run in O(1//spl epsi//sup 3/) time at each client and in O(n + 1//spl epsi//sup 3/) time at the server. Experiments suggest that performance of the distributed algorithm degrades gracefully as clients fail to complete their part of the computation. Dezhen Song, Anatoly Pashkevich, Kenneth Y. Goldberg |
IROS | 3 |
| 2003 | The co-opticon: shared access to a robotic streaming video cameraabstractThe "co-opticon" is a robotic pan, tilt, and zoom streaming video camera controlled by simultaneous frame requests from remote users. Robotic webcameras are commercially available but currently restrict control to only one user at a time. The co-opticon introduces a new interface that allows simultaneous control by many users. We will demonstrate the implemented system using a Java-based interface at the conference linked via the Internet to a camera on the UC Berkeley campus. We will also discuss system architecture and several new algorithms we've developed to compute optimal camera paramters based on user frame requests. The co-opticon can be tested online at: www.tele-actor.net/co-opticon. Dezhen Song, Kenneth Y. Goldberg |
ACM Multimedia | 2 |
| 2003 | Collaborative teleoperation using networked spatial dynamic votingabstractWe describe a networked teleoperation system that allows groups of participants to collaboratively explore live remote environments. Participants collaborate using a spatial dynamic voting (SDV) interface that allows them to vote on a sequence of images via a network such as the Internet. The SDV interface runs on each client computer and communicates with a central server that collects, displays, and analyzes time sequences of spatial votes. The results are conveyed to the "tele-actor", a skilled human with cameras and microphones who navigates and performs actions in the remote environment. This paper formulates analysis in terms of spatial interest functions and consensus regions, and presents system architecture, interface, and algorithms for processing voting data. Kenneth Y. Goldberg, Dezhen Song, Anthony Levandowski |
Proc. IEEE | 1 |
| 2002 | Fixturing Hinged PolygonsabstractWe study the problem of fixturing a chain of hinged objects in a given placement with frictionless point contacts. We define the notions of immobility and robust immobility - which are comparable to the second and first order immobility for a single object - to capture the intuitive requirement for the fixture of a chain of hinged objects. Robust immobility differs from immobility in that it additionally requires insensitivity to small perturbations of contacts. We show that (p+2) frictionless point contacts can immobilize any chain of p/spl ne/3 polygons without parallel edges; six contacts can immobilize any chain of three such polygons. Any chain of p arbitrary polygons can be immobilized with at most (p+4) contacts. We also show that /spl lceil/(6/5)(p+2)/spl rceil/ contacts suffice to robustly immobilize p polygons without parallel edges, and that /spl lceil/(5/4)(p+2)/spl rceil/ contacts can robustly immobilize p/spl ne/3 arbitrary polygons, and eight contacts can robustly immobilize three polygons. Jae-Sook Cheong, Kenneth Y. Goldberg, Mark H. Overmars, A. Frank van der Stappen |
ICRA | 2 |
| 2002 | Collaborative Online Teleoperation with Spatial Dynamic Voting and a Human "Tele-Actor"abstractInternet-based "online robots" now provide public access to remote locations such as museums and laboratories. The Tele-Actor is a collaborative online teleoperation system for distance learning that allows many students to simultaneously share control of a single mobile resource. Our goal is to preserve the educational advantages of field trips without the drawbacks of group travel. We propose the "spatial dynamic voting" (SDV) interface for multiple operator single robot (MOSR) teleoperation. The SDV collects, displays, and analyzes a sequence of spatial votes from multiple online operators at their Internet browsers. The votes drive the motion of a single mobile robot or human "Tele-Actor". The paper describes Version 3.0 of the system architecture, SDV interface, algorithms for automated goal selection, and metrics for collaboration and leadership. We report results from a July 2001 field test with 56 remote users. Kenneth Y. Goldberg, Dezhen Song, Yoek-Nam Khor, David Pescovitz, Anthony Levandowski, Jesse C. Himmelstein, Janice Shih, Annamarie Ho, Eric Paulos, Judith S. Donath |
ICRA | 1 |
| 2002 | Gripping Parts at Concave VerticesabstractA simple gripper with two vertical cylindrical jaws can make contact with external or internal concavities in polygonal and polyhedral parts to align and grip parts in form closure. This is called a /spl nu/-grip. We begin by defining 2D /spl nu/-grips, where a pair of frictionless point jaws makes contact with a pair of polygonal part concavities to achieve form-closure. We define a /spl nu/-grip quality metric based on the maximum possible change in the part's orientation when jaw position is relaxed infinitesimally. For a polygonal part with polygonal holes, we give an algorithm for computing and ranking 2D /spl nu/-grips. We also extend the definition to jaws with non-zero radii. In 3D, /spl nu/-grips are achieved with a pair of frictionless vertical cylinders. We define 3D /spl nu/-grips and give a numerical algorithm for computing all 3D /spl nu/-grips of a polyhedral part. If n is the number of vertices that describe the part and k is the number of concave vertices, we can compute all 2D /spl nu/-grips in O(n+k/sup 2/) time. Measures of complexity are given for computing offsets for jaws with nonzero radii, a ranked list of 2D /spl nu/-grips based on the quality metric, and all 3D /spl nu/-grips. A Java implementation of the 2D algorithm is available. K. Gopalakrishnan 0002, Kenneth Y. Goldberg |
ICRA | 2 |
| 2002 | Orienting Micro-Scale Parts with Squeeze and Roll PrimitivesabstractOrienting parts that measure only a few micrometers in diameter introduces several challenges that need not be considered at the macro-scale. First, there are several kinds of sticking effects due to Van der Waals forces and static electricity which complicate hand-off motions and release of a part. Second, the degrees of freedom of micromanipulators are limited. The paper proposes a pair of manipulation primitives and a complete algorithm that addresses these challenges. We show that a sequence of these two manipulation primitives can uniquely orient any asymmetric part while maintaining contact without sensing. This allows us to apply the same plan to many (identical) parts simultaneously. For asymmetric parts we can find a plan of length O(n) in O(n) time that orients the part, where n is the number of vertices. Mark Moll, Kenneth Y. Goldberg, Michael A. Erdmann, Ronald S. Fearing |
ICRA | 2 |
| 2002 | Exact and Distributed Algorithms for Collaborative Camera Control
Dezhen Song, A. Frank van der Stappen, Kenneth Y. Goldberg |
WAFR | 3 |
| 2002 | Gripper point contacts for part alignmentabstractThe initial resting pose of many industrial parts differs from the orientation desired for assembly. We show that it is possible to align parts during grasping using a standard parallel-jaw gripper. A solution is an arrangement of four gripper point contacts that will align the part in the vertical plane as the jaws close. Given a n-sided polygonal part and k uniformly distributed sample points on each side, we present an O(n/sup 4/k/sup 4/) numerical algorithm to compute a set of solutions or a report that no solution exists. The algorithm combines toppling, accessibility, and form-closure analysis. We have implemented the algorithm and report sensitivity data from physical experiments. Mike Tao Zhang, Kenneth Y. Goldberg |
IEEE Trans. Robotics Autom. | 2 |
| 2001 | Orienting Parts by Inside-out PullingabstractA common task in automated manufacturing processes is that of orienting (or feeding) parts prior to assembly. We propose a new type of feeder. We consider sensorless orientation of polygonal parts with elevated edges by pull actions with an overhead finger. We show that any asymmetric convex polygonal part can be oriented by a sequence of pull operations. We give an O(n/sup 3/) algorithm to compute the shortest sequence of pull operations to orient a convex polygonal part with n vertices, if such a sequence exists. We also show that there exist non-convex parts that cannot be fed by a sequence of pull operations. Robert-Paul Berretty, Kenneth Y. Goldberg, Mark H. Overmars, A. Frank van der Stappen |
ICRA | 2 |
| 2001 | Design of Robot Gripper Jaws based on Trapezoidal ModulesabstractAssembly line reliability can be increased if grippers are carefully designed to capture and align parts. Jaw design depends on the geometric and mechanical properties of the part as well as its desired orientation. We propose a class of modular jaws based on rapidly machineable trapezoidal modules. An optimal jaw design is an arrangement of trapezoidal jaw modules that maximizes contacts between the gripper and the part at its desired final orientation over the constraints that: the jaws will capture and rotate the part to its desired orientation and achieve a form-closure grasp. Given the n-sided 2D convex projection of an extruded polygonal part, we develop an implemented O(n/sup 5/) algorithm to find the optimal jaw design. The algorithm combines toppling, jamming, liftoff, accessibility, and form-closure analysis. We verify resulting designs by physical experiments. Mike Tao Zhang, Kenneth Y. Goldberg |
ICRA | 2 |
| 2001 | Collaborative control of robot motion: robustness to errorabstractWe consider collaborative control systems, where multiple sources share control of a single robot. These sources could come from multiple sensors (sensor fusion), multiple control processes (subsumption), or multiple human operators. Reports suggest that such systems are highly fault tolerant, even with large numbers of sources. We develop a formal model, modeling sources with finite automata. A collaborative ensemble of sources generates a single stream of incremental steps to control the motion of a point robot moving in the plane. We first analyze system performance with a uniform ensemble of well-behaved deterministic sources. We then model malfunctioning sources that go silent or generate inverted control signals. We discover that performance initially improves in the presence of malfunctioning sources and remains robust even when a sizeable fraction of sources malfunction. Initial tests suggest similar results with non-deterministic (random) sources. The formal model may also provide insight into how humans can share control of an online robot. Kenneth Y. Goldberg, Billy Chen |
IROS | 1 |
| 2001 | Shape tolerance for robot gripper jawsabstractIn previous work (2001), we identified a class of robot grippers that can orient and grasp parts using an arrangement of trapezoidal jaw modules. In this paper, we define a parametric tolerance class for the jaws, such that the part alignment is guaranteed for all jaw geometry in the class. This tolerance class is derived based on the analysis of toppling, motion trajectory, and form-closure. We describe the O(n log n) algorithm to compute the parametric tolerance class based on maximal and minimal jaw specifications. We implemented the algorithm and report the results of physical experiments. Mike Tao Zhang, Lawrence Cheung, Kenneth Y. Goldberg |
IROS | 3 |
| 2001 | Eigentaste: A Constant Time Collaborative Filtering Algorithm
Kenneth Y. Goldberg, Theresa M. Roeder, Dhruv Gupta 0003, Chris Perkins |
Inf. Retr. | 1 |
| 2000 | The 2-center problem with obstaclesabstractArticle The 2-center problem with obstacles Share on Authors: Dan Halperin Department of Computer Science, Tel Aviv University, Tel-Aviv, 69978, Israel Department of Computer Science, Tel Aviv University, Tel-Aviv, 69978, IsraelView Profile , Micha Sharir School of Mathematical Sciences, Tel Aviv University, Tel-Aviv, 69978, Israel, and Courant Institute of Mathematical Sciences, New York, University, New York, NY School of Mathematical Sciences, Tel Aviv University, Tel-Aviv, 69978, Israel, and Courant Institute of Mathematical Sciences, New York, University, New York, NYView Profile , Ken Goldberg Department of Industrial Engineering and Operations Research, University of California, Berkeley, CA Department of Industrial Engineering and Operations Research, University of California, Berkeley, CAView Profile Authors Info & Claims SCG '00: Proceedings of the sixteenth annual symposium on Computational geometryMay 2000 Pages 80–90https://doi.org/10.1145/336154.336184Online:01 May 2000Publication History 3citation299DownloadsMetricsTotal Citations3Total Downloads299Last 12 Months2Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access Dan Halperin, Micha Sharir, Kenneth Y. Goldberg |
SCG | 3 |
| 2000 | Collaborative Teleoperation via the InternetabstractWe describe a system that allows a distributed group of users to simultaneously teleoperate an industrial robot arm via the Internet. A Java applet at each client streams mouse motion vectors from up to 3D users; a sewer aggregates these inputs to produce a single control stream for the robot. Users receive visual feedback from a digital camera mounted above the robot arm. To our knowledge, this is the first collaboratively controlled robot on the Internet. Kenneth Y. Goldberg, Billy Chen, Rory Solomon, Steve Bui, Bobak Farzin, Jacob Heitler, Derek Poon, Gordon Smith |
ICRA | 1 |
| 2000 | The Toppling Graph: Designing Pin Sequences for Part FeedingabstractWe consider a sensorless approach to feeding parts on a conveyor belt using pins (rigid barriers) to topple parts into desired orientations. Given the 2D projection of an n-sided convex polyhedral part, its center of mass, and coefficients of friction, we give an O(n/sup 2/) algorithm to compute the "toppling graph", a new data structure that represents the mechanics of toppling, including rolling and jamming. The toppling graph can be used to identify critical pin heights that permit toppling. We compare pin heights predicted by the graph with physical experiments, and give a complete O(n/sup 3n/) algorithm for designing pin sequences. Mike Tao Zhang, Gordon Smith, Robert-Paul Berretty, Mark H. Overmars, Kenneth Y. Goldberg |
ICRA | 5 |
| 2000 | Algorithms for Sensorless Manipulation Using a Vibrating Surface
Karl-Friedrich Böhringer, Vivek Bhatt, Bruce Randall Donald, Kenneth Y. Goldberg |
Algorithmica | 4 |
| 2000 | Geometric Eccentricity and the Complexity of Manipulation Plans
A. Frank van der Stappen, Kenneth Y. Goldberg |
Algorithmica | 2 |
| 1999 | Geometric Algorithms for Trap DesignabstractGeometric algorithms have successfully been applied to solve or give insight into problems in robotic manipulation. In this paper, we present a framework to filter polygonal parts on a track. The filter consists of a polygonal hole in the track; we refer to the filters as traps. For an n-sided polygonal part and an m-sided polygonal trap, we give an O((nm(n + m)) 1+ffl ) algorithm to decide whether the part in a specific orientation will safely move across the trap or will fall through the trap and thus be filtered out. Furthermore, we show how to design various parameterized traps, ranging from simple gaps to arbitrary polygons which will filter out all but one of the different stable orientations of a given part. 1 Introduction Geometric algorithms are important tools to solve or give insight into real world problems. In this paper, we transform the problem of designing traps into geometric problems, which are then solved with the use of techniques from computational geometry. ... Robert-Paul Berretty, Kenneth Y. Goldberg, Mark H. Overmars, A. Frank van der Stappen |
SCG | 2 |
| 1999 | Trap Design for Vibratory Bowl FeedersabstractThe vibratory bowl feeder is the oldest and still most common approach to the automated feeding (orienting) of industrial parts. We consider a class of vibratory bowl filters that can be described by removing polygonal sections from the track; we refer to this class of filters as traps. For an n-sided convex polygonal part and m-sided convex polygonal trap, we give an O((n+m)log(n+m)) algorithm to decide if the part will be rejected by the trap, and an O((nm(n+m))/sup 1+/spl epsiv//) algorithm which deals with non-convex parts and traps. We then consider the problem of designing traps for a given part, and consider two rectilinear subclasses, balconies and gaps. We give linear and O(n/sup 2/) algorithms for designing feeders and have tested the results with physical experiments using a commercial inline vibratory feeder. Robert-Paul Berretty, Kenneth Y. Goldberg, Lawrence Cheung, Mark H. Overmars, Gordon Smith, A. Frank van der Stappen |
ICRA | 2 |
| 1999 | Computing Parallel-Jaw GripsabstractWe propose a new model for gripping parts with an industrial parallel-jaw gripper. In contrast to many previous models, we use two grip points and define candidate grips that are resistant to slipping and torque about the part's center of mass. These grips must also be accessible and robust to perturbations in part position. We give an O(n/sup 3/) algorithm for computing and ranking O(n/sup 2/) such grips on an n-sided polygonal slice through the part. The algorithm will be part of a design and simulation system that can rapidly provide feedback to designers; thus it must run quickly and reliably. We have also implemented the algorithm in a Java applet with a graphical user interface that allows Internet users to define a part; the applet computes, ranks, and displays the set of computed grips. Gordon Smith, Kenneth Y. Goldberg, Karl-Friedrich Böhringer, John Craig |
ICRA | 3 |
| 1999 | Jester 2.0: Evaluation of an New Linear Time Collaborative Filtering Algorithm (poster abstract)abstractNo abstract available. Dhruv Gupta 0003, Mark Digiovanni, Hiro Narita, Kenneth Y. Goldberg |
SIGIR | 4 |
| 1999 | Jester 2.0: Collaborative Filtering to Retrieve Jokes (demonstration abstract)
Dhruv Gupta 0003, Mark Digiovanni, Hiro Narita, Kenneth Y. Goldberg |
SIGIR | 4 |
| 1999 | Part pose statistics: estimators and experimentsabstractMany of the most fundamental examples in probability involve the pose statistics of coins and dice as they are dropped on a flat surface. For these parts, the probability assigned to each stable face is justified based on part symmetry, although most gamblers are familiar with the possibility of loaded dice. In industrial part feeding, parts also arrive in random orientations. We consider the following problem: given part geometry and parameters such as center of mass, estimate the probability of encountering each stable pose of the part. We describe three estimators for solving this problem for polyhedral parts with known center of mass. The first estimator uses a quasistatic motion model that is computed in time O(n log n) for a part with n vertices. The second estimator has the same time complexity but takes into account a measure of dynamic stability based on perturbation. The third estimator uses repeated Monte Carlo experiments with a mechanics simulation package. To evaluate these estimators, we used a robot and computer vision system to record the pose statistics based on 3595 physical drop experiments with four different parts. We compare this data to the results from each estimator. We believe this is the first paper to systematically compare alternative estimators and to correlate their performance with statistically significant experiments on industrial parts. Kenneth Y. Goldberg, Brian Mirtich, John Craig, Brian Carlisle, John F. Canny |
IEEE Trans. Robotics Autom. | 1 |
| 1998 | Parallel Microassembly with Electrostatic Force FieldsabstractMicroscopic (submillimeter) parts are often fabricated in parallel at high density but must then be assembled into patterns with lower spatial density. We propose a new approach to microassembly using: 1) ultrasonic vibration to eliminate friction and adhesion; and 2) electrostatic forces to position and align parts in parallel. We describe experiments on the dynamic and frictional properties of collections of microscopic parts under these conditions. We first demonstrate that ultrasonic vibration can be used to overcome adhesive forces; we also compare part behavior in air and vacuum. Next, we demonstrate that parts can be positioned and aligned using a combination of vibration and electrostatic forces. Finally, we demonstrate part sorting by size. Our goal is a systematic method for designing implementable planar force fields for microassembly based on part geometry. Karl-Friedrich Böhringer, Kenneth Y. Goldberg, Michael Cohn, Roger Howe, Albert P. Pisano |
ICRA | 2 |
| 1998 | Estimating and optimizing throughput of a robotic part feeder using queueing theoryabstractWe study a programmable robotic part feeder that relies on a sequence of three conveyor belts to singulate and re-circulate parts. In industrial practice, belt speeds are set in an ad-hoc fashion. Experience with real feeders reveals that throughput can suffer due to (1) starvation where no parts are visible to the camera and (2) saturation, where too many parts are visible, which prevents identifying part pose or grasping due to obstruction by nearby parts. This motivates our search for a systematic approach to setting belt speeds. This paper introduces models based on a 2D Poisson process for both intermittent and continuous motion feeding. For intermittent motion feeding we apply renewal theory to approximate and optimize the theoretical throughput. For continuous motion feeding we use a M/G/1 queue with customer impatience to approximate and optimize the theoretical throughput. We show that the analytic theory compares very well with simulation studies. For both models we show how to optimize the throughput when there is a constraint on the expected number of times a part should go through the system. Dadi Gudmundsson, Kenneth Y. Goldberg |
IROS | 2 |
| 1998 | Computing fence designs for orienting parts
Robert-Paul Berretty, Kenneth Y. Goldberg, Mark H. Overmars, A. Frank van der Stappen |
Comput. Geom. | 2 |
| 1997 | On Fence Design and the Complexity of Push Plans for Orienting PartsabstractA common task in automated manufacturing processes is to orient parts prior to assembly. We consider sensorless orientation of a polygonal part by a sequence of push actions. We show that any polygonal part can be oriented by a sequence of fences placed along a conveyor belt, thereby settling a conjecture by Wiegley et al. [25], and present the first polynomial-time algorithm to compute the shortest such sequence. The algorithm is simple and runs in time O(n 3 log n), where n is the number of vertices of the part. Even though pathological parts can be constructed that require \\Omega\\Gamma n) push actions, it turns out that almost all parts can be oriented using a small number of pushes. We deduce a new bound on the length of the shortest push plan that depends on the thinness -- or eccentricity -- of the part. The bound shows that only O(1) pushes are required for the large class of parts with non-square minimum-width bounding boxes. 1 Introduction Many automated manufacturing pro... Robert-Paul Berretty, Kenneth Y. Goldberg, Mark H. Overmars, A. Frank van der Stappen |
SCG | 2 |
| 1997 | Tuning robotic part feeder parameters to maximize throughputabstractWe study a programmable robotic part feeder that relies on a sequence of three conveyor belts to separate and re-circulate parts. In industrial practice, belt speeds are set in an ad-hoc fashion. Experience with real feeders reveals that throughput can suffer due to (1) starvation where no parts are visible to the camera and (2) saturation, where too many parts are visible, which prevents identifying part pose or grasping due to obstruction by nearby parts. This motivates our search for a systematic approach to setting belt speeds. Our goal is to optimize throughput, measured in terms of how many parts per second are delivered from the robotic feeder. We describe a planar model of the belts with a Poisson arrival process to stochastically model how belt speeds affect throughput. Initially, we study the finite case where N parts are placed into the feeder and re-circulated until they are all delivered by the robot. Our first insight is that the vision belt should be run at maximum achievable velocity. We run simulations to empirically determine optimal buffer belt velocity as a function of lot size. Finally, we develop a theoretical model for the case where N=/spl infin/ which approximates common usage where the buffer is replenished before it becomes empty. From this model, we derive the optimal buffer belt velocity and show that it produces throughput five times greater than that achieved with ad-hoc settings. Dadi Gudmundsson, Kenneth Y. Goldberg |
ICRA | 2 |
| 1997 | Simplifying complex CAD geometry with conservative bounding contoursabstractMegabytes of data are often required for today's detailed CAD models. Users are often willing to sacrifice some of this detail for efficient rendering, transmission over networks, and robot motion planning. For example, many robotics and automation applications such as collision checking require only the simple bounding envelope of a complex set of nested CAD models. In contrast to approximate graphics rendering, these applications require that the simplified envelope be conservative: that the true CAD model be strictly contained inside it. One conservative bounding envelope is the bounding box. Another is the convex hull. Generally each conservative bounding envelope can be "trimmed" by a conservative bounding contour (CBC) of its 2D projection. This yields a hierarchy of conservative bounding envelopes of the original CAD model. In this paper we describe a method for generating the CBC at different levels of simplification. We give an algorithm that computes the CBC of a given 2D projection of a wireframe model. Since the wireframe requires neither boundary representation (BREP) nor constructive solid geometry (CSG) our method is compatible with almost all existing CAD standards. Kenneth Y. Goldberg, Mary Pickett |
ICRA | 2 |
| 1997 | Analysis of part motion on a longitudinally vibrating plateabstractWe analyze the dynamics of part motion for a novel type of planar parts feeder consisting of a longitudinally vibrating flat plate and a part placed on its surface. For each vibration cycle, the plate's velocity is held positive (forward motion) for a longer time than it is held negative (backward motion). This type of asymmetric vibration combined with the nonlinear nature of Coulomb friction causes the part to accelerate along a straight line to a terminal velocity called the "feed rate". The average force exerted by the plate on the part is shown to be proportional to the latter's deviation from the feed rate. In other words, the part behaves as if it were immersed in a forward moving viscous fluid. Expressions for the feed rate and viscosity constants are derived with respect to various physical and control parameters. Rigid-body dynamic simulation results are shown to be in good agreement with the analysis. Dan Reznik, John F. Canny, Kenneth Y. Goldberg |
IROS | 3 |
| 1997 | Interfacing reality: exploring emerging trends between humans and machines (panel)abstractArticle Interfacing reality (panel): exploring emerging trends between humans and machines Share on Authors: Eric Paulos University of California, Berkeley University of California, BerkeleyView Profile , John Canny University of California, Berkeley University of California, BerkeleyView Profile , Eduardo Kac School of the Art Institute of Chicago School of the Art Institute of ChicagoView Profile , Ken Goldberg University of California, Berkeley University of California, BerkeleyView Profile , Mark Pauline Survival Research Laboratories Survival Research LaboratoriesView Profile , Stelarc Performance Artist Performance ArtistView Profile Authors Info & Claims SIGGRAPH '97: Proceedings of the 24th annual conference on Computer graphics and interactive techniquesAugust 1997 Pages 448–451https://doi.org/10.1145/258734.258904Online:03 August 1997Publication History 0citation384DownloadsMetricsTotal Citations0Total Downloads384Last 12 Months0Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access Eric Paulos, John F. Canny, Eduardo Kac, Kenneth Y. Goldberg, Mark Pauline |
SIGGRAPH | 4 |
| 1997 | FixtureNet: interactive computer-aided design via the World Wide Web
Rick Wagner, Giuseppe Castanotto, Kenneth Y. Goldberg |
Int. J. Hum. Comput. Stud. | 3 |
| 1996 | Compiling assembly plans into hard automationabstractFor high-volume assembly, "hard automation" systems consisting of a pipelined series of simple operations are often preferred over multi-axis robots for reasons of cost, speed, and precision. Given a desired final assembly, designing an appropriate assembly system requires the intuition of experienced designers who require up to six months to produce a design and may overlook a better design or fail to find a design when one exists. Thus there is a need for CAD tools that can assist the designers of assembly hardware and to give rapid feedback to product designers to facilitate design for assembly. We introduce a class of vertical assembly (VA) plans such that any valid VA plan can be "compiled" into a hard automation assembly system. VA plans consist of vertical insertions alternated with rotations such that parts and subassemblies at each stage sit stably on conveyor belts without fixtures. For an assembly with n vertices, we can decide if a VA plan exists in time O(n/sup 4/). We describe a complete and implemented planning algorithm that takes as input a 2D slice of an assembly consisting of polygonal parts and returns all valid VA plans ranked by a quality metric. Kenneth Y. Goldberg, Hadi Moradi |
ICRA | 1 |
| 1996 | Estimating pose statistics for robotic part feedersabstractIn automated assembly lines, part feeders often impose a bottleneck that restricts throughput. To facilitate the design of parts and assembly lines, the authors estimate feedrates based on CAD models of parts. A previous paper (Golberg and Craig, 1995) described how to predict throughput for a vision-based robotic part feeder given the distribution of part poses when parts are randomly dropped on a conveyor belt. Estimating this distribution is also useful for the design of traditional feeders such as vibratory bowls. In this paper the authors describe three algorithms for estimating pose distributions. The authors review the quasi-static estimate reported in Wiegley et al. (1992) and introduce a refinement that takes into account some measure of dynamic stability. The perturbed quasi-static estimate can be computed very rapidly and is more accurate than the quasi-static. Still more accurate are estimates based on Monte Carlo simulation using Impulse, although the latter comes at the penalty of increased computation time. The authors compare estimates from all three algorithms with physical experiments. Brian Mirtich, Kenneth Y. Goldberg, John Craig, Rob Zanutta, Brian Carlisle, John F. Canny |
ICRA | 3 |
| 1996 | A complete algorithm for designing passive fences to orient partsabstractPeshkin and Sanderson (1988) showed that parts can be aligned as they move on a conveyor belt against a passive sequence of fences. In this paper the authors describe the first complete algorithm to design such sequences for a given convex polygonal part. The algorithm is complete in the sense that it is guaranteed to find a design if one exists and to terminate with a negative report otherwise. Based on an exact breadth-first search of the design space, the algorithm is also guaranteed to find the design requiring the fewest fences. The authors describe the algorithm and compare results with those previously reported. The authors conjecture that a fence design exists to orient any convex polygonal part defined by a sequence of rational vertices. Jeff Wiegley, Kenneth Y. Goldberg, Michael A. Peshkin, Mike Brokowski |
ICRA | 2 |
| 1996 | Computing Grasp Functions
Anil S. Rao, Kenneth Y. Goldberg |
Comput. Geom. | 2 |
| 1996 | A complete algorithm for designing planar fixtures using modular componentsabstractWe present an implemented algorithm that accepts a polygonal description of a part's silhouette, and efficiently constructs the set of all feasible fixture designs that kinematically constrain the part in the plane. Each fixture is composed of three locators rigidly attached to the lattice and one sliding clamp, and constrains the part without relying on friction. The algorithm is based on an efficient enumeration of admissible designs that exploits part geometry and graphical force analysis. The algorithm run time is linear in the number of designs found, which is bounded by a polynomial in the number of part edges and the part's maximal diameter in lattice units. Our review of the literature suggests that this is the first fixturing algorithm that is complete in the sense that it is guaranteed to find all admissible fixture designs for an arbitrary polygonal silhouette and to identify the optimal fixture relative to an arbitrary quality metric. The algorithm does not consider out-of-plane forces or motions; however, we view this planar result as an essential component of a larger algorithm that solves the 3D fixture design problem by treating the planar and out-of-plane constraint problems separately. This approach is analogous to the widely used 3-2-1 fixture design heuristic. Randy C. Brost, Kenneth Y. Goldberg |
IEEE Trans. Robotics Autom. | 2 |
| 1996 | Complete algorithms for feeding polyhedral parts using pivot graspsabstractTo rapidly feed industrial parts on an assembly line, Carlisle et al. (1994) proposed a flexible part feeding system that drops parts on a flat conveyor belt, determines position and orientation of each part with a vision system, and then moves them into a desired orientation. When a part is grasped with two hard finger contacts and lifted, it pivots under gravity into a stable configuration. The authors refer to the sequence of picking up the part, allowing it to pivot, and replacing it on the table as a pivot grasp. The authors show that under idealized conditions, a robot arm with four degrees of freedom (DOF) can move (feed) parts arbitrarily in 6 DOF using pivot grasps. This paper considers the following planning problem: Given a polyhedral part shape, coefficient of friction, and a pair of stable configurations as input, find pairs of grasp points that will cause the part to pivot from one stable configuration to the other. For a part with n faces and m stable configurations, the authors give an O(m/sup 2/n log n) algorithm to generate the m/spl times/m matrix of pivot grasps. When the part is star-shaped, this reduces to O(m/sup 2/n). Since pivot grasps may not exist for some transitions, multiple steps may be needed. Alternatively, the authors consider the set of grasps where the part pivots to a configuration within a "capture region" around the stable configuration; when the part is released, it will tumble to the desired configuration. Both algorithms are complete in that they are guaranteed to find pivot grasps when they exist. Anil S. Rao, David J. Kriegman, Kenneth Y. Goldberg |
IEEE Trans. Robotics Autom. | 3 |
| 1995 | Complete Algorithms for Reorienting Polyhedral Parts Using a Pivoting GripperabstractNo abstract available. Anil S. Rao, David J. Kriegman, Kenneth Y. Goldberg |
SCG | 3 |
| 1995 | Sensorless manipulation using transverse vibrations of a plateabstractThe existing industrial parts feeders move the parts through a sequence of mechanical filters that reject parts in unwanted orientations. In this paper we develop a new setup that uses a different vibratory mechanism to systematically manipulate parts, by actively orienting and localizing them. The idea is to generate and change dynamic modes for a plate by varying the applied frequency of oscillation. Depending on the node shapes of the plate for these frequencies, the position and orientation of the parts can be controlled. We develop an analysis of the underlying dynamics, and show that it can be used to predict the behavior of objects placed on the vibrating plate. Using this analysis, we propose that the applied frequencies can be automatically sequenced to obtain a "sensorless" strategy for manipulating a given object. Karl-Friedrich Böhringer, Vivek Bhatt, Kenneth Y. Goldberg |
ICRA | 3 |
| 1995 | Destop Teleoperation via the World Wide WebabstractWe built a system that allows a robot manipulator to be teleoperated via the WWW. Although the field of teleoperation dates back over 50 years, the WWW provides a low-cost and widely-available interface that can make teleoperated resources accessible to anyone with a desktop or laptop computer and modem. The "Mercury Project" consists of an industrial robot arm fitted with a CCD camera and a pneumatic system. We placed a sandbox filled with buried artifacts in the robot workspace. Using the ISMAP feature of HTTP, users can remotely move the camera to view desired locations or direct a short burst of compressed air into the sand to view the newly cleared region. To our knowledge, the Mercury Project is the first system to permit WWW users to remotely view and alter the real world. This paper focuses on interface design, robot hardware, and system architecture. Kenneth Y. Goldberg, Michael Mascha, Steve Gentner, Nick Rothenberg, Carl Sutter, Jeff Wiegley |
ICRA | 1 |
| 1995 | Complete Algorithms for Reorienting Polyhedral Parts Using a Pivoting GripperabstractTo rapidly feed industrial parts on an assembly line, Carlisle et. al. (1994) proposed a flexible part feeding system that drops parts on a flat conveyor belt, determines the pose of parts with a vision system and manipulates them into a desired pose. A robot arm with 4-DOF is capable of moving parts through 6-DOF when equipped with a passive pivoting axis between the parallel jaws of its gripper. We refer to these actions as pivot grasps. This paper considers the planning problem. Given a polyhedral part shape, coefficient of friction and a pair of stable configurations as input, find pairs of grasp points that will cause the part to pivot from one stable configuration to the other. For some transitions, pivot grasps may not exist. For a part with n faces and m stable configurations, we give an O(m/sup 2/n log n) algorithm to generate the m/spl times/m matrix of pivot grasps. When the part is star shaped, this reduces to O(m/sup 2/n). We also study a generalization that considers "capture regions" around stable configurations. Both algorithms are complete in that they are guaranteed to find pivot grasps when they exist. Anil S. Rao, David J. Kriegman, Kenneth Y. Goldberg |
ICRA | 3 |
| 1995 | Beyond the Web: Manipulating the Real World
Kenneth Y. Goldberg, Michael Mascha, Steve Gentner, Jürgen Rossman, Nick Rothenberg, Carl Sutter, Jeff Wiegley |
Comput. Networks ISDN Syst. | 1 |
| 1995 | Sorting parts by random graspingabstractAs a low-cost alternative to machine vision, the authors consider how a modified parallel-jaw gripper can be used to classify parts according to shape by grasping and measuring the diameter: the distance between the jaws. Since more than one part may give rise to the same diameter and the sensor may be corrupted by noise due to surface compliance and backlash, the authors show how the most probable part can be estimated using a sequence of random grasps with a Bayesian decision procedure. This procedure allows the authors to define a statistical measure of the "similarity" of a set of parts. Laboratory experiments confirm that the random strategy is effective for sorting parts.> Dukhyun Kang, Kenneth Y. Goldberg |
IEEE Trans. Robotics Autom. | 2 |
| 1995 | Manipulating algebraic parts in the planeabstractWhen manipulating parts, it is important to determine the orientation of the part with respect to the gripper. This orientation may not be known precisely or may be disturbed by the act of grasping. In some cases, it is possible to use mechanical compliance to orient parts during grasping. Goldberg (1993) showed that any part with polygonal boundary can be oriented and grasped in this manner using a parallel-jaw gripper. Many of the curves currently used in engineering design are algebraic but nonlinear. Although these curves can be approximated as polygons for the purpose of visualization, such approximations can lead to false conclusions about mechanical behavior. In this paper we consider the class of parts whose planar projection has a piecewise algebraic convex hull. Our primary result is a proof that a grasp plan exists for any such part. We give a planning algorithm that produces the shortest plan and runs in time O(n/sup 2/ log n+N), where n is the number of transitions in the grasp function and N is the length of the plan produced. We believe this to be the first paper to address the problem of manipulating algebraic parts when nothing is know about their initial orientation.> Anil S. Rao, Kenneth Y. Goldberg |
IEEE Trans. Robotics Autom. | 2 |
| 1994 | A Complete Algorithm for Synthesizing Modular Fixtures for Polygonal PartsabstractCommercially-available modular fixturing systems typically include a square lattice of tapped and bushed holes with precise spacing and an assortment of precision locating and clamping elements that can be rigidly attached to the lattice using dowel pins or expanding mandrels. Currently, human expertise is required to synthesize a suitable arrangement of these elements to hold a given part. Besides being time consuming, if the set of alternatives is not systematically explored, the designer may fail to find an acceptable fixture or may settle upon a suboptimal fixture. The authors consider a class of modular fixtures that prevent a part from translating or rotating in the plane using four point contacts on the part's boundary. These fixtures are based on three round locators, each centered on a lattice point, and one translating clamp. The authors present an algorithm that accepts a polygonal part shape as input and synthesizes the set of all fixture designs that achieve form closure for the given part. The algorithm also allows the user to specify geometric access constraints on fixtures. If the part has n edges and its maximal diameter is d lattice units, the asymptotic running time of the algorithm is O(n/sup 5/d/sup 5/). The authors have implemented the algorithm and present example fixtures that it has synthesized. This implementation includes a metric to rank fixtures based on their ability to resist applied forces. The authors believe this is the first fixture synthesis algorithm that is complete in the sense that it is guaranteed to find an admissible fixture if one exists. Furthermore, the algorithm is guaranteed to find the optimal fixture, relative to any well-defined quality metric.> Randy C. Brost, Kenneth Y. Goldberg |
ICRA | 2 |
| 1994 | "RISC" for Industrial Robotics: Recent Results and Open ProblemsabstractAt the intersection of robotics, computational geometry, and manufacturing engineering, we have identified collection of research problems with near-term industrial applications. The common thread is robot systems with reduced intricacy in sensing and control (RISC), such as light beam sensors and parallel-jaw grippers. We conjecture that such systems, coupled with appropriate algorithms, are capable of recognizing and orienting a broad class of industrial parts. When compared with general-purpose robots, the resulting systems could be: (1) lower in cost, (2) more reliable, and (3) easier to reconfigure. The proposed hardware bears a close resemblance to existing "hard" automation; what is new is the application of computational methods for robust design and control of these systems, and more extensive use of simple sensors. By focusing on a small vocabulary of simple hardware, planning become computationally tractable and we can in some cases make guarantees about the existence of solutions. We borrowed the RISC acronym from computer architecture to acknowledge a common theme: identifying a minimal set of hardware primitives and matching these primitives with highly efficient software. In this paper, we review recent algorithms for locating, feeding, inserting and fixturing industrial parts. We discuss related work and propose a set of open problems for future research.> John F. Canny, Kenneth Y. Goldberg |
ICRA | 2 |
| 1994 | A Pivoting Gripper for Feeding Industrial PartsabstractTo be cost effective and highly precise, many industrial assembly robots have only four degrees of freedom (DOF) plus a binary pneumatic gripper. Such robots commonly permit parts to be rotated only about a vertical axis. However it is often necessary to reorient parts about other axes prior to assembly. In this paper the authors describe a way to orient parts about an arbitrary axis by introducing a rotating bearing between the jaws of a simple gripper. Based on this mechanism, the authors are developing a rapidly configurable vision-based system for feeding parts. In this system, a camera determines initial part pose; the robot then reorients the part to achieve a desired final pose. The authors have implemented a prototype version in their laboratory using a commercially-available robot system.> Brian Carlisle, Kenneth Y. Goldberg, Anil S. Rao, Jeff Wiegley |
ICRA | 2 |
| 1994 | On the Existence of Modular FixturesabstractModular fixtures are gaining wide use for flexible manufacturing and job shop machining. A modular fixture is an arrangement of fixture elements (fixels) that will locate and securely hold a given part. Typically, a human combines intuition with trial-and-error to design fixtures. In some cases designers are unable to design a fixture with given fixels and must resort to custom tooling. It is possible that human designers have overlooked a solution. It is also possible that no solution exists. In this paper the authors explore the existential question: given a fixture model and a part, does a fixture exist that will hold this part in form closure? If so, the authors say that the part is fixturable. The authors consider two classes of fixtures, one using 3 locators and a clamp, the other using 4 clamps. The authors provide one negative result-a class of cross-sections that is not fixturable-and two positive results-two classes of cross-sections that are guaranteed to be fixturable. These results give insight into the application range for different models of modular fixtures.> Kenneth Y. Goldberg, Yiu-Chung Wong |
ICRA | 2 |
| 1993 | Orienting Polygonal Parts Without Sensors
Kenneth Y. Goldberg |
Algorithmica | 1 |
| 1992 | A shape metric for design-for-assemblyabstractThe authors introduce a quantitative shape metric for planar parts, called feedability, based on a stochastic parts feeding algorithm reported by K. Goldberg et al., (1990,1991). Although feedability is only one possible metric for design-for-assembly (DFA), it relates one aspect of assembly cost directly to part geometry. Therefore, based on feedability, a set of geometric redesign strategies can be produced. The application of feedability to DFA is illustrated and compared with other DFA methods.> Gerard Jounghyun Kim, George A. Bekey, Kenneth Y. Goldberg |
ICRA | 3 |
| 1992 | Orienting generalized polygonal partsabstractK.Y. Goldberg (1990) described an algorithm for orienting planar polygonal parts using a modified parallel-jaw gripper. The authors extend this algorithm to handle parts with curved edges. They define a generalized polygon as a planar figure made up of piecewise linear and circular edges, and a generalized polygonal part as one of constant cross section, the convex hull of which is a generalized polygon. For this class of parts a complete algorithm for determining an optimal parts-orienting strategy is presented.> Anil S. Rao, Kenneth Y. Goldberg |
ICRA | 2 |
| 1992 | Shape Recognition By Random graspingabstractOur objective is to recognize parts in a structured environment (such as a factory) using inexpensive and widely available hardware. In this paper we consider the class of parts with constant polygonal cross section. We aim to recover the shape of a part’s cross section by randomly grasping t,he part with a parallel-jaw gripper and measuring the distance between the jaws (the diameter). Since more than one part may give rise to the same diameter and the diameter sensor may be corrupted by noise due to surface compliance and backlaqh, we use a Bayesian decision procedure to estimate the most probable part. We describe preliminary experiments on synthetic and physical data and report on the sensitivity of recognition time to sensor noise, sensor resolution, and probability threshold. Our experiments suggest that even the most unsophisticated grasp strategy - random grasping - can be sufficient for recognizing parts. Dukhyun Kang, Kenneth Y. Goldberg |
IROS | 2 |
| 1991 | Generating stochastic plans for a programmable parts feederabstractA programmable parts feeder, a mechanism that can be reprogrammed to handle differently shaped parts, is discussed. The authors present a planning algorithm that accepts an n-sided polygonal part as input and, in time O(n/sup 2/), generates a program (plan) for the feeder that maximizes expected feedrate. They have implemented the planner and verified some of the resulting plans in the laboratory. This work illustrates a stochastic framework for manipulation planning.> Kenneth Y. Goldberg, Matthew T. Mason, Michael A. Erdmann |
ICRA | 1 |
| 1990 | Bayesian graspingabstractA Bayesian approach to the problem of autonomous manipulation in the presence of state uncertainty is described. Uncertainty is modeled with a probability distribution on the state space. Each plan (sequence of actions) defines a mapping on the state space and hence a posterior probability distribution. An attempt is made to find a plan for optimizing expected performance. The Bayesian framework is applied to a grasping problem. A planar polygon whose initial orientation is described by a uniform distribution and a frictionless parallel-jaw gripper is assumed in order to plan automatically a sequence of open-loop squeezing operations to reduce orientational uncertainty and grasp the object. Although many different performance measures are possible depending on the application, the approach is illustrated by searching for plans that optimize the robot's expected throughput.> Kenneth Y. Goldberg, Matthew T. Mason |
ICRA | 1 |
| 1988 | Using Backpropagation with Temporal Windows to Learn the Dynamics of the CMU Direct-Drive Arm II
Kenneth Y. Goldberg, Barak A. Pearlmutter |
NIPS | 1 |
| 1987 | Conditions for symmetric running in single- and double-supportabstractEven and odd functions of time - symmetric functions - have been used to simplify the control of running robots and have been observed in the body and leg motion of running animals. This paper explores the relationship between symmetric motion of the body and symmetric use of the legs in three planar models. The algebra of symmetric functions is applied to the equations of motion to show that symmetric leg actuation is required for symmetric body motion in the single-support case, but not in the double-support case unless additional constraints are imposed. Kenneth Y. Goldberg, Marc H. Raibert |
ICRA | 1 |