We present an empirical investigation of a new mapping system based on a graph of panoramic depth images. Panoramic images efficiently capture range measurements taken by a spinning lidar sensor, recording fine detail on the order of a few centimeters within maps of expansive scope on the order of tens of millions of cubic meters. The flexibility of the system is demonstrated by running the same mapping software against data collected by hand-carrying a paired lidar and IMU around a laboratory space at walking pace, moving them outdoors through a campus environment at running pace, driving the sensors on a small wheeled vehicle on- and off-road, flying the sensors through a forest, carrying them on the back of a legged robot navigating an underground coal mine, and mounting them on the roof of a car driven on public roads. The full 3D maps are built online with a median update time of less than ten milliseconds on an embedded NVIDIA Jetson AGX Xavier system.

In this work, we present an information based exploration strategy tailored for the generation of high resolution 3D maps. We employ RGBD panoramas because they have been shown to provide memory efficient high quality representations of space. Robots explore the environment by selecting locations with maximal Cauchy-Schwarz Quadratic Mutual Information (CSQMI) computed on an angle enhanced occupancy grid to collect these RGBD panoramas. By employing the angle enhanced occupancy grid, the resulting exploration strategy emphasizes perspective in addition to binary coverage. Furthermore, the goal selection strategy is improved by using image morphology to reduce the search space over which CSQMI is computed. We present experimental results demonstrating the improved performance in perception related tasks by capturing panoramas using this approach, near frontier exploration, and a control of logging images at regular intervals while teleoperating the robot through the workspace. Collect imagery was passed through an object detection library with our perspective aware approach yielding a greater number of successful detections compared to near frontier exploration.

This work demonstrates the use of depth panoramas in the construction of detailed 3D models of extended environments. The paper describes an approach to the acquisition of such panoramic images using a robotic platform that collects sequences of depth images with a commodity depth sensor. These sequences are stitched into panoramic images that efficiently capture scene information while reducing noise in the captured imagery. Scene structure is extracted from the panoramas and used to register a collection of panoramas to obtain coverage of extended areas. The presented approach maintains fine geometry detail in 3D reconstructions, uses a modest amount of memory, and enjoys excellent scaling characteristics across large environments.

Mobile manipulators have brought a new level of flexibility to traditional automation tasks such as tabletop manipulation, but are not yet capable of the same speed and reliability as industrial automation. We present approaches to 3D perception and manipulator motion planning that enable a general purpose robotic platform to recognize and manipulate a variety of objects at a rate of one pick-and-place operation every 6.7 s, and work with a conveyor belt carrying objects at a speed of 33cm/s.

Transport of individual cells or chemical payloads on a subcellular scale is an enabling tool for the study of cellular communication, cell migration, and other localized phenomena. We present a magnetically actuated robotic system for the fully automated manipulation of cells and microbeads. Our strategy uses autofluorescent robotic transporters and fluorescently labeled microbeads to aid tracking and control in optically obstructed environments. We demonstrate automated delivery of microbeads infused with chemicals to specified positions on neurons.

Multi-robot map building has advanced to the point where high quality occupancy grid data may be collected by multiple robots collaborating with only intermittent connectivity. However, the tasking of these agents to most efficiently build the map is a problem that has seen less attention. Unfamiliar, highly cluttered environments can con found exploration strategies that rely solely on occupancy grid frontier identification or even semantic classification methods keyed on geometric features. To reason about partial maps of novel, highly cluttered locations, hypotheses about significant structure in the disposition of free space may be used to guide exploration task assignment. A parsing of map data into places with semantic significance to the exploration task provides a foundation from which one may infer an efficient exploration strategy.

For such biomedical applications as single cell manipulation and targeted delivery of chemicals, it is important to fabricate microstructures that can be powered and controlled without a tether in fluidic environments. In this work, we describe the construction and operation of micronsized, biocompatible ferromagnetic microtransporters driven by external magnetic fields capable of exerting forces at the pico Newton scale. We develop microtransporters using a simple, single step micro fabrication technique that allows us to produce large numbers in the same step. We also fabricate microgels to deliver drugs. We demonstrate that the microtransporters can be navigated to separate individual targeted cells with micron-scale precision, and deliver microgels without disturbing the cells in the neighborhood and the local microenvironment.

This paper presents a scheme which takes as input a 3D point cloud and an associated color image and parses the scene into a collection of salient planar surfaces. The scheme makes use of a fast color segmentation scheme to divide the color image into coherent regions and the groupings suggested by this procedure are used to inform and accelerate a RANSAC based interpretation process. Results on real data sets are presented.

The decomposition of robotics software into a collection of loosely coupled processes has become a core design principle of virtually every large robotics software engineering effort over the past decade. Recently, the ROS software platform from Willow Garage has gained significant traction due to its adoption of sound design principles and significant software library contributions from Willow Garage itself. This paper describes a binding from the Haskell programming language to basic ROS interfaces. The novelty of these bindings is that they allow for, and encourage, a higher level of abstraction in writing programs for robots that treat streams of values as first-class citizens. This approach makes the fusing, transforming, and filtering of streams fully generic and compositional while maintaining full compatibility with the existing ROS ecosystem.

The management of finite resources is central to many robot behaviors. Some robotic systems must maintain invariants regarding the disposition of feet for balancing, others have grippers for manipulating their environments, while yet others must respect strict rules governing the usage of objects in the environment. Yet the specifics of such resource management responsibilities are almost universally locked behind opaque controllers whose lack of type information greatly impedes rigorous static analysis. We present an application of dependent type theory and linear logic for the static analysis of robot behavior programs that manage both robot and environment state, with a worked assembly task example. This approach offers static, formal guarantees with respect to safety requirements attached to primitive actions, as well as introspection of expected state at each step of a scripted sequence of actions allowing for the automatic generation of dynamic, sensor-based, runtime verification of successful execution.

This paper describes a feature based approach to segmenting images into coherent regions. The method draws inspiration from earlier work on randomized projection schemes for approximate nearest neighbor computation. The method proceeds by first computing a descriptor vector for each of the pixels in the image. These vectors are then randomly hashed to yield binary vectors. Salient clusters in the hash space are automatically identified by considering the populations associated with various hash codes. Since the method avoids the explicit vector distance computations associated with other methods, it is very amenable to fast implementation. Experimental results are presented on standard data sets. 1 Introduction and Related Work Segmentation, the problem of breaking an image into coherent regions is, of course, a fundamental problem in Computer Vision. This paper proposes a new approach to the segmentation problem that leverages ideas developed in the Theoretical Computer Science literature to derive a new feature space based clustering algorithm that is amenable to real time implementation.

A functional approach to programming robot swarms brings with it well-defined properties that allow for automated concurrency and distributed execution. Further, the particular expressiveness of a pure functional language with first-class closures captures so cleanly certain biologically-inspired behaviors that program specification often becomes compact enough to allow a programmer to visually inspect the program code for the entire swarm at once. This benefit comes in contrast to more piece-meal construction methods used to build-up robot software from discrete components. While such programming models capture the engineered structure of a robotic system, the dynamic, decentralized qualities sought after in robot swarms are well-matched by the idioms of functional concurrent programming.

Inter-component communication has received considerable attention by robotic software architects as various frameworks and toolkits have matured. While the resulting software platforms have proven useful for abstracting hardware interfaces and the complex networking issues that are often present in robot deployment scenarios, they typically present users with arduous paths for integrating new hardware and software, while making no allowance for humans as peers on the network. We explore how mature standards and protocols born of the Internet community can be leveraged to empower software designed for robots and sensor networks. Here we present software approaches that facilitate the integration of new hardware devices, software platforms, and humans in ways that are natural and intuitive for each

Communication is essential for coordination in most cooperative control and sensing paradigms. In this paper, we present an experimental study of strategies for maintaining end-to-end communication links for tasks such as surveillance and search and rescue where team connectivity is essential for providing situational awareness to a base station. We consider the differences between monitoring point-to-point signal strength versus data throughput and present experimental results with our multi-robot testbed in outdoor environments

We describe our implementation of a distributed sensor database that was designed to support the activities of teams of mobile robots as they explore an environment. Importantly, this approach effectively separates the process of acquiring sensor data from that of exploiting it. This allows us to develop applications where robots and human users can automatically discover and utilize sensor measurements acquired by other robots in the team. We also explain our approach to implementing distributed queries, an important capability that allows us to perform queries in a way that makes best use of the limited available communication bandwidth. Finally, we briefly describe how we have used this system to support situational awareness tasks.

This paper presents ROCI, a framework for developing applications for multi-robot teams. In ROCI, each robot is considered a node, which contains several modules and may export different types of services and capabilities to other nodes. Each node runs a kernel that mediates the interactions of the robots in a team. This kernel keeps an updated database of all nodes and the functionalities that they export. Multi-robot applications can be built dynamically by connecting modules that may be running on different nodes over the network. As an example, we present an obstacle avoidance task implemented using our framework and also discuss the use of ROCI in a multi-robot scenario.