This paper presents an innovative robotic inspection system designed to enhance the detection and analysis of structural defects in concrete infrastructure. The proposed inspection system is comprised of three modules: a robotic data collection module, a visual inspection module, and a subsurface mapping module. The robotic data collection module features an omnidirectional robotic platform, designed to move sideways without spinning. It is equipped with Ground Penetrating Radar (GPR) and RGB-D cameras, facilitating systematic data collection across construction sites. The visual inspection module employs a learning-based method, InspectionNet++, to analyze the frames for surface defects such as cracks, spalls, and stains, providing high accuracy and metric measurements of the defects. The subsurface mapping module processes the GPR data to detect and visualize hidden defects, creating a comprehensive map that correlates these with visible surface anomalies. Field tests demonstrate the system’s ability to automate construction structural inspection with improved efficiency and precision. Additionally, the customized visualization software is introduced to enable intuitive and interactive exploration of the detected defects within a unified interface. By automating data collection and enhancing defect detection through learning algorithms, the system not only speeds up the inspection process but also increases the reliability of infrastructure evaluations, supporting more informed maintenance decisions. Note to Practitioners—This paper introduces a robotic solution for inspection and condition assessment of concrete infrastructure. The system uses an omnidirectional robot equipped with GPR and RGB-D cameras to automatically collect data across construction sites. By harnessing vision-based positioning technology, our system empowers the robot to scan the ground surface in free motion pattern. This eliminates the need for time-consuming grid line setup traditionally required for manual GPR data collection. Our approach combines multi-sensor data analytics with advanced software, which enables the detection and visualization of both surface defects (cracks, spalls, stains) and subsurface anomalies. For practitioners, this automated approach offers several key benefits over manual inspections: increased inspection speed and coverage, rapid data collection enabled by robotic free motion, higher detection accuracy, quantitative defect measurements, and unified visualization correlating surface/subsurface conditions. This system has the potential to revolutionize infrastructure assessment practices. By facilitating more frequent and reliable inspections with minimal human intervention, it paves the way for proactive maintenance and ensures the sustainability of critical infrastructure. A current limitation is the relatively small training dataset for visual inspection, which may affect generalizability across diverse concrete structures. Future work aims to expand the dataset, improve irrelevant feature filtering, and utilize other NDE sensors (e.g., impact echo) for infrastructure inspection.

Ground Penetrating Radar (GPR) is one of the most important non-destructive evaluation (NDE) instruments to detect and locate underground objects (i.e. rebars, utility pipes). Many of the previous researches focus on GPR image-based feature detection only, and none can process sparse GPR measurements to successfully reconstruct a very fine and detailed 3D model of underground objects for better visualization. To address this problem, this paper presents a novel robotic system to collect GPR data, localize the underground utilities, and reconstruct the underground objects’ dense point cloud model. This system is composed of three modules: 1) visual-inertial-based GPR data collection module which tags the GPR measurements with positioning information provided by an omnidirectional robot; 2) a deep neural network (DNN) migration module to interpret the raw GPR B-scan image into a cross-section of object model; 3) a DNN-based 3D reconstruction module, i.e., GPRNet, to generate underground utility model with the fine 3D point cloud. In this paper, both the quantitative and qualitative experiment results verify our method that can generate a dense and complete point cloud model of pipe-shaped utilities based on a sparse input, i.e., GPR raw data, with incompleteness and various noise. The experiment results on synthetic data as well as field test data further support the effectiveness of our approach.

Localization and reconstruction of underground targets, the problem of estimating the position and geometry of the objects from Ground Penetration Radar (GPR), still lies at the core of non-destructive testing (NDT). In this paper, we present MigrationNet, a learning-based approach to detect and visualize subsurface objects. Compared with the existing learning-based method of GPR, our proposed approach could not only detect the hyperbola feature in the raw B-scan image but also interpret hyperbola features into the cross-section image of subsurface pipes. Furthermore, to compare the proposed method with the conventional back-projection methods for GPR data interpretation, a synthetic GPR dataset that mimics the real NDT environment is also introduced in this work. The study indicates the effectiveness of our method, it uses less GPR data for underground pipes reconstruction, produces better GPR imaging results with less computation, and shows the robustness to noise.

Ground Penetrating Radar (GPR) is one of the most important non-destructive evaluation (NDE) devices to detect the subsurface objects (i.e. rebars, utility pipes) and reveal the underground scene. One of the biggest challenges in GPR based inspection is the subsurface targets reconstruction. In order to address this issue, this paper presents a 3D GPR migration and dielectric prediction system to detect and reconstruct underground targets. This system is composed of three modules: 1) visual inertial fusion (VIF) module to generate the pose information of GPR device, 2) deep neural network module (i.e., DepthNet) which detects B-scan of GPR image, extracts hyperbola features to remove the noise in B-scan data and predicts dielectric to determine the depth of the objects, 3) 3D GPR migration module which synchronizes the pose information with GPR scan data processed by DepthNet to reconstruct and visualize the 3D underground targets. Our proposed DepthNet processes the GPR data by removing the noise in B-scan image as well as predicting depth of subsurface objects. For DepthNet model training and testing, we collect the real GPR data in the concrete test pit at Geophysical Survey System Inc. (GSSI) and create the synthetic GPR data by using gprMax3.0 simulator. The dataset we create includes 350 labeled GPR images. The DepthNet achieves an average accuracy of 92.64% for B-scan feature detection and an 0.112 average error for underground target depth prediction. In addition, the experimental results verify that our proposed method improve the migration accuracy and performance in generating 3D GPR image compared with the traditional migration methods.

This paper presents a novel metric inspection robot system using a deep neural network to detect and measure surface flaws (i.e., crack and spalling) on concrete structures performed by a wall-climbing robot. The system consists of four modules: robotics data collection module to obtain RGB-D images and IMU measurement, visual-inertial SLAM module to generate pose coupled key-frames with depth information, InspectionNet module to classify each pixel into three classes (back-ground, crack and spalling), and 3D registration and map fusion module to register the flaw patch into registered 3D model overlaid and highlighted with detected flaws for spatial-contextual visualization. The system enables the metric model of each surface flaw patch with pixel-level accuracy and determines its location in 3D space that is significant for structural health assessment and monitoring. The InspectionNet achieves an average accuracy of 87.64% for crack and spalling inspection. We also demonstrate our InspectionNet is robust to view angle, scale and illumination variation. Finally, we design a metric voxel volume map to highlight the flaw in 3D model and provide location and metric information.

This paper presents a new holistic vision-based mobile assistive navigation system to help blind and visually impaired people with indoor independent travel. The system detects dynamic obstacles and adjusts path planning in real-time to improve navigation safety. First, we develop an indoor map editor to parse geometric information from architectural models and generate a semantic map consisting of a global 2D traversable grid map layer and context-aware layers. By leveraging the visual positioning service (VPS) within the Google Tango device, we design a map alignment algorithm to bridge the visual area description file (ADF) and semantic map to achieve semantic localization. Using the on-board RGB-D camera, we develop an efficient obstacle detection and avoidance approach based on a time-stamped map Kalman filter (TSM-KF) algorithm. A multi-modal human-machine interface (HMI) is designed with speech-audio interaction and robust haptic interaction through an electronic SmartCane. Finally, field experiments by blindfolded and blind subjects demonstrate that the proposed system provides an effective tool to help blind individuals with indoor navigation and wayfinding.

We present a novel collaborative mapping and autonomous parking system for semi-structured multi-story parking garages, based on cooperative 3-D LiDAR point cloud registration and Bayesian probabilistic updating. First, an inertial-enhanced (IE) generalized iterative closest point (G-ICP) approach is presented to perform high accuracy registration for LiDAR odometry, which is loosely coupled with inertial measurement unit using multi-state extended Kalman filter fusion. Second, the IE G-ICP is utilized to reconstruct the 3-D point cloud model for each vehicle, and then the individual model maps are merged and updated into a global probabilistic 2-D grid map. A collaborative multiple layer semantic map is constructed to support autonomous parking. Finally, we propose a collaborative navigation approach for path planning when there are multiple vehicles in the parking garage through vehicle-to-vehicle communication. A global path planner is designed to explore the minimum cost path based on the semantic map, and local motion planning is performed using a random exploring algorithm for obstacle avoidance and path smoothing. Our pilot experimental evaluation provides a proof of concept for indoor autonomous parking by collaborative perception, map merging, and updating methodologies.

This paper introduces GUMS, a complete projection model for omnidirectional stereo vision systems. GUMS is based on the existing generalized unified model (GUM), which we extend in order to satisfy a tight relationship among a pair of omnidirectional views for fixed baseline sensors. We exemplify the proposed model's calibration via a single-camera coaxial omnistereo system in a joint bundle-adjusted fashion. We compare our coupled method against the naive approach where the calibration of intrinsic parameters is first performed individually for each omnidirectional view using existing monocular implementations, to then solve for the extrinsic parameters as an additional step that has no effect on the intrinsic model solutions initially computed. We validate GUMS and its calibration effectiveness using both real and synthetic systems against ground-truth data. Our calibration method proves successful for correcting the unavoidable misalignment present in vertically-configured catadioptric rigs. We also generate 3D point clouds employing the calibrated GUMS systems in order to demonstrate the qualitative outcome of our contribution.

This paper considers the problem of generating dynamically feasible and collision free trajectory for unmanned aerial vehicles(UAVs) in cluttered environments. General random-based searching algorithms output piecewise linear paths, which cause big discrepancy when used as navigation reference for UAVs with high speed. Meanwhile, the disturbance may also occur to lead the UAVs into danger. In order to obtain agile autonomy without potential dangers, this paper introduces a three-step method to generate feasible reference. In the first step, a six-order Bezier curve, which uses Tuning Rotation to decrease the curvature, is introduced to smooth the output of the path planner. Then a forward simulation is implemented to find the potential dangerous regions. Finally, the path is reshaped by local optimal reshaping planner to eliminate residual dangers. The three steps form a circulation, the reshaped path sent to the first step again to check dynamic feasibility and safety. The method combining Six-order Bezier curve, Tuning Rotation, and local optimal reshaping is proposed by us for the first time, where the Tuning Rotation is able to meet various curvature requirements without violating the previous path, local optimal reshaping obtains both temporal and spatial reshaping with high time efficiency. The method addresses the system dynamics to achieve agile autonomy, which provides the geometry reference as well as the low level control. The effectiveness of the proposed method is demonstrated by the simulations.

This paper proposes a novel assistive navigation system based on simultaneous localization and mapping (SLAM) and semantic path planning to help visually impaired users navigate in indoor environments. The system integrates multiple wearable sensors and feedback devices including a RGB-D sensor and an inertial measurement unit (IMU) on the waist, a head mounted camera, a microphone and an earplug/speaker. We develop a visual odometry algorithm based on RGB-D data to estimate the user's position and orientation, and refine the orientation error using the IMU. We employ the head mounted camera to recognize the door numbers and the RGB-D sensor to detect major landmarks such as corridor corners. By matching the detected landmarks against the corresponding features on the digitalized floor map, the system localizes the user, and provides verbal instruction to guide the user to the desired destination. The software modules of our system are implemented in Robotics Operating System (ROS). The prototype of the proposed assistive navigation system is evaluated by blindfolded sight persons. The field tests confirm the feasibility of the proposed algorithms and the system prototype.

This paper lays the ground work for assistive navigation using wearable sensors and social sensors to foster situational awareness for the blind. Our system acquires social media messages to gauge the relevant aspects of an event and to create alerts. We propose social semantics that captures the parameters required for querying and reasoning an event-of-interest, such as what, where, who, when, severity, and action from the Internet of things, using an event summarization algorithm. Our approach integrates wearable sensors in the physical world to estimate user location based on metric and landmark localization. Streaming data from the cyber world are employed to provide awareness by summarizing the events around the user based on the situation awareness factor. It is illustrated using disaster and socialization event scenarios. Discovered local events are fed back using sound localization so that the user can actively participate in a social event or get early warning of any hazardous events. A feasibility evaluation of our proposed algorithm included comparing the output of the algorithm to ground truth, a survey with sighted participants about the algorithm output, and a sound localization user interface study with blind-folded sighted participants. Thus, our framework supports the navigation problem for the blind by combining the advantages of our real-time localization technologies so that the user is being made aware of the world, a necessity for independent travel.

This paper provides a framework for context-aware navigation services for vision impaired people. Integrating advanced intelligence into navigation requires knowledge of the semantic properties of the objects around the user's environment. This interaction is required to enhance communication about objects and places to improve travel decisions. Our intelligent system is a human-in-the-loop cyber-physical system that interprets ubiquitous semantic entities by interacting with the physical world and the cyber domain, viz., 1) visual cues and distance sensing of material objects as line-of-sight interaction to interpret location-context information, and 2) data (tweets) from social media as event-based interaction to interpret situational vibes. The case study elaborates our proposed localization methods (viz., topological, landmark, metric, crowdsourced, and sound localization) for applications in way finding, way confirmation, user tracking, socialization, and situation alerts. Our pilot evaluation provides a proof of concept for an assistive navigation system.

In this paper we present a navigation system for Micro Aerial Vehicles (MAV) based on information provided by a visual odometry algorithm processing data from an RGB-D camera. The visual odometry algorithm uses an uncertainty analysis of the depth information to align newly observed features against a global sparse model of previously detected 3D features. The visual odometry provides updates at roughly 30 Hz that is fused at 1 KHz with the inertial sensor data through a Kalman Filter. The high-rate pose estimation is used as feedback for the controller, enabling autonomous flight. We developed a 4DOF path planner and implemented a real-time 3D SLAM where all the system runs on-board. The experimental results and live video demonstrates the autonomous flight and 3D SLAM capabilities of the quadrotor with our system.

The fleet size of a multi-robot system is an important parameter to be considered for real robotics applications since it will determine the cost and the time of execution of any given task. Unfortunately, it is a topic that has received little attention in the robotics literature. The study of the fleet size will allow for the design and implementation of more effective techniques and coordination methods for multi-robot systems. In this paper we study the effects of the fleet size on the time of exploration of a structured environment. We present an analysis that allows us to specify the maximum fleet size that provides the maximum reduction on the exploration time when the structured environment is modeled as a tree. The analysis is applied to the Multi-Robot Depth First Search (MR-DFS) algorithm that allows for maximum parallelism when an exploration process starts from a single point. The analysis provides an expression for the average time of exploration of a tree and for the maximum number of robots that produces a significant reduction on the exploration time.

An RGB-D camera is a sensor which outputs color and depth and information about the scene it observes. In this paper, we present a real-time visual odometry and mapping system for RGB-D cameras. The system runs at frequencies of 30Hz and higher in a single thread on a desktop CPU with no GPU acceleration required. We recover the unconstrained 6-DoF trajectory of a moving camera by aligning sparse features observed in the current RGB-D image against a model of previous features. The model is persistent and dynamically updated from new observations using a Kalman Filter. We formulate a novel uncertainty measure for sparse RGD-B features based on a Gaussian mixture model for the filtering stage. Our registration algorithm is capable of closing small-scale loops in indoor environments online without any additional SLAM back-end techniques.

This paper presents a wall-climbing robot for reconnaissance in anti-hijacking application. A novel biped-wheel hybrid locomotion mechanism is proposed, which is composed of a planetary gear train, a vacuum adhesion module and a negative pressure adhesion module. The bipedal, wheeled and hybrid locomotion modes are analyzed respectively. A prototype of the wall-climbing robot with compact size and low power consumption has been developed and a lot of performance tests have been conducted. The experimental results demonstrate that the wall-climbing robot has such characteristics as fast moving speed, excellent surface adaptability and obstacle negotiation capability.

The aim of this paper is to design an inexpensive conceivable wearable navigation system that can aid in the navigation of a visually impaired user. A novel approach of utilizing the floor plan map posted on the buildings is used to acquire a semantic plan. The extracted landmarks such as room numbers, doors, etc act as a parameter to infer the way points to each room. This provides a mental mapping of the environment to design a navigation framework for future use. A human motion model is used to predict a path based on how real humans ambulate towards a goal by avoiding obstacles. We demonstrate the possibilities of augmented reality (AR) as a blind user interface to perceive the physical constraints of the real world using haptic and voice augmentation. The haptic belt vibrates to direct the user towards the travel destination based on the metric localization at each step. Moreover, travel route is presented using voice guidance, which is achieved by accurate estimation of the user's location and confirmed by extracting the landmarks, based on landmark localization. The results show that it is feasible to assist a blind user to travel independently by providing the constraints required for safe navigation with user oriented augmented reality.

An RGB-D camera is a sensor which outputs range and color information about objects. Recent technological advances in this area have introduced affordable RGB-D devices in the robotics community. In this paper, we present a real-time technique for 6-DoF camera pose estimation through the incremental registration of RGB-D images. First, a set of edge features are computed from the depth and color images. An initial motion estimation is calculated through aligning the features. This initial guess is refined by applying the Iterative Closest Point algorithm on the dense point cloud data. A rigorous error analysis assesses several sets of RGB-D ground truth data via an error accumulation metric. We show that the proposed two-stage approach significantly reduces error in the pose estimation, compared to a state-of-the-art ICP registration technique.

In this paper, we present a multirobot exploration algorithm that aims at reducing the exploration time and to minimize the overall traverse distance of the robots by coordinating the movement of the robots performing the exploration. Modeling the environment as a tree, we consider a coordination model that restricts the number of robots allowed to traverse an edge and to enter a vertex during each step. This coordination is achieved in a decentralized manner by the robots using a set of active landmarks that are dropped by them at explored vertices. We mathematically analyze the algorithm on trees, obtaining its main properties and specifying its bounds on the exploration time. We also define three metrics of performance for multirobot algorithms. We simulate and compare the performance of this new algorithm with those of our multirobot depth first search (MR-DFS) approach presented in our recent paper and classic single-robot DFS.

This paper presents the implementation of an open-source 6-DoF pose estimation system for micro-air vehicles and considers the future implications and benefits of open-source robotics. The system is designed to provide high frequency pose estimates in unknown, GPS-denied indoor environments. It requires a minimal set of sensors including a planar laser range-finder and an IMU sensor. The code is optimized to run entirely onboard, so no wireless link and ground station are explicitly needed. A major focus in our work is modularity, allowing each component to be benchmarked individually, or swapped out for a different implementation, without change to the rest of the system. We demonstrate how the pose estimation can be used for 2D SLAM or 3D mapping experiments. All the software and hardware which we have developed, as well as extensive documentation and test data, is available online.

We present a novel catadioptric-stereo rig consisting of a coaxially-aligned perspective camera and two spherical mirrors with distinct radii in a “folded” configuration. We recover a nearly-spherical dense depth panorama (360°×153°) by fusing depth from optical flow and stereo. We observe that for motion in a horizontal plane, optical flow and stereo generate nearly complementary distributions of depth resolution. While optical flow provides strong depth cues in the periphery and near the poles of the view-sphere, stereo generates reliable depth in a narrow band about the equator. We exploit this principle by modeling the depth resolution of optical flow and stereo in order to fuse them probabilistically in a spherical panorama. To aid the designer in achieving a desired field-of-view and resolution, we derive a linearized model of the rig in terms of three parameters (radii of the two mirrors plus axial separation from their centers). We analyze the error due to the violation of the Single Viewpoint (SVP) constraint and formulate additional constraints on the design to minimize the error. Performance is evaluated through simulation and with a real prototype by computing dense spherical panoramas in cluttered indoor settings.

This video demonstrates the progress of the CityFlyer project. The goal of the project is to develop an autonomous quadrotor helicopter capable of autonomous flight in indoor and outdoor 3D environments such as hallways, stairwells, forests, caves and tunnels. Currently we are focused on developing 3D mapping and navigation capabilities for indoor environments. This video shows our 6 Degree of Freedom pose estimation system, a novel 3D map structure called Multi Volume Occupancy Grids, and the development of a groundstation.

The searching for a location-unknown radio transmitter is a challenging task for autonomous robot. We propose an adaptive searching algorithm named theseus gradient guide (TGG) which is designed for solving the searching problem in indoor environments using received signal strength (RSS). While the RSS gradient serves as the main guide, the robot prefers to move to the places which have never been traveled. Thus the robot will not get stuck in the local maxima. Moreover, unlike the commonly used random kick strategy the TGG drives the robot escaping the local maxima with low cost in terms of travel distance. Meanwhile, TGG is not sensitive to motion errors. Simulation results show that the searches using TGG cost much less compared with those using other gradient based methods in our testing indoor environment. Guided by TGG, the robot can successfully reach the location-unknown radio transmitter with a ratio over 97% when the standard deviation of motion error is up to 20% of the step length.

This paper presents a new method for mapping multi-floor buildings. The method combines laser range sensor for metric mapping and barometric pressure sensor for detecting floor transitions and map segmentation. We exploit the fact that the barometric pressure is a function of the elevation, and it varies between different floors. The method is tested with a real robot in a typical indoor environment, and the results show that physically consistent multi5floor representations are achievable.

In this paper, we present an algorithm for the exploration of an unknown graph by multiple robots, which is never worse than depth-first search with a single robot. On trees, we prove that the algorithm is optimal for two robots. For k robots, the algorithm has an optimal dependence on the size of the tree but not on its radius. We believe that the algorithm performs well on any tree, and this is substantiated by simulations. For trees with e edges and radius r, the exploration time is less than 2e/k + (1 + (k/r))k-1(2/k!)rk-1= (2e/k) + O((k + r)k-1) (for r >; k,k-1), thereby improving a recent method with time O((e/logk) + r) [2], and almost reaching the lower bound max((2e/k), 2r). The model underlying undirected-graph exploration is a set of rooms connected by opaque passages; thus, the algorithm is appropriate for scenarios like indoor navigation or cave exploration. In this framework, communication can be realized by bookkeeping devices being dropped by the robots at explored vertices, the states of which are read and changed by further visiting robots. Simulations have been performed in both tree and graph explorations to corroborate the mathematical results.

In this paper we study the problem of multi-robot exploration of unknown indoor environments that are modeled as trees. Specifically, our approach consider that robots deploy and communicate with active landmarks in every intersection they encounter. We present a novel algorithm that is guaranteed to completely explore any tree with m edges and diameter D, by allowing k robots to be fed into the tree one at a time. We prove that the exploration time of the algorithm grows in linear proportion with the size of the tree and is not bigger than D+m. Simulation results are presented that corroborate the theoretical analysis.

Advancing research into autonomous micro aerial vehicle navigation requires data structures capable of representing indoor and outdoor 3D environments. The vehicle must be able to update the map structure in real time using readings from range-finding sensors when mapping unknown areas; it must also be able to look up occupancy information from the map for the purposes of localization and path-planning. Mapping models that have been used for these tasks include voxel grids, multi-level surface maps, and octrees. In this paper, we suggest a new approach to 3D mapping using a multi-volume occupancy grid, or MVOG. MVOGs explicitly store information about both obstacles and free space. This allows us to correct previous potentially erroneous sensor readings by incrementally fusing in new positive or negative sensor information. In turn, this enables extracting more reliable probabilistic information about the occupancy of 3D space. MVOGs outperform existing probabilistic 3D mapping methods in terms of memory usage, due to the fact that observations are grouped together into continuous vertical volumes to save space. We describe the techniques required for mapping using MVOGs, and analyze their performance using indoor and outdoor experimental data.

Stereo system plays an important role in the navigation of MAVs. In this paper, we design a single-camera catadioptric omnistereo system for MAV, which consists of one hyperboloidal mirror, one hyperboloidal-planar combined mirror, and one conventional camera. System parameters are optimized based on the analysis of constraints and each parameter's influence on performance. Projective model of this system is derived, which provides a foundation for sphere-based calibration algorithm. It calibrates not only the conventional camera parameters, but also the mirror parameters. We also prove that a minimum of two spheres are needed to calibrate the seven parameters.

This video presents a practical navigation scheme for indoor mobile robots using hybrid maps. The method makes use of metric maps for local navigation and a topological map for global path planning. Metric maps are generated as occupancy grids by a laser range finder to represent local information about partial areas. The global topological map is used to indicate the connectivity of the `places-of-interests' in the environment and the interconnectivity of the local maps. Visual tags on the ceiling to be detected by the robot provide valuable information and contribute to reliable localization. The navigation scheme based on the hybrid metric-topological maps saves memory space and is also scalable and adaptable since new local maps can be easily added to the global topology, and the method can be deployed with minimum amount of modification if new areas are to be explored. The video demonstrated that the method is implemented successfully on physical robot in a hospital environment, which provides a practical solution for indoor navigation.

In this paper we present an algorithm for the exploration of an unknown graph with k robots, which is guaranteed to succeed on any graph, and which on trees we prove to be near-optimal for two robots, having optimal dependence on the size of the tree but not on its radius. We believe that the algorithm performs well on any graph, and this is substantiated by simulations. For trees with n edges and radius r, the exploration time is 2n/k + O(rk-1), improving a recent method with O(n/log k + r) [1], and almost reaching the lower bound max (2n/k, 2r). The algorithm is meant to be used in indoor navigation or cave search scenarios where the environment can be modeled as a graph. In this scenario, communication is realized by the devices being dropped by the robots at explored vertices, and the states of which are read and changed by further visiting robots. Simulations on Player/Stage platform have been performed in both tree and graph exploration which corroborate the mathematical results.

This paper presents a novel technique to register a set of two 3D laser scans obtained from a ground robot and a wall-climbing robot which operates on the ceiling to construct a complete map of the indoor environment. Traditional laser scan registration methods like the Iterative Closest Point (ICP) algorithm will not converge to a global minimum without a good initial estimate of the transformation matrix. Our technique uses an overhead camera on the wall-climbing robot to keep line of sight with the ground robot and solves the Perspective Three Point (P3P) Problem to obtain the transformation matrix between the wall-climbing robot and the ground robot, which serves as a good initial estimate for the ICP algorithm to further refine the transformation matrix. We propose a novel particle filter algorithm to identify the real pose of the wall-climbing robot out of up to four possible solutions to P3P problem using Grunert's algorithm. The initial estimate ensures convergence of the ICP algorithm to a global minimum at all times. The simulation and experimental results indicate that the resulting composite laser map is accurate. In addition, the vision-based approach increases the efficiency by reducing the number of iterations of the ICP algorithm.

Biologically-inspired swarm of robots with collaboration towards a common mission has a broad range of applications. However, the required dynamic localization among autonomous robots for such swarm collaboration, though usually implicitly assumed, has not been properly studied. In this paper, we address the roles of multiple antennas in localization and energy-efficient wireless communications for a swarm of robots. Following the gradient of signal powers along a trajectory, a robot can track the direction of a source robot. With three or more properly placed antennas that sense different phase shifts of carrier, a robot can localize a source. By lateration, three collaborative robots can localize a source with known distances to it. Via angulation technique, three robots can determine their geometric relationship with knowing two angles and one distance between them. The techniques can be extended from the 2-D to the 3-D space for application of wall-climbing robots. On the basis of knowledge of robot locations, beamforming techniques can be employed to receive and transmit signal towards the desired robot therefore improving energy efficiency and prolonging robot lifetime.

Finding traversable paths using computer vision is one of the most important components of an intelligent mobile robot system. For a wall climbing robot that operates in an urban environment, it is essential to automatically detect surface types and orientations for switching between moving and climbing, and for applying different adhesive forces both to save energy and ensure its own safety. This paper presents a novel segmentation-based stereovision approach in order to rapidly obtain accurate 3D estimations of urban scenes with largely textureless areas and sharp depth changes. The new approach takes advantage of the fact that many man-made objects in an urban setting consist of planar surfaces. Our approach has three main components: extraction of natural (planar) matching primitives, stereo matching via three-step algorithm (global match, local match and plane fitting), and plane merging and parameter refinement. Experimental results are provided for real indoor scenes.

In this paper, we propose a novel approach of signal power gradient by which a robot adaptively searches a location-unknown sensor. While moving, the robot measures signal strength and estimates the direction of power gradient along which the robot moves in the next step. The correctness of estimated direction is analyzed and the probability of correct direction is obtained. Since the robot continuously measures signal strength while moving, it can effectively overcome the motion errors. Simulation results demonstrate that the robot can successfully reach the location-unknown sensor with probability close to one when the signal to noise ratio at the initial location is as low as 0 dB and the standard deviation of motion error is 10% step size.

This paper presents a novel method to construct a complete 3D map that includes all surfaces (ceiling, wall, and furniture tops, etc.) in indoor environments. A team of four robots, including three ground robots and one wall-climbing robot is deployed in a tetrahedron configuration that satisfies the perspective three point (P3P) problem. P3P problem is to estimate the pose of a perspective camera on the wall-climbing robot viewing three ground robots, which will produce up to four solutions using Grunert's algorithm while only one of them is genuine. We propose a probabilistic Bayesian algorithm that identifies the unique solution of the P3P problem using the mobility of the camera. Based on this technique, we introduce an intra-robot localization method to determine the geometric relationship among four robots. Each ground robot is equipped with a rotary laser range finder (LRF), a pan-tilt-zoom camera, and a LED cluster. The wall-climbing robot is fitted with a LRF, a perspective camera, and a motion sensor. Through the vision sensors, the robots obtain their relative poses by solving the P3P problem. Through the LRF on each robot, 4 laser point cloud maps are produced from each robot's point of view. With the information of relative poses of the multiple robots and the calibration data of each LRF and camera pair, the 4 partial maps are fused to acquire a complete 3D map that is rich with information of all surfaces. Our approach outperforms the traditional range image fusion algorithms in terms of time complexity and is suitable for real-time implementation. Real experiments verified the effectiveness of the method.

The angular vestibulo-ocular and opto-kinetic reflexes (aVOR and OKR) combine to provide compensation for head rotations in space to help maintain a steady image on the retina. We previously implemented an artificial angular vestibulo-ocular reflex with a fully articulated binocular control system in a quadruped robot head. In this paper, we describe the implementation of artificial opto-kinetic and linear vestibular ocular reflexes (OKR and lVOR) that use inputs from an artificial vestibular sensor and a binocular camera system to compensate for linear movements of the head and visual motion to stabilize images on the cameras. The object tracking algorithm was able to fixate a steady object in the camera's field of view during linear perturbations of the robot's head in space at low frequencies of movement (0.2-0.6 Hz), simulating the linear VOR. We implemented an algorithm that combines the linear VOR and OKR model and computes changes in relative pose of the cameras with respect to the object being tracked. The system provides compensatory angular movements of the Ocular Servo Module (OSM) to stabilize images as the robot is moved laterally.

There is a real need in the robotics and wireless sensor network (WSN) communities for the estimation of the geolocation of wireless agents. The received signal strength indicator (RSSI), a common metric in most networking hardware, has been reputed as a very unreliable method for doing the job, due to its vulnerability to environmental factors. Nevertheless, it still remains as the most prevalent estimator of distance between agents on many research projects. Multipath fading, shadowing and other effects that the environment exerts over a signal while propagating are regarded as the main cause of such vulnerability. Although some success has been obtained using RSSI outdoors where the effects are less noticeable, indoor settings remain an unconquered territory. The main motivation of this paper is to establish whether, in real time applications, the use of preprocessing techniques over partial raw collected data helps the RSSI to be a suitable estimator of distance. We propose one such technique and the results suggest that its use may indeed assist the obtainment of more accurate distance estimations while using RSSI.

The video presents recent progress of the wall-climbing robot project at the City College of New York. The robots are named as City-Climbers which adopt a novel adhesive mechanism based on aerodynamic attraction to achieve good balance between strong adhesion force and high mobility. The video demonstrates that the City-Climber robots can operate on virtually any kind of smooth or rough surfaces and have the capabilities to move on the ground, climb walls, and transit between them. The modular design achieves both fast motion of each module on planar surfaces and smooth transition between the surfaces by a set of two modules. The video also displays the Fluent simulation results of the aerodynamic attraction with the aim to optimize the design. DSP-based control system is introduced which enables the robot to operate both manually and autonomously

Studies of primate locomotion have shown that the head and eyes are stabilized in space through the vestibulo-collic and vestibulo ocular reflexes (VCR, VOR). The VOR is a reflex eye movement control system that stabilizes the image on the retina of the eye during head movements in space. This stabilization helps maintain objects of interest approximately fixed on the retina during locomotion. In this paper we present the design and implementation of an artificial vestibular system, which drives a fully articulated binocular vision system for quadrupedal robots to maintain accurate gaze. The complete robot head has 9 degrees of freedom (DOF): pitch, yaw, and roll for the head and 3 DOF for left and right cameras. The SONY AIBOreg quadruped robot has been modified with additional hardware to emulate the vestibular system and the vestibulo-ocular reflex in primates.

This paper presents a new approach to the intralocalization among a team of robots working in constrained 3D space of urban environments. As the base formation, a team of three ground robots and one wall-climbing robot are deployed on ground and on a wall or ceiling, respectively. The three ground robots localize themselves using an existing panoramic vision-based method. However, no existing method can uniquely determine the pose of the climbing robot based on the positions of three ground robots in its image and in the world coordinate systems; up to four valid solutions could exist using known algorithms, although only one is genuine. By carefully examining these methods, two new algorithms for uniquely locating the climbing robot are proposed. The first algorithm makes use of the straight line constraint of robot motion and can uniquely determine the pose of climbing robot by moving the climbing robot straightly for two small steps. The second algorithm is based on the principle of Bayesian filter and take advantage of the motion sensor readings to loose the straight line constraint. The algorithm could continuously determine the climbing robot’s pose after the initial pose is obtained. Extensive simulations are conducted to validate the soundness and robustness of our algorithms. Preliminary experiments are also carried out to examine the feasibility in applying these algorithms in real robot applications.

The vague assumption of a fixed communication range has facilitated the development of major areas of study in mobile sensor networks such as connectivity, coverage, and self-deployment algorithms by providing a stable element as a foundation for their analysis, but more precise communication model is required in order to convert theoretical simulations into realistic applications. This paper studies the effects of more realistic communication models on mobile sensor networks/ad hoc networks taking into consideration multi-access interference and noise and their impact on the communication range of the nodes. The study shows that the communication range changes with the characteristics of the network (network parameters and node distribution). Simulation results justify our findings and future research directions are discussed

Resource-constrained miniature robots require small but high-performance onboard processing unit and reconfigurable electrical hardware to carry out different missions. The advances in field programmable gate array (FPGA) technology offer a system on programmable chip (SoPC) solution to this demand. This paper describes the technical achievement in developing FPGA-based control system which utilizes the hardware/software re-configurable feature of the advanced FPGA device to achieve the goal. We have implemented the hardware module inside the FPGA chip to generate PWM output and to count quadrate encoder pulses, which are the basic building blocks to drive DC motors. Software is developed to implement PID control algorithm using the PWM-encoder module and on-chip processor. The functional correctness of the closed-loop control system is verified by the experimental tests. The performance analysis shows that the hardware module occupies less FPGA space and the power dissipation is very much comparable to other design alternatives of similar caliber. The preliminary results demonstrate that the PWM-encoder module, in the form of user intellectual property (IP), can be duplicated and re-configured to control as many motors as needed. The success gives us a confidence boost to continue our effort to build user IP library for robotics applications which will benefit the robotics community by providing development tools and FPGA building blocks to satisfy the demand for an extremely flexible, high-performance, onboard computing unit

This paper presents a self-localization strategy for a team of heterogeneous mobile robots including ground mobile robots of various sizes and wall-climbing robots. These robots are equipped with various visual sensors, such as miniature webcams, omnidirectional cameras, and PTZ cameras. As the core of this work, a formation of four-robot team is constructed to operate in a 3D space, e.g., moving on ground, climbing on walls and clinging to ceilings. The four robots could dynamically localize themselves asynchronously by employing cooperative vision techniques. Three of them on the ground mutually view each other and determine their relative poses with 6 degrees of freedom (DOFs). A wall-climbing robot, which significantly extends the work space of the robot team to 3D, is at a vantage point (e.g., on the ceiling) such that it can see all the three teammates, thus determining its own location and orientation. The four-robot formation theory and algorithms are presented, and experimental results with both simulated and real image data are provided to demonstrate the feasibility of this formation. Two 3D localization and control strategies are designed for applications such as search and rescue and surveillance in 3D urban environments where robots must be deployed in a full 3D space

In this paper, we investigate the leader following based formation control of multiple nonholonomic mobile robots. We present a new kinematics model for the leader-follower system using Cartesian coordinates rather than the commonly used polar coordinates in literature. Based on this new model and the idea of integrator backstepping, a globally stable controller is derived for the whole system. Simulation results are included to verify the efficacy of the presented new model and controller.

In this paper, we investigate the mechanism of the phenomena of swarm flocking, which can be widely observed in such as schooling fish and flocking migrate birds in nature. By combining the ideas of virtual force and nearest neighborhood law, we propose a decentralized controller to explain such phenomena. This decentralized controller can enable all swarm members to converge to a common velocity with bounded errors, no matter the swarm topology is fixed or dynamic. The advantage of this controller is that it just needs local information to achieve the stable group behavior. Simulation results are included to verify the proposed controller.

This paper presents a fuzzy navigation and motion control system for miniature climbing robots. After the introduction of robot structure, a navigation approach based on a fuzzy multi-sensor data fusion scheme is proposed, which integrates task scheduling, sensing, planning and real-time execution in a unified task reference framework and makes the task synchronization much easier. For a highly nonlinear system as the miniature climbing robot, a fuzzy motion controller with gravity compensation is designed to reduce energy consumption and improve the robot performance at different situations. Experimental results prove the validity of the proposed method.

This paper aims at developing a force-guided micromanipulation technology with in-situ PVDF beam force sensing and hybrid force/position control based on an infinite dimensional system model. By using the designed PVDF force sensing cantilever composite structure with high sensitivity, the micro contact force/impact signal and its derivative can be extracted and processed. As the sensor structure installed at the end of micromanipulator is a soft beam, when manipulation is performed, the cantilever beam is necessary to be considered as a distributed parameter flexible link, then we developed a hybrid micro contact force/position control scheme on the basis of an infinite dimension system model. Experimental results verify the performance of the developed micro force sensing and hybrid control scheme. Ultimately the technology will provide a critical and major step towards the development of automated manufacturing processes for batch assembly of micro devices.

This paper presents a distributed model for cooperative multiple mobile robot systems. In a multiple robot system, each mobile robot has sensing, computation and communication capabilities. The mobile robots spread out across certain area and share sensory information through an ad hoc wireless network. The multiple mobile robot system is therefore a mobile sensor network. In this paper, Voronoi diagram and Delaunay triangulation are introduced to model the area coverage and cooperation of mobile sensor networks. Based on the model, this paper discusses a fault tolerant algorithm for autonomous deployment of the mobile robots. The algorithm enables the system to reconfigure itself such that the area covered by the system can be enlarged. The proposed formation control algorithm allows the mobile sensor network to track moving target and sweep a larger area along specified paths.

This paper presents a fuzzy system approach that incorporates sensing, control and planning to enable micro wall climbing robots navigating in unstructured environments. Based on a fuzzy multi-sensor data fusion scheme, a novel strategy for integrating task scheduling, sensing, planning and real-time execution is proposed so that task scheduling, action planning and motion control can be treated in a unified framework, and the design of task synchronization becomes much easier. A hybrid method is employed to create robot's gaits by switching the robot between continuous motion modes with the help of a finite state machine driven by the signals from robot sensors. For a highly non-linear system as the micro wall climbing robot, a fuzzy motion controller is designed to improve the robot's performance at different situations. Experimental results prove the validity of the proposed method.

This paper presents a fuzzy control system that incorporates sensing, control and planning to improve the performance of the wall-climbing microrobots in unstructured environments. After introduction of the robot system, a task reference method is proposed which is based on a fuzzy multisensor data fusion scheme. The method provides a novel mechanism to efficiently integrate task scheduling, action planning and motion control in a unified framework. A robot gait generation method is described which switches the robot locomotion between different motion modes with the help of a finite state machine driven by sensory information. A fuzzy motion controller is designed to improve control performance and reduce power consumption by the suitable selection of fuzzy sets and inference methods, as well as the definition of corresponding membership functions and control rule bases. A fuzzy logic compensator is developed to compensate the gravitational effects according to different robot configurations and task situations. Experimental results prove the validity of the proposed methods.

This paper describes a gait generation and control approach of a bipedal climbing robot with under-actuated mechanism. The special mechanical structure enables the robot to perform exploration tasks using "crawling", "pivoting" or "climbing" gaits. Multiple sensors are synthesized to generate successful gaits using a finite state machine. Experiments are conducted which demonstrate the effectiveness of proposed approach.

This paper describes the motion planning of a bipedal crawling robot with an under-actuated mechanism and multiple locomotion modes. A hybrid configuration space is proposed to incorporate the continuous configuration space with discrete motion status space imposed by the kinematic constraints. Under the hybrid configuration space framework, a motion planning method is developed which consists of a global planner and a local planner to generate a collision-free path and a feasible motion sequence to travel along the path. A cost function is defined based on the motion status information to guide the search for an optimal path. Simulation and experimental results have verified the theoretical development.

This paper presents the modeling and control of our second generation prototype miniature crawler robot which was targeted to applications in constrained environments. The mechanical design and the drive mechanism of the robot are first discussed A kinematic model is then derived and the motion planning is analyzed. A description of the Texas Instrument DSP-based embedded controller is presented. Finally, experimental results are presented for evaluation of the robot performance.

A robust backpropagation training algorithm with a dead zone scheme is used for the online tuning of the neuralnetwork (NN) tracking control system. This assures the convergence of the multilayered NN in the presence of disturbance. It is proved in this paper that the selection of a smaller range of the dead zone leads to a smaller estimate error of the NN, and hence a smaller tracking error of the NN tracking controller. The proposed algorithm is applied to a three-layered network with adjustable weights and a complete convergence proof is provided. The results can also be extended to the network with more hidden layers.