In this paper, we leverage self-supervised vision transformer models and their emergent semantic abilities to improve the generalization abilities of imitation learning policies. We introduce DVK, an imitation learning algorithm that leverages rich pre-trained Visual Transformer patch-level embeddings to obtain better generalization when learning through demonstrations. Our learner sees the world by clustering appearance features into groups associated with semantic concepts, forming stable keypoints that generalize across a wide range of appearance variations and object types. We demonstrate how this representation enables generalized behaviour by evaluating imitation learning across a diverse dataset of object manipulation tasks. To facilitate further study of generalization in Imitation Learning, all of our code for the method and evaluation, as well as the dataset, is made available.

Robots operating in an open world will encounter novel objects with unknown physical properties, such as mass, friction, or size. These robots will need to sense these properties through interaction prior to performing downstream tasks with the objects. We propose a method that autonomously learns tactile exploration policies by developing a generative world model that is leveraged to 1) estimate the object's physical parameters using a differentiable Bayesian filtering algorithm and 2) develop an exploration policy using an information-gathering model predictive controller. We evaluate our method on three simulated tasks where the goal is to estimate a desired object property (mass, height or toppling height) through physical interaction. We find that our method is able to discover policies that efficiently gather information about the desired property in an intuitive manner. Finally, we validate our method on a real robot system for the height estimation task, where our method is able to successfully learn and execute an information-gathering policy from scratch.

The common-sense reasoning abilities and vast general knowledge of large language models (LLMs) make them a natural fit for interpreting user requests in a smart home assistant context. LLMs, however, lack specific knowledge about the user and their home, which limits their potential impact. Smart home agent with grounded execution (SAGE), overcomes these and other limitations by using a scheme in which a user request triggers an LLM-controlled sequence of discrete actions. These actions can be used to retrieve information, interact with the user, or manipulate device states. SAGE controls this process through a dynamically constructed tree of LLM prompts, which help it decide which action to take next, whether an action was successful, and when to terminate the process. The SAGE action set augments an LLM’s capabilities to support some of the most critical requirements for a smart home assistant. These include: flexible and scalable user preference management (“Is my team playing tonight?”), access to any smart device’s full functionality without device-specific code via API reading (“Turn down the screen brightness on my dryer”), persistent device state monitoring (“Remind me to throw out the milk when I open the fridge”), natural device references using only a photo of the room (“Turn on the lamp on the dresser”), and more. We introduce a benchmark of 50 new and challenging smart home tasks where SAGE achieves a 76% success rate, significantly outperforming existing LLM-enabled baselines (30% success rate).

Contact-rich tasks continue to present many challenges for robotic manipulation. In this work, we leverage a multimodal visuotactile sensor within the framework of imitation learning (IL) to perform contact-rich tasks that involve relative motion (e.g., slipping and sliding) between the end-effector and the manipulated object. We introduce two algorithmic contributions,tactile force matchingandlearned mode switching, as complimentary methods for improving IL. Tactile force matching enhances kinesthetic teaching by reading approximate forces during the demonstration and generating an adapted robot trajectory that recreates the recorded forces. Learned mode switching uses IL to couple visual and tactile sensor modes with the learned motion policy, simplifying the transition from reaching to contacting. We perform robotic manipulation experiments on four door-opening tasks with a variety of observation and algorithm configurations to study the utility of multimodal visuotactile sensing and our proposed improvements. Our results show that the inclusion of force matching raises average policy success rates by 62.5%, visuotactile mode switching by 30.3%, and visuotactile data as a policy input by 42.5%, emphasizing the value of see-through tactile sensing for IL, both for data collection to allow force matching, and for policy execution to enable accurate task feedback.

Digital twins are expected to play an important role in the widespread adaptation of AI-based networking solutions in the real world. The calibration of these virtual replicas is critical to ensure a trustworthy replication of the real environment. This work focuses on the input parameter calibration of radio access network (RAN) simulators using real network performance metrics as supervision signals. Usually, the RAN digital twin is considered a black-box function and each calibration problem is viewed as a standalone search problem. RAN simulators are slow and non-differentiable, often posing as the bottleneck in the execution time for these search problems. In this work, we aim to accelerate the search process by reducing the number of interactions with the simulator by leveraging RAN interactions from previous problems. We present a sequential Bayesian optimization framework that uses information from the past to warm-start the calibration process. Assuming that the network performance exhibits gradual and periodic changes, the stored information can be reused in future calibrations. We test our method across multiple physical sites over one week and show that using the proposed framework, we can obtain better calibration with a smaller number of interactions with the simulator during the search phase.

This paper advances the field of pedestrian localization by introducing a unifying framework for opportunistic positioning based on nonlinear factor graph optimization. While many existing approaches assume constant availability of one or multiple sensing signals, our methodology employs IMU-based pedestrian inertial navigation as the backbone for sensor fusion, opportunistically integrating Ultra- Wideband (UWB), Bluetooth Low Energy (BLE), and WiFi signals when they are available in the environment. The proposed PEOPLEx framework is designed to incorporate sensing data as it becomes available, operating without any prior knowledge about the environment (e.g. anchor locations, radio frequency maps, etc.), Our contributions are twofold: 1) we introduce an opportunistic multi-sensor and real-time pedestrian positioning framework fusing the available sensor measurements; 2) we develop novel factors for adaptive scaling and coarse loop closures, significantly improving the precision of indoor positioning. Experimental validation confirms that our approach achieves accurate localization estimates in real indoor scenarios using commercial smartphones.

With the global aim of reducing carbon emissions, energy saving for communication systems has gained tremendous attention. Efficient energy-saving solutions are not only required to accommodate the fast growth in communication demand but solutions are also challenged by the complex nature of the load dynamics. Recent reinforcement learning (RL)-based methods have shown promising performance for network optimization problems, such as base station energy saving. However, a major limitation of these methods is the requirement of online exploration of potential solutions using a high-fidelity simulator or the need to perform exploration in a real-world environment. We circumvent this issue by proposing an offline reinforcement learning energy saving (ORES) framework that allows us to learn an efficient control policy using previously collected data. We first deploy a behavior energy-saving policy on base stations and generate a set of interaction experiences. Then, using a robust deep offline reinforcement learning algorithm, we learn an energy-saving control policy based on the collected experiences. Results from experiments conducted on a diverse collection of communication scenarios with different behavior policies showcase the effectiveness of the proposed energy-saving algorithms.

Planning a public transit network is a challenging optimization problem, but essential in order to realize the benefits of autonomous buses. We propose a novel algorithm for planning networks of routes for autonomous buses. We first train a graph neural net model as a policy for constructing route networks, and then use the policy as one of several mutation operators in a evolutionary algorithm. We evaluate this algorithm on a standard set of benchmarks for transit network design, and find that it outperforms the learned policy alone by up to 20% and a plain evolutionary algorithm approach by up to 53% on realistic benchmark instances.

In this paper, we investigate a hybrid scheme that combines nonlinear model predictive control (MPC) and model-based reinforcement learning (RL) for navigation planning of an autonomous model car across offroad, unstructured terrains without relying on predefined maps. Our innovative approach takes inspiration from BADGR, an LSTM-based network that primarily concentrates on environment modeling, but distinguishes itself by substituting LSTM modules with transformers to greatly elevate the performance of our model. Addressing uncertainty within the system, we train an ensemble of predictive models and estimate the mutual information between model weights and outputs, facilitating dynamic horizon planning through the introduction of variable speeds. Further enhancing our methodology, we incorporate a nonlinear MPC controller that accounts for the intricacies of the vehicle’s model and states. The model-based RL facet produces steering angles and quantifies inherent uncertainty. At the same time, the nonlinear MPC suggests optimal throttle settings, striking a balance between goal attainment speed and managing model uncertainty influenced by velocity. In the conducted studies, our approach excels over the existing baseline by consistently achieving higher metric values in predicting future events and seamlessly integrating the vehicle’s kinematic model for enhanced decision-making. The code and the evaluation data are available at (Github-repo).

This work explores the capacity of large language models (LLMs) to address problems at the intersection of spatial planning and natural language interfaces for navigation. We focus on following complex instructions that are more akin to natural conversation than traditional explicit procedural directives typically seen in robotics. Unlike most prior work where navigation directives are provided as simple imperative commands (e.g., "go to the fridge"), we examine implicit directives obtained through conversational interactions.We leverage the 3D simulator AI2Thor to create household query scenarios at scale, and augment it by adding complex language queries for 40 object types. We demonstrate that a robot using our method CARTIER (Cartographic lAnguage Reasoning Targeted at Instruction Execution for Robots) can parse descriptive language queries up to 42% more reliably than existing LLM-enabled methods by exploiting the ability of LLMs to interpret the user interaction in the context of the objects in the scenario.

We introduce PhotoBot, a framework for fully automated photo acquisition based on an interplay between high-level human language guidance and a robot photographer. We propose to communicate photography suggestions to the user via reference images that are selected from a curated gallery. We leverage a visual language model (VLM) and an object detector to characterize the reference images via textual descriptions and then use a large language model (LLM) to retrieve relevant reference images based on a user’s language query through text-based reasoning. To correspond the reference image and the observed scene, we exploit pretrained features from a vision transformer capable of capturing semantic similarity across marked appearance variations. Using these features, we compute suggested pose adjustments for an RGB-D camera by solving a perspective-n-point (PnP) problem. We demonstrate our approach using a manipulator equipped with a wrist camera. Our user studies show that photos taken by PhotoBot are often more aesthetically pleasing than those taken by users themselves, as measured by human feedback. We also show that PhotoBot can generalize to other reference sources such as paintings.

We present placing via picking (PvP), a method to autonomously collect real-world demonstrations for a family of placing tasks in which objects must be manipulated to specific, contact-constrained locations. With PvP, we approach the collection of robotic object placement demonstrations by reversing the grasping process and exploiting the inherent symmetry of the pick and place problems. Specifically, we obtain placing demonstrations from a set of grasp sequences of objects initially located at their target placement locations. Our system can collect hundreds of demonstrations in contact-constrained environments without human intervention using two modules: compliant control for grasping and tactile regrasping. We train a policy directly from visual observations through behavioural cloning, using the autonomously-collected demonstrations. By doing so, the policy can generalize to object placement scenarios outside of the training environment without privileged information (e.g., placing a plate picked up from a table). We validate our approach in home robot scenarios that include dishwasher loading and table setting. Our approach yields robotic placing policies that outperform policies trained with kinesthetic teaching, both in terms of success rate and data efficiency, while requiring no human supervision.

In this paper, hypernetworks are trained to generate behaviors across a range of unseen task conditions, via a novel TD-based training objective and data from a set of near-optimal RL solutions for training tasks. This work relates to meta RL, contextual RL, and transfer learning, with a particular focus on zero-shot performance at test time, enabled by knowledge of the task parameters (also known as context). Our technical approach is based upon viewing each RL algorithm as a mapping from the MDP specifics to the near-optimal value function and policy and seek to approximate it with a hypernetwork that can generate near-optimal value functions and policies, given the parameters of the MDP. We show that, under certain conditions, this mapping can be considered as a supervised learning problem. We empirically evaluate the effectiveness of our method for zero-shot transfer to new reward and transition dynamics on a series of continuous control tasks from DeepMind Control Suite. Our method demonstrates significant improvements over baselines from multitask and meta RL approaches.

To deal with notorious delays in communication systems, it is crucial to forecast key system characteristics, such as the communication load. Most existing studies aggregate data from multiple edge nodes for improving the forecasting accuracy. However, the bandwidth cost of such data aggregation could be unacceptably high from the perspective of system operators. To achieve both the high forecasting accuracy and bandwidth efficiency, this paper proposes an Adaptive Multi-Teacher Weighting in Teacher-Student Learning approach, namely AdaTeacher, for communication load forecasting of multiple edge nodes. Each edge node trains a local model on its own data. A target node collects multiple models from its neighbor nodes and treats these models as teachers. Then, the target node trains a student model from teachers via Teacher-Student (T-S) learning. Unlike most existing T-S learning approaches that treat teachers evenly, resulting in a limited performance, AdaTeacher introduces a bilevel optimization algorithm to dynamically learn an importance weight for each teacher toward a more effective and accurate T-S learning process. Compared to the state-of-the-art methods, Ada Teacher not only reduces the bandwidth cost by 53.85%, but also improves the load forecasting accuracy by 21.56% and 24.24% on two real-world datasets.

With the increasing use of data-intensive mobile applications and the number of mobile users, the demand for wireless data services has been increasing exponentially in recent years. In order to address this demand, a large number of new cellular base stations are being deployed around the world, leading to a significant increase in energy consumption and greenhouse gas emission. Consequently, energy consumption has emerged as a key concern in the fifth-generation (5G) network era and beyond. Reinforcement learning (RL), which aims to learn a control policy via interacting with the environment, has been shown to be effective in addressing network optimization problems. However, for reinforcement learning, especially deep reinforcement learning, a large number of interactions with the environment are required. This often limits its applicability in the real world. In this work, to better deal with dynamic traffic scenarios and improve real-world applicability, we propose a transfer deep reinforcement learning framework for energy optimization in cellular communication networks. Specifically, we first pre-train a set of RL-based energy-saving policies on source base stations and then transfer the most suitable policy to the given target base station in an unsupervised learning manner. Experimental results demonstrate that base station energy consumption can be reduced significantly using this approach.

The association of mobile devices with network resources (e.g., base stations, frequency bands/channels), known as load balancing, is critical to reduce communication traffic congestion and network performance. Reinforcement learning (RL) has shown to be effective for communication load balancing and achieves better performance than currently used rule-based methods, especially when the traffic load changes quickly. However, RL-based methods usually need to interact with the environment for a large number of time steps to learn an effective policy and can be difficult to tune. In this work, we aim to improve the data efficiency of RL-based solutions to make them more suitable and applicable for real-world applications. Specifically, we propose a simple, yet efficient and effective deep RL-based wireless network load balancing framework. In this solution, a set of good initialization values for control actions are selected with some cost-efficient approach to center the training of the RL agent. Then, a deep RL-based agent is trained to find offsets from the initialization values that optimize the load balancing problem. Experimental evaluation on a set of dynamic traffic scenarios demonstrates the effectiveness and efficiency of the proposed method.

In cellular networks, User Equipment (UE) handoff from one Base Station (BS) to another, giving rise to the load balancing problem among the BSs. To address this problem, BSs can work collaboratively to deliver a smooth migration (or handoff) and satisfy the UEs' service requirements. This paper formulates the load balancing problem as a Markov game and proposes a Robust Multi-agent Attention Actor-Critic (Robust-MA3C) algorithm that can facilitate collaboration among the BSs (i.e., agents). In particular, to solve the Markov game and find a Nash equilibrium policy, we embrace the idea of adopting a nature agent to model the system uncertainty. Moreover, we utilize the self-attention mechanism, which encourages high-performance BSs to assist low-performance BSs. In addition, we consider two types of schemes, which can facilitate load balancing for both active UEs and idle UEs. We carry out extensive evaluations by simulations, and simulation results illustrate that, compared to the state-of-the-art MARL methods, Robust-MA3C scheme can improve the overall performance by up to 45%.

Communication load balancing aims to balance the load between different available resources, and thus improve the quality of service for network systems. After formulating the load balancing (LB) as a Markov decision process problem, reinforcement learning (RL) has recently proven effective in addressing the LB problem. To leverage the benefits of classical RL for load balancing, however, we need an explicit reward definition. Engineering this reward function is challenging, because it involves the need for expert knowledge and there lacks a general consensus on the form of an optimal reward function. In this work, we tackle the communication load balancing problem from an inverse reinforcement learning (IRL) approach. To the best of our knowledge, this is the first time IRL has been successfully applied in the field of communication load balancing. Specifically, first, we infer a reward function from a set of demonstrations, and then learn a reinforcement learning load balancing policy with the inferred reward function. Compared to classical RL-based solution, the proposed solution can be more general and more suitable for real-world scenarios. Experimental evaluations implemented on different simulated traffic scenarios have shown our method to be effective and better than other baselines by a considerable margin.

Radio Access Networks (RANs) for telecommunications represent large agglomerations of interconnected hardware consisting of hundreds of thousands of transmitting devices (cells). Such networks undergo frequent and often heterogeneous changes caused by network operators, who are seeking to tune their system parameters for optimal performance. The effects of such changes are challenging to predict and will become even more so with the adoption of fifth-generation/sixth-generation (5G/6G) networks. Therefore, RAN monitoring is vital for network operators. We propose a self-supervised learning framework that leverages self-attention and self-distillation for this task. It works by detecting changes in Performance Measurement data, a collection of time-varying metrics which reflect a set of diverse measurements of the network performance at the cell level. Experimental results show that our approach outperforms the state of the art by 4% on a real-world based dataset consisting of about hundred thousands time series. It also has the merits of being scalable and generalizable. This allows it to provide deep insight into the specifics of mode of operation changes while relying minimally on expert knowledge.

With the continuous growth in communication network complexity and traffic volume, communication load balancing solutions are receiving increasing attention. Specifically, reinforcement learning (RL)-based methods have shown impressive performance compared with traditional rule-based methods. However, standard RL methods generally require an enormous amount of data to train, and generalize poorly to scenarios that are not encountered during training. We propose a policy reuse framework in which a policy selector chooses the most suitable pre-trained RL policy to execute based on the current traffic condition. Our method hinges on a policy bank composed of policies trained on a diverse set of traffic scenarios. When deploying to an unknown traffic scenario, we select a policy from the policy bank based on the similarity between the previous-day traffic of the current scenario and the traffic observed during training. Experiments demonstrate that this framework can outperform classical and adaptive rule-based methods by a large margin.

Digital twins have shown a great potential in supporting the development of wireless networks. They are virtual representations of 5G/6G systems enabling the design of machine learning and optimization-based techniques. Field data replication is one of the critical aspects of building a simulation-based twin, where the objective is to calibrate the simulation to match field performance measurements. Since wireless networks involve a variety of key performance indicators (KPIs), the replication process becomes a multi-objective optimization problem in which the purpose is to minimize the error between the simulated and field data KPIs. Unlike previous works, we focus on designing a data-driven search method to calibrate the simulator and achieve accurate and reliable reproduction of field performance. This work proposes a search-based algorithm based on mixed-variable particle swarm optimization (PSO) to find the optimal simulation parameters. Furthermore, we extend this solution to account for potential conflicts between the KPIs using a-fairness concept to adjust the importance attributed to each KPI during the search. Experiments on field data showcase the effectiveness of our approach to (i) improve the accuracy of the replication, (ii) enhance the fairness between the different KPIs, and (iii) guarantee faster convergence compared to other methods.

Our work examines the way in which large language models can be used for robotic planning and sampling in the context of automated photographic documentation. Specifically, we illustrate how to produce a photo-taking robot with an exceptional level of semantic awareness by leveraging recent advances in general purpose language (LM) and vision-language (VLM) models. Given a high-level description of an event we use an LM to generate a natural-language list of photo descriptions that one would expect a photographer to capture at the event. We then use a VLM to identify the best matches to these descriptions in the robot's video stream. The photo portfolios generated by our method are consistently rated as more appropriate to the event by human evaluators than those generated by existing methods.

We consider the detection of faults in robotic manipulators, with particular emphasis on faults that have not been observed or identified in advance, which naturally includes those that occur very infrequently. Recent studies indicate that the reward function obtained through Inverse Reinforcement Learning (IRL) can help detect anomalies caused by faults in a control system (i.e. fault detection). Current IRL methods for fault detection, however, either use a linear reward representation or require extensive sampling from the environment to estimate the policy, rendering them inappropriate for safety-critical situations where sampling of failure observations via fault injection can be expensive and dangerous. To address this issue, this paper proposes a zero-shot and exogenous fault detector based on an approximate variational reward imitation learning (AVRIL) structure. The fault detector recovers a reward signal as a function of externally observable information to describe the normal operation, which can then be used to detect anomalies caused by faults. Our method incorporates expert knowledge through a customizable reward prior distribution, allowing the fault detector to learn the reward solely from normal operation samples, without the need for a simulator or costly interactions with the environment. We evaluate our approach for exogenous partial fault detection in multi-stage robotic manipulator tasks, comparing it with several baseline methods. The results demonstrate that our method more effectively identifies unseen faults even when they occur within just three controller time steps.

Recent studies show that some security features that blockchains grant to decentralized networks on the internet can be ported to swarm robotics. Although the integration of blockchain technology and swarm robotics shows great promise, thus far, research has been limited to proof-of-concept scenarios where the blockchain-based mechanisms are tailored to a particular swarm task and operating environment. In this study, we propose a generic framework based on a blockchain smart contract that enables robot swarms to achieve secure consensus in an arbitrary observation space. This means that our framework can be customized to fit different swarm robotics missions, while providing methods to identify and neutralize Byzantine robots, that is, robots which exhibit detrimental behaviours stemming from faults or malicious tampering.

Due to their large bandwidth and impressive data speed, millimeter-wave (mmWave) radios are expected to play a key role in the 5G and beyond (e.g., 6G) communication networks. Yet, to release mmWave’s true power, the highly directional mmWave beams need to be aligned perfectly. Most existing beam alignment methods adopt an exhaustive or semi-exhaustive space scanning, which introduces up to seconds of delays. To eliminate the need for complex space scanning, this article presents an Ultra-wideband (UWB)-assisted mmWave communication framework, which leverages the co-located UWB antennas to estimate the best angles for mmWave beam alignment. One major challenge of applying this idea in the real world is the barrier of limited antenna numbers. Commercial-Off-The-Shelf (COTS) devices are usually equipped with only a small number of UWB antennas, which are not enough for the existing algorithms to provide an accurate angle estimation. To solve this challenge, we design a novel Multi-Frequency MUltiple SIgnal Classification (MF-MUSIC) algorithm, which extends the classic MUltiple SIgnal Classification (MUSIC) algorithm to the frequency domain and overcomes the antenna limitation barrier in the spatial domain. Extensive real-world experiments and numerical simulations illustrate the advantage of the proposed MF-MUSIC algorithm. MF-MUSIC uses only three antennas to achieve an accurate angle estimation, which is a mere 0.15° (or a relative difference of 3.6%) different from the state-of-the-art 16-antenna-based angle estimation method.

Advanced communication network functions, such as resource allocation and dynamic spectrum management, heavily rely on the accurate forecasting of traffic. Data-driven solutions, e.g., Neural Network (NN) based forecasting methods, have been proven to be effective only when sufficient data is available. However, Base Stations (BSs) have limited data in the real world, since big data for communication networks could be extremely expensive to collect, store, and migrate. Therefore, most existing traffic forecasting methods have limited accuracy in reality due to the lack of big data. To tackle this problem, our key observation is that, despite the data “amount” in a BS is limited, the data “source” is rich and diverse, i.e., in addition to Internet traffic logs, there are logs of Call and SMS. More importantly, our analysis shows a high correlation between different sources, which can be utilized to improve the forecasting accuracy. Motivated by this, we introduce AdaSource, a Multi-Source Adaptive Feature Boosting approach, which utilizes data source correlations for accurate traffic forecasting even on data-limited BSs. The core idea of AdaSource is a novel two-branch NN structure that adaptively trains multiple Encoder-Decoders for refining different data sources and multiple Encoder-Predictors for utilizing data source correlations to improve the accuracy. The experiments on a real-world dataset show that AdaSource improves the forecasting accuracy by up to 30.14%, compared to the state-of-the-art methods.

The anticipated huge number of devices and large traffic volumes impose new challenges on the communication system requirements and design. One of the main requirements of massive machine-type communication (mMTC) is to support network energy efficiency. Data compression is a widely adopted technique that enables higher energy efficiency, lower latency, and better bandwidth utilization. Unfortunately, the current compression techniques are mainly designed for human-type communications (HTC). Therefore, they consider the reconstruction fidelity, rather than the accuracy of inferred decisions, as the sole performance metric. In this work, we propose a novel encoder for data compression in mMTC communications, which is termed Distillation Encoder (DE). Unlike prior work, the design of the proposed DE aims to achieve high compression ratios while preserving the accuracy of the inferred decisions. DE inherits the knowledge of a large teacher model (trained on the raw data) through knowledge distillation. Evaluating the proposed framework on several public datasets shows a clear performance advantage compared with baseline models in terms of the inferred decision accuracy and generalizing to yet-unseen data. Moreover, the DE can be applied to learn efficient quantizers, as shown in the results.

Although reinforcement learning (RL) shows advantages in cellular network load balancing, it suffers from a low generalization ability, preventing it from real-world applications. Specifically, if network traffic pattern changes, the learned RL policy cannot adapt accordingly, resulting in system performance degradation. To address this issue, we propose a Multi-teacher MOdel BAsed Reinforcement Learning algorithm (MOBA), which leverages multi-teacher knowledge distillation theory to learn a generalized load balancing policy for adapting the real-world traffic pattern changes. The key is that different teachers represent different traffic patterns, and can learn various system models. By distilling and transferring the teacher knowledge, the student network is able to learn a generalized system model that covers different traffic patterns and unseen situations. Moreover, to improve the robustness of multi-teacher knowledge transfer, we learn a set of student models and use an ensemble method to jointly predict system dynamics. Results show that, compared with state-of-the-art RL methods, MOBA improves the minimal throughput and total throughput of a cellular network by up to 28.6% and 23.2%. Results also show that MOBA improves the training efficiency by up to 64%.

The modern power system is transitioning towards increasing penetration of renewable energy generation and demand from different types of electrical appliances. With this transition, residential load forecasting, especially short-term load forecasting (STLF), is becoming more and more challenging and important. Accurate short-term load forecasting can help improve energy dispatching efficiency and, as a consequence, reduce overall power system operation cost. Most current load forecasting algorithms assume that there is a large amount of training data available upon which to learn a reliable load forecasting model. However, this assumption can be challenging for real-world applications. In this work, we first propose the use of transfer learning and an attention mechanism to improve short-term load forecasting for a target domain with only a limited amount of available data. Furthermore, we extend the proposed method to utilize heterogeneous features which enables the approach to deal with more complex scenarios in the real world. Experimental results using real-world data sets show that the proposed methods can improve forecasting accuracy by a large margin over several existing baselines.

Prediction of future traffic load is a crucial task to support the automatic Operations, Administration, and Management (OAM) of communication networks. Existing Machine Learning (ML) models require big data to accomplish this task. However, large data sets are not always available, due to the limited storage capacity and the high storage cost at Base Stations (BSs). To solve the problem, we leverage the spatial-temporal correlation among different BSs, which allows other BSs’ data to be used for the prediction of the target BS. One major challenge in realizing this idea is the imbalance of data amounts between neighbor BSs and a prediction target BS. If one simply aggregates the data from both neighbors and target, the target’s traffic features would be overwhelmed by the neighbors’ data. To address this challenge, we propose a Spatial-Temporal Transfer (STT) framework, which trains a base model with an aggregated data set from multiple BSs, and then carefully refines the base model to serve a target BS. To strike a perfect balance between general tendency and individual features, STT adopts an advanced transfer learning technique that exploits regularization on model parameters. Experiments show the efficiency of the proposed STT framework.

Accurate traffic volume estimation and prediction are essential for advanced communication network functions, such as automatic operations and predictive resource allocation. Although machine learning (ML)-based approaches achieve great success in accomplishing this goal, existing approaches suffer from two drawbacks that limit their real-world applications. First, the ML-based prediction models developed in the past might be obsolete now, since the communication traffic patterns and volumes keep changing in the real world, leading to prediction errors. Second, most Base Stations (BSs) can only save a small amount of data due to the limited storage capacity and high storage costs, which prevents from training an accurate prediction model. In this paper, we propose a novel framework that adapts the prediction model to the constantly changing traffic with only a few current traffic data. Specifically, the framework first learns the knowledge of historical traffic data as much as possible by using a proposed two-branch neural network design, which includes a prediction and a reconstruction module. Then, the framework transfers the knowledge from an old (past) prediction model to a new (current) model for the model update by using a proposed continual knowledge distillation technique. Evaluations on a real-world dataset show that the proposed framework reduces the Mean Absolute Error (MAE) of traffic prediction by up to 9.62% compared to the state-of-the-art prediction methods.

Due to the rapid increase in wireless communication traffic in recent years, load balancing is becoming increasingly important for ensuring the quality of service. However, variations in traffic patterns near different serving base stations make this task challenging. On one hand, crafting a single control policy that performs well across all base station sectors is often difficult. On the other hand, maintaining separate controllers for every sector introduces overhead, and leads to redundancy if some of the sectors experience similar traffic patterns. In this paper, we propose to construct a concise set of controllers that cover a wide range of traffic scenarios, allowing the operator to select a suitable controller for each sector based on local traffic conditions. To construct these controllers, we present a method that clusters similar scenarios and learns a general control policy for each cluster. We use deep reinforcement learning (RL) to first train separate control policies on diverse traffic scenarios, and then incrementally merge together similar RL policies via knowledge distillation. Experimental results show that our concise policy set reduces redundancy with very minor performance degradation compared to policies trained separately on each traffic scenario. Our method also outperforms handcrafted control parameters, joint learning on all tasks, and two popular clustering methods.

Mobile edge computing (MEC) networks have been recently adopted to accommodate the fast-growing number of mobile devices performing complicated tasks with limited hardware capability. Recently, edge nodes with communication, computation, and caching capacities are starting to be deployed in MEC networks. Due to the physical separation of these resources, efficient coordination and scheduling are important for efficient resource utilization and optimal network performance. In this paper, we study mobility load balancing for communication, computation, and caching-enabled heterogeneous MEC networks. Specifically, we propose to tackle this problem via a multi-agent deep reinforcement learning-based framework. Users served by overloaded edge nodes are handed over to less loaded ones, to minimize the load in the most loaded base station in the network. In this framework, the handover decision for each user is made based on the user’s own observation which comprises the user’s task at hand and the load status of the MEC network. Simulation results show that our proposed multi-agent deep reinforcement learning-based approach can reduce the time-average maximum load by up to 30% and the end-to-end delay by 50% compared to baseline algorithms.

With the increasing adoption of renewable energy generation and electric devices, electric load forecasting, especially short-term load forecasting (STLF), is becoming more and more important. The widespread adoption of smart meters makes it possible to utilize complex machine learning models for both aggregated load and single-home residential load forecasting. Similar homes in nearby locations are likely to have similar load consumption patterns and this similarity can be used to improve the overall forecasting performance. However, most current work on load forecasting focuses on single learning task without exploiting the benefit of joint learning. In this paper, we propose the use of the multi-task learning (MTL) framework with long short-term memory (LSTM) recurrent neural networks for both aggregated and single home STLF. We propose a MTL-based forecasting algorithm for aggregated load forecasting in which single home forecasting is formulated as a single learning task within the MTL framework. This algorithm is extended for single home load forecasting in which load forecasting for a particular home becomes the primary learning task. Experimental results on real-world data sets demonstrate that residential load forecasting for both aggregated load and a single home can be improved within the MTL framework.

With the increasing popularity of electric vehicles and the growing trend of working from home, electricity consumption in the residential sector is expected to continue to grow rapidly over the next few years. As a consequence, short-term residential load forecasting is becoming even more vital for the reliability and sustainability of the smart grid. Although deep learning models have shown impressive success in different areas including short-term electric load forecasting, such models require a large amount of training data. For many real-world load forecasting cases, we may not have enough training data to learn a reliable forecasting model. In this paper, we address this challenge through the use of boosting-based transfer learning with multiple sources. We first train a set of deep regression models on source houses that can provide relatively abundant data. We then transfer these learned models via the boosting framework to support data-scarce target houses. The transfer process is selective and customized for each target house to minimize the potential for negative transfer. Experimental results, based on real-world residential data sets, show that the proposed method can significantly improve forecasting accuracy.

Manipulating objects with dexterity requires timely feedback that simultaneously leverages the senses of vision and touch. In this paper, we focus on the problem setting where both visual and tactile sensors provide pixel-level feedback for Visuotactile reinforcement learning agents. We investigate the challenges associated with multimodal learning and propose several improvements to existing RL methods; including tactile gating, tactile data augmentation, and visual degradation. When compared with visual-only and tactile-only baselines, our Visuotactile-RL agents showcase (1) significant improvements in contact-rich tasks; (2) improved robustness to visual changes (lighting/camera view) in the workspace; and (3) resilience to physical changes in the task environment (weight/friction of objects).

In this paper, we present the Sample Efficient Social Navigation from Observation (SESNO) algorithm that efficiently learns socially-compliant navigation policies from observations of human trajectories. SESNO is an inverse reinforcement learning (IRL)-based algorithm that learns from human trajectory observations without knowledge of their actions. We improve the sample-efficiency over previous IRL-based methods by introducing a shared experience replay buffer that allows reuse of past trajectory experiences to estimate the policy and the reward. We evaluate SESNO using publicly available pedestrian motion data sets and compare its performance to related baseline methods in the literature. We show that SESNO yields performance superior to existing baselines while dramatically improving the sample complexity by using as few as a hundredth of the samples required by existing baselines.

We propose an approach that enables simultaneous interpretable learning of a high-level discrete behaviour and its low-level rhythmic sub-behaviour. We do this though a unified reward function, where a reward function that only describes low-level behaviour, with less impact on learning of other behaviours is recovered from few-shot motion demonstrations. To this end, we first extract local behaviour motifs from state-only human demonstrations and random driving samples using an adaptive motif discovery approach derived from the Matrix Profile algorithm. We then optimize parameters for motif discovery by maximizing the sum and entropy over motif sizes. Interacting Multiple Model (IMM) estimators are constructed on top of linear-Gaussian dynamics of discovered motifs, the cumulative distributions over motifs estimated by IMMs serve as the basis of the reward function. By combining the recovered reward with the terrain type signal gathered from the environment, we are able to train a dual-objective off-road vehicle controller that demonstrates both terrain selection and human-like driving behaviours. Compared with related approaches across 10 people, our rhythmic behaviour reward recovery approach enables the controller to produce higher preference over human driving demonstrations. In addition to performing more stable across different people with 87% less variance than the best baseline in rhythmic behaviour indicator, our method reduces the negative effects on higher-level behaviour learning while maintaining high interpretability at all stages of the algorithm.

Emerging Internet of Things (IoT) applications, such as cashier-less shopping, mobile ads targeting, and geo-based augmented reality (AR), are expected to bring us much more convenience and infotainment. To realize this amazing future, we need to feed these applications with user locations of (sub)meter-level resolution anytime and anywhere. Unfortunately, many widely used location sources are either unavailable indoor (e.g., global positioning system) or coarse grained (e.g., user check-ins). In order to provide ubiquitous localization services, the widespread WiFi signals are being leveraged to establish (sub)meter-level localization systems. Fine-grained WiFi propagation characteristics, which are sensitive to human body locations, have been employed to create location fingerprints. However, these WiFi characteristics are also sensitive to: 1) the body shapes of different users and 2) the objects in the background environment. Consequently, systems based on WiFi fingerprints are vulnerable in the presence of: 1) new users with different body shapes and 2) daily changes of the environment, e.g., opening/closing doors. To tackle this issue, this article proposes a WiFi-based localization system based on domain-adaptation with cluster assumption, named Fidora. Fidora is able to: 1) localize different users with labeled data from only one or two example users and 2) localize the same user in a changed environment without labeling any new data. To achieve these, Fidora integrates two major modules. It first adopts a data augmenter that introduces data diversity using a variational autoencoder (VAE). It then trains a domain-adaptive classifier that adjusts itself to newly collected unlabeled data using a joint classification-reconstruction structure. We conducted real-world experiments to evaluate Fidora against the state of the art. It is demonstrated that when tested on an unlabeled user, Fidora increases the average$F1$score by 17.8% and improves the worst case accuracy by 20.2%. Moreover, when applied in a varied environment, Fidora outperforms the state of the art by 23.1%.

Load balancing has become a key technique to handle the increasing traffic demand and improve the user experience. It evenly distributes the traffic across network resources by offloading users from overloaded base stations or channels to less crowded ones. Load balancing is a multi-objective optimization problem involving the automatic adjustment of several parameters to simultaneously maximize multiple network performance indicators. However, the existing methods mostly rely on single-objective approaches which lead to sub-optimal solutions. In this paper, we introduce the first multi-objective reinforcement learning (MORL) framework for load balancing. Specifically, we propose a solution based on meta-reinforcement learning (meta-RL) to learn a general policy capable of quickly adapting to new trade-offs between the objectives. We further enhance the generalization of our proposed solution using policy distillation techniques. To showcase the effectiveness of our framework, experiments are conducted based on real-world traffic scenarios. Our results show that our load balancing framework can (i) significantly outperform the existing rule-based and single-objective solutions, (ii) compute better Pareto front approximations compared to MORL baselines, and (iii) quickly adapt to new objective trade-offs.

Predicting the future interaction of objects when they come into contact with their environment is key for autonomous agents to take intelligent and anticipatory actions. This paper presents a perception framework that fuses visual and tactile feedback to make predictions about the expected motion of objects in dynamic scenes. Visual information captures object properties such as 3D shape and location, while tactile information provides critical cues about interaction forces and resulting object motion when it makes contact with the environment. Utilizing a novel See-Through-your-Skin (STS) sensor that provides high resolution multimodal sensing of contact surfaces, our system captures both the visual appearance and the tactile properties of objects. We interpret the dual stream signals from the sensor using a Multimodal Variational Autoencoder (MVAE), allowing us to capture both modalities of contacting objects and to develop a mapping from visual to tactile interaction and vice-versa. Additionally, the perceptual system can be used to infer the outcome of future physical interactions, which we validate through simulated and real-world experiments in which the resting state of an object is predicted from given initial conditions.

By placing the computing, storage and networking resources close to the end users, distributed edge computing greatly benefits the performance of 5G communication systems. However, as a tradeoff, resources on the edge are usually limited and imbalanced among the heterogeneous edge nodes. To overcome this drawback, this paper proposes a Transfer Learning based Prediction (TLP) framework that allows the edge nodes to share their resources and data in an efficient manner. In particular, the TLP framework focuses on the prediction of the future traffic load, which is a key reference for many automated network functions. To enhance the efficiency of data and bandwidth, TLP first learns a base model on a data-abundant edge node (the source), and then transfers this model (instead of data) to other data-limited nodes (the targets). To achieve a delicate balance between maintaining common features and learning target-specific features, we develop a new transfer learning technique named Similarity-based Elastic Weight Con-solidation (SEWC), and integrate it into TLP. Experiments on real-world data illustrate that, compared to the state-of-the-art methods, TLP-SEWC reduces the Mean Absolute Error (MAE) of traffic prediction by up to 57.9%.

Prediction of key system characteristics, such as the communication load, is required to overcome the delays in wireless communication systems. State-of-The-Art (SOTA) approaches mostly apply existing Neural Network (NN) structures, and extract latent features purely based on their sensitivity to the forecasting accuracy. This way of feature extraction may neglect some non-obvious yet informative dimensions in the model input, leading to inaccurate forecasting results. In this paper, we present an Adaptive Feature Boosting (AFB) approach, which integrates multiple AutoEncoders (AEs) to automatically extract robust and comprehensive latent features for communication load forecasting. The recurrent and residual connections among the AEs make sure that the extracted latent features are representative for all input dimensions. With more comprehensive information extracted from the history, the forecasting accuracy is thus improved. We evaluate AFB against existing approaches on a real-world dataset that contains Call Detail Records (CDRs) of the Milan city over a period of two months. The evaluation shows that our AFB-based approach achieves 35.2% more accurate load forecasting results than the SOTA deep approaches.

We present a generative adversarial method that uses deep learning to identify network load traffic conditions in which network optimization algorithms under-perform other known algorithms: the Deep Convolutional Failure Generator (DCFG). The spatial distribution of network load presents challenges for network operators for tasks such as load balancing, in which a network optimizer attempts to maintain high quality communication while at the same time abiding capacity constraints. Testing a network optimizer for all possible load distributions is challenging if not impossible. We propose a novel method that searches for load situations where a target network optimization method underperforms baseline, which are key test cases that can be used for future refinement and performance optimization. By modeling a realistic network simulator's quality assessments with a deep network and, in parallel, optimizing a load generation network, our method efficiently searches the high dimensional space of load patterns and reliably finds cases in which a target network optimization method under-performs a baseline by a significant margin.

Within a cellular network, load balancing between different cells is of critical importance to network performance and quality of service. Most existing load balancing algorithms are manually designed and tuned rule-based methods where near-optimality is almost impossible to achieve. These rule-based meth-ods are difficult to adapt quickly to traffic changes in real-world environments. Given the success of Reinforcement Learning (RL) algorithms in many application domains, there have been a number of efforts to tackle load balancing for communication systems using RL-based methods. To our knowledge, none of these efforts have addressed the need for data efficiency within the RL framework, which is one of the main obstacles in applying RL to wireless network load balancing. In this paper, we formulate the communication load balancing problem as a Markov Decision Process and propose a data-efficient transfer deep reinforcement learning algorithm to address it. Experimental results show that the proposed method can significantly improve the system performance over other baselines and is more robust to environmental changes.

Due to the uneven demographic distribution and people’s daily activities, communication systems usually experience highly imbalanced load across different cells. This imbalance leads to unsatisfied users in the congested cells and under-utilized resources in the less-loaded cells. To deal with this issue, existing work migrates the load from heavily loaded cells to lightly loaded cells, by either handing over active mode User Equipment (UEs) to other serving cells, or re-selecting the camping cells for idle mode UEs. In this paper, we further advance the research on Load Balancing (LB) with a hybrid control of both active and idle UEs. This task is challenging, due to the conflicts between Active-UE LB (AULB) and Idle-UE LB (IULB) policies. To overcome this challenge, we propose a Hierarchical Policy Learning (HPL) framework, which coordinates the actions between LB policies with a two-level learning structure. In this way, HPL produces AULB and IULB policies that are better aligned with each other. Extensive simulation results illustrate the efficiency and efficacy of the proposed HPL.

Due to their large bandwidth and impressive data speed, millimeter-wave (mmWave) radios are expected to play a key role in the 5G and beyond (e.g., 6G) communication networks. Yet, to release mmWave’s true power, the highly directional mmWave beams need to be aligned perfectly. Most existing beam alignment methods adopt an exhaustive or semi-exhaustive space scanning, which introduces up to seconds of delays.To eliminate the need of a complex space scanning, this paper presents an Ultra-wideband (UWB)-assisted mmWave communication framework, which leverages the co-located UWB antennas to estimate the best angles for mmWave beam alignment. One major challenge to apply this idea in real-world is the barrier of limited antenna numbers. Commercial-Off-The-Shelf (COTS) devices are usually equipped with only a few number of UWB antennas, which are not enough for the existing algorithms to provide an accurate angle estimation. To solve this challenge, we design a novel Multi-Frequency MUSIC (MF-MUSIC) algorithm, which extends the classic MUSIC algorithm to the frequency domain and overcomes the antenna limitation barrier in the spatial domain. By doing this, our framework uses only 3 antennas to achieve an accurate angle estimation, which is merely 0.15° different from the state-of-the-art 16-antenna method.

Although existing cellular network base stations are typically immobile, the recent development of small form factor base stations and self driving cars has enabled the possibility of deploying a team of continuously moving base stations that can reorganize the network infrastructure to adapt to changing network traffic usage patterns. Given such a system of mobile base stations (MBSes) that can freely move on the road, how should their path be planned in an effort to optimize the experience of the users? This paper addresses this question by modeling the problem as a Markov Decision Process where the actions correspond to the MBSes deciding which direction to go at traffic intersections; states corresponds to the position of MBSes; and rewards correspond to minimization of packet loss in the network. A Monte Carlo Tree Search (MCTS)-based anytime algorithm that produces path plans for multiple base stations while optimizing expected packet loss is proposed. Simulated experiments in the city of Verdun, QC, Canada with varying user equipment (UE) densities and random initial conditions show that the proposed approach consistently outperforms myopic planners, and is able to achieve near-optimal performance.

Dynamics modeling in outdoor and unstructured environments is difficult because different elements in the environment interact with the robot in ways that can be hard to predict. Leveraging multiple sensors to perceive maximal information about the robot’s environment is thus crucial when building a model to perform predictions about the robot’s dynamics with the goal of doing motion planning. We design a model capable of long-horizon motion predictions, leveraging vision, lidar and proprioception, which is robust to arbitrarily missing modalities at test time. We demonstrate in simulation that our model is able to leverage vision to predict traction changes. We then test our model using a real-world challenging dataset of a robot navigating through a forest, performing predictions in trajectories unseen during training. We try different modality combinations at test time and show that, while our model performs best when all modalities are present, it is still able to perform better than the baseline even when receiving only raw vision input and no proprioception, as well as when only receiving proprioception. Overall, our study demonstrates the importance of leveraging multiple sensors when doing dynamics modeling in outdoor conditions.

Model-free reinforcement learning has become a viable approach for vision-based robot control. However, sample complexity and adaptability to domain shifts remain persistent challenges when operating in high-dimensional observation spaces (images, LiDAR), such as those that are involved in autonomous driving. In this paper, we propose a flexible framework by which a policy’s observations are augmented with robust attention representations in the latent space to guide the agent’s attention during training. Our method encodes local and global descriptors of the augmented state representations into a compact latent vector, and scene dynamics are approximated by a recurrent network that processes the latent vectors in sequence. We outline two approaches for constructing attention maps; a supervised pipeline leveraging semantic segmentation networks, and an unsupervised pipeline relying only on classical image processing techniques. We conduct our experiments in simulation and test the learned policy against varying seasonal effects and weather conditions. Our design decisions are supported in a series of ablation studies. The results demonstrate that our state augmentation method both improves learning efficiency and encourages robust domain adaptation when compared to common end-to-end frameworks and methods that learn directly from intermediate representations.

We present a reward-predictive, model-based learning method featuring trajectory-constrained visual attention for use in mapless, local visual navigation tasks. Our method learns to place visual attention at locations in latent image space which follow trajectories caused by vehicle control actions to later enhance predictive accuracy during planning. Our attention model is jointly optimized by the task-specific loss and additional trajectory-constraint loss, allowing adaptability yet encouraging a regularized structure for improved generalization and reliability. Importantly, visual attention is applied in latent feature map space instead of raw image space to promote efficient planning. We validated our model in visual navigation tasks of planning low turbulence, collision-free trajectories in off-road settings and hill climbing with locking differentials in the presence of slippery terrain. Experiments involved randomized procedural generated simulation and real-world environments. We found our method improved generalization and learning efficiency when compared to no-attention and self-attention alternatives.

We introduce a new class of vision-based sensor and associated algorithmic processes that combine visual imaging with high-resolution tactile sending, all in a uniform hardware and computational architecture. We demonstrate the sensor's efficacy for both multi-modal object recognition and metrology. Object recognition is typically formulated as an unimodal task, but by combining two sensor modalities we show that we can achieve several significant performance improvements. This sensor, named the See-Through-your-Skin sensor (STS), is designed to provide rich multi-modal sensing of contact surfaces. Inspired by recent developments in optical tactile sensing technology, we address a key missing feature of these sensors: the ability to capture a visual perspective of the region beyond the contact surface. Whereas optical tactile sensors are typically opaque, we present a sensor with a semitransparent skin that has the dual capabilities of acting as a tactile sensor and/or as a visual camera depending on its internal lighting conditions. This paper details the design of the sensor, showcases its dual sensing capabilities, and presents a deep learning architecture that fuses vision and touch. We validate the ability of the sensor to classify household objects, recognize fine textures, and infer their physical properties both through numerical simulations and experiments with a smart countertop prototype.

Recent work on semantic simultaneous localization and mapping (SLAM) have shown the utility of natural objects as landmarks for improving localization accuracy and robustness. In this paper we present a monocular semantic SLAM system that uses object identity and inter-object geometry for view-invariant loop detection and drift correction. Our system's ability to recognize an area of the scene even under large changes in viewing direction allows it to surpass the mapping accuracy of ORB-SLAM, which uses only local appearance-based features that are not robust to large viewpoint changes. Experiments on real indoor scenes show that our method achieves mean drift reduction of 70% when compared directly to ORB-SLAM. Additionally, we propose a method for object orientation estimation, where we leverage the tracked pose of a moving camera under the SLAM setting to overcome ambiguities caused by object symmetry. This allows our SLAM system to produce geometrically detailed semantic maps with object orientation, translation, and scale.

We present a method for learning to drive on smooth terrain while simultaneously avoiding collisions in challenging off-road and unstructured outdoor environments using only visual inputs. Our approach applies a hybrid model-based and model-free reinforcement learning method that is entirely self-supervised in labeling terrain roughness and collisions using on-board sensors. Notably, we provide both first-person and overhead aerial image inputs to our model. We nd that the fusion of these complementary inputs improves planning foresight and makes the model robust to visual obstructions. Our results show the ability to generalize to environments with plentiful vegetation, various types of rock, and sandy trails. During evaluation, our policy attained 90% smooth terrain traversal and reduced the proportion of rough terrain driven over by 6.1 times compared to a model using only first-person imagery. Video and project details can be found at www.cim.mcgill.ca/mrl/offroad_driving/.

In this paper, we propose a real-time deep learning approach for determining the 6D relative pose of Autonomous Underwater Vehicles (AUV) from a single image. A team of autonomous robots localizing themselves in a communication-constrained underwater environment is essential for many applications such as underwater exploration, mapping, multi-robot convoying, and other multi-robot tasks. Due to the profound difficulty of collecting ground truth images with accurate 6D poses underwater, this work utilizes rendered images from the Unreal Game Engine simulation for training. An image-to-image translation network is employed to bridge the gap between the rendered and the real images producing synthetic images for training. The proposed method predicts the 6D pose of an AUV from a single image as 2D image keypoints representing 8 corners of the 3D model of the AUV, and then the 6D pose in the camera coordinates is determined using RANSAC-based PnP. Experimental results in real-world underwater environments (swimming pool and ocean) with different cameras demonstrate the robustness and accuracy of the proposed technique in terms of translation error and orientation error over the state-of-the-art methods. The code is publicly available.

We consider the task of underwater robot navigation for the purpose of collecting scientifically relevant video data for environmental monitoring. The majority of field robots that currently perform monitoring tasks in unstructured natural environments navigate via path-tracking a pre-specified sequence of waypoints. Although this navigation method is often necessary, it is limiting because the robot does not have a model of what the scientist deems to be relevant visual observations. Thus, the robot can neither visually search for particular types of objects, nor focus its attention on parts of the scene that might be more relevant than the pre-specified waypoints and viewpoints. In this paper we propose a method that enables informed visual navigation via a learned visual similarity operator that guides the robot's visual search towards parts of the scene that look like an exemplar image, which is given by the user as a high-level specification for data collection. We propose and evaluate a weakly supervised video representation learning method that outperforms ImageNet embeddings for similarity tasks in the underwater domain. We also demonstrate the deployment of this similarity operator during informed visual navigation in collaborative environmental monitoring scenarios, in large-scale field trials, where the robot and a human scientist collaboratively search for relevant visual content. Code: https://github.com/rvl-lab-utoronto/visual_search_in_the_wild.

This paper considers the problem of learning behaviors in simulation without knowledge of the precise dynamical properties of the target robot platform(s). In this context, our learning goal is to mutually maximize task efficacy on each environment considered and generalization across the widest possible range of environmental conditions. The physical parameters of the simulator are modified by a component of our technique that learns the Domain Randomization (DR) that is appropriate at each learning epoch to maximally challenge the current behavior policy, without being overly challenging, which can hinder learning progress. This so-called sweet spot distribution is a selection of simulated domains with the following properties: 1) The trained policy should be successful in environments sampled from the domain randomization distribution; and 2) The DR distribution made as wide as possible, to increase variability in the environments. These properties aim to ensure the trajectories encountered in the target system are close to those observed during training, as existing methods in machine learning are better suited for interpolation than extrapolation. We show how adapting the DR distribution while training context-conditioned policies results in improvements on jump-start and asymptotic performance when transferring a learned policy to the target environment1.

Passive sensing with ambient WiFi signals is a promising technique that will enable new types of human-robot interactions while preserving users' privacy. Here, we present PresSense, a system for human respiration sensing in noisy environments. Unlike existing WiFi-based respiration sensors, we employ a human presence detector, improving the robustness in scenarios where no human is present in an Area Of Interest (AOI). We also integrate our novel feature, Peak Distance Histogram (PDH), with other classic WiFi features to achieve better accuracy when someone is present in the AOI. We tested our system using commodity WiFi devices in an office room. Our PresSense outperforms the state of the arts in both respiration rate estimation and presence detection.

To fully support the emerging location-aware applications, location information with meter-level resolution (or even higher) is required anytime and anywhere. Unfortunately, most of the current location sources (e.g., GPS and check-in data) either are unavailable indoor or provide only house-level resolutions. To fill the gap, this paper utilizes the ubiquitous WiFi signals to establish a (sub)meter-level localization system, which employs WiFi propagation characteristics as location fingerprints. However, an unsolved issue of these WiFi fingerprints lies in their inconsistency across different users. In other words, WiFi fingerprints collected from one user may not be used to localize another user. To address this issue, we propose a WiFi-based Domain-adaptive system FiDo, which is able to localize many different users with labelled data from only one or two example users. FiDo contains two modules: 1) a data augmenter that introduces data diversity using a Variational Autoencoder (VAE); and 2) a domain-adaptive classifier that adjusts itself to newly collected unlabelled data using a joint classification-reconstruction structure. Compared to the state of the art, FiDo increases average F1 score by 11.8% and improves the worst-case accuracy by 20.2%.

In recent years self-driving vehicles have become more commonplace on public roads, with the promise of bringing safety and efficiency to modern transportation systems. Increasing the reliability of these vehicles on the road requires an extensive suite of software tests, ideally performed on high-fidelity simulators, where multiple vehicles and pedestrians interact with the self-driving vehicle. It is therefore of critical importance to ensure that self-driving software is assessed against a wide range of challenging simulated driving scenarios. The state of the art in driving scenario generation, as adopted by some of the front-runners of the self-driving car industry, still relies on human input [1]. In this paper we propose to automate the process using Bayesian optimization to generate adversarial self-driving scenarios that expose poorly-engineered or poorly-trained self-driving policies, and increase the risk of collision with simulated pedestrians and vehicles. We show that by incorporating the generated scenarios into the training set of the self-driving policy, and by fine-tuning the policy using vision-based imitation learning we obtain safer self-driving behavior.

In this paper we consider inter-robot communication in the context of joint activities. In particular, we focus on convoying and passive communication for radio-denied environments by using whole-body gestures to provide cues regarding future actions. We develop a communication protocol whereby information described by codewords is transmitted by a series of actions executed by a swimming robot. These action sequences are chosen to optimize robustness and transmission duration given the observability, natural activity of the robot and the frequency of different messages. Our approach uses a convolutional network to make core observations of the pose of the robot being tracked, which is sending messages. The observer robot then uses an adaptation of classical decoding methods to infer a message that is being transmitted. The system is trained and validated using simulated data, tested in the pool and is targeted for deployment in the open ocean. Our decoder achieves.94 precision and.66 recall on real footage of robot gesture execution recorded in a swimming pool.

We propose a system for visual simultaneous localization and mapping (SLAM) that combines traditional local appearance-based features with semantically meaningful object landmarks to achieve both accurate local tracking and highly view-invariant object-driven relocalization. Our mapping process uses a sampling-based approach to efficiently infer the 3D pose of object landmarks from 2D bounding box object detections. These 3D landmarks then serve as a view-invariant representation which we leverage to achieve camera relocalization even when the viewing angle changes by more than 125 degrees. This level of view-invariance cannot be attained by local appearance-based features (e.g. SIFT) since the same set of surfaces are not even visible when the viewpoint changes significantly. Our experiments show that even when existing methods fail completely for viewpoint changes of more than 70 degrees, our method continues to achieve a relocalization rate of around 90%, with a mean rotational error of around 8 degrees.

Physical sampling of water for off-site analysis is necessary for many applications like monitoring the quality of drinking water in reservoirs, understanding marine ecosystems, and measuring contamination levels in fresh-water systems. In this paper, the focus is on algorithms for efficient measurement and sampling using a multi-robot, data-driven, water-sampling behavior, where autonomous surface vehicles plan and execute water sampling using the chlorophyll density as a cue for plankton-rich water samples. We use two Autonomous Surface Vehicles (ASVs), one equipped with a water quality sensor and the other equipped with a water-sampling apparatus. The ASV with the sensor acts as an explorer, measuring and building a spatial map of chlorophyll density in the given region of interest. The ASV equipped with the water sampling apparatus makes decisions in real time on where to sample the water based on the suggestions made by the explorer robot. We evaluate the system in the context of measuring chlorophyll distributions. We do this both in simulation based on real geophysical data from MODIS measurements, and on real robots in a water reservoir. We demonstrate the effectiveness of the proposed approach in several ways including in terms of mean error in the interpolated data as a function of distance traveled.

We address the integrated prediction, planning, and control problem that enables a single follower robot (the photographer) to quickly re-establish visual contact with a moving target (the subject) that has escaped the follower's field of view. We deal with this scenario, which reactive controllers are typically ill-equipped to handle, by making plausible predictions about the long- and short-term behavior of the target, and planning pursuit paths that will maximize the chance of seeing the target again. At the core of our pursuit method is the use of predictive models of target behavior, which help narrow down the set of possible future locations of the target to a few discrete hypotheses, as well as the use of combinatorial search in physical space to check those hypotheses efficiently. We model target behavior in terms of a learned navigation reward function, using Inverse Reinforcement Learning, based on semantic terrain features of satellite maps. Our pursuit algorithm continuously predicts the latent destination of the target and its position in the future, and relies on efficient graph representation and search methods in order to navigate to locations at which the target is most likely to be seen at an anticipated time. We perform extensive evaluation of our predictive pursuit algorithm over multiple satellite maps, thousands of simulation scenarios, against state-of-the art MDP and POMDP solvers. We show that our method significantly outperforms them by exploiting domain-specific knowledge, while being able to run in real-time.

This paper considers a spatial coverage problem in which a network of passive floating sensors is used to collect samples in a body of water. We employ an iterative measurement and modeling scheme to incrementally deploy sensors so as to achieve spatial coverage, despite only controlling the initial sample point. Once deployed, sensors are moved about a survey area by ambient surface currents. We demonstrate our results in simulation on 40 different ocean flow fields and compare against several baselines. This work provides a computational tool for scientists seeking a low-cost, autonomous marine surveying system. Although in this paper, we concentrate on ocean drifters, our approach can be extended to other domains where a spatial distribution of passive nodes in a flow field can be modeled.

We present an algorithm for rapidly learning neural network policies for robotics systems. The algorithm follows the model-based reinforcement learning paradigm and improves upon existing algorithms: PILeO and a sample-based version of PILeo with neural network dynamics (Deep-PILeO). To improve convergence, we propose a model-based algorithm that uses fixed random numbers and clips gradients during optimization. We propose training a neural network dynamics model using variational dropout with truncated Log-Normal noise. These improvements enable data-efficient synthesis of complex neural network policies. We test our approach on a variety of benchmark tasks, demonstrating data-efficiency that is competitive with that of PILeO, while being able to optimize complex neural network controllers. Finally, we assess the performance of the algorithm for learning motor controllers for a six legged autonomous underwater vehicle. This demonstrates the potential of the algorithm for scaling up the dimensionality and dataset sizes, in more complex tasks.

Visual localization under large changes in scale is an important capability in many robotic mapping applications, such as localizing at low altitudes in maps built at high altitudes, or performing loop closure over long distances. Existing approaches, however, are robust only up to about a 3× difference in scale between map and query images. We propose a novel combination of deep-learning-based object features and state-of-the-art SIFT point-features that yields improved robustness to scale change. This technique is training-free and class-agnostic, and in principle can be deployed in any environment out-of-the-box. We evaluate the proposed technique on the KITTI Odometry benchmark and on a novel dataset of outdoor images exhibiting changes in visual scale of 7× and greater, which we have released to the public. Our technique consistently outperforms localization using either SIFT features or the proposed object features alone, achieving both greater accuracy and much lower failure rates under large changes in scale.

We address the problem of learning vision-based, collision-avoiding, and target-selecting controllers in 3D, specifically in underwater environments densely populated with coral reefs. Using a highly maneuverable, dynamic, six-legged (or flippered) vehicle to swim underwater, we exploit real time visual feedback to make close-range navigation decisions that would be hard to achieve with other sensors. Our approach uses computer vision as the sole mechanism for both collision avoidance and visual target selection. In particular, we seek to swim close to the reef to make observations while avoiding both collisions and barren, coral-deprived regions. To carry out path selection while avoiding collisions, we use monocular image data processed in real time. The proposed system uses a convolutional neural network that takes an image from a forward-facing camera as input and predicts unscaled and relative path changes. The network is trained to encode our desired obstacle-avoidance and reef-exploration objectives via supervised learning from human-labeled data. The predictions from the network are transformed into absolute path changes via a combination of a temporally-smoothed proportional controller for heading targets and a low-level motor controller. This system enables safe and autonomous coral reef navigation in underwater environments. We validate our approach using an untethered and fully autonomous robot swimming through coral reef in the open ocean. Our robot successfully traverses 1000 m of the ocean floor collision-free while collecting close-up footage of coral reefs.

We present an approach to learning control policies for physical robots that achieves high efficiency by adjusting existing policies that have been learned on similar source systems, such as a similar robot with different physical parameters, or an approximate dynamics model simulator. This can be viewed as calibrating a policy learned on a source system, to match a desired behaviour in similar target systems. Our approach assumes that the trajectories described by the source robot are feasible on the target robot. By making this assumption, we only need to learn a mapping from the source robot state and action spaces to the target robot action space, which we call a policy adjustment model. We demonstrate our approach in simulation in the cart-pole balancing task and a two link double pendulum. We also validate our approach with a physical cart-pole system, where we adjust a learned policy under changes to the weight of the pole.

Many interesting natural phenomena are sparsely distributed and discrete. Locating the hotspots of such sparsely distributed phenomena is often difficult because their density gradient is likely to be very noisy. We present a novel approach to this search problem, where we model the co-occurrence relations between a robot's observations with a Bayesian nonparametric topic model. This approach makes it possible to produce a robust estimate of the spatial distribution of the target, even in the absence of direct target observations. We apply the proposed approach to the problem of finding the spatial locations of the hotspots of a specific phytoplankton taxon in the ocean. We use classified image data from Imaging FlowCytobot (IFCB), which automatically measures individual microscopic cells and colonies of cells. Given these individual taxon-specific observations, we learn a phytoplankton community model that characterizes the co-occurrence relations between taxa. We present experiments with simulated robot missions drawn from real observation data collected during a research cruise traversing the US Atlantic coast. Our results show that the proposed approach outperforms nearest neighbor and k-means based methods for predicting the spatial distribution of hotspots from in-situ observations.

We propose an approach to vision-based pose estimation using object recognition and identity. Whereas feature based scene recognition and pose estimation methods are well established as effective means for estimating motion and recognizing locations, feature-based methods depend critically on the detection of common local features from one view of a scene to another. We focus on place recognition and pose change estimation in the context of large changes in viewing position, even to the extent that no common surfaces are seen between the two views. Our approach is based on using object identities and their inter-relationship to compute pose change. An important secondary outcome of our method is that it simultaneously infers the 3D poses of objects in the scene that are used as features. Such an object-based approach is inspired by a vast literature on human perception and has the potential for great robustness, albeit at the expense of accuracy. We propose a formulation of the problem using pairwise contextual constraints and develop an efficient algorithmic solution. We validate the approach and quantify its performance using the publicly available TUM SLAM dataset [1].

This paper addresses adaptive coverage of a spatial field without prior knowledge. Our application in this paper is to cover a region of the sea surface using a robotic boat, although the algorithmic approach has wider applicability. We propose an anytime planning technique for efficient data gathering using point-sampling based on non-uniform data-driven coverage. Our goal is to sense a particular region of interest in the environment and be able to reconstruct the measured spatial field. Since there are autonomous agents involved, there is a need to consider the costs involved in terms of energy consumed and time required to finish the task. An ideal map of the scalar field requires complete coverage of the region, but can be approximated by a good sparse coverage strategy along with an efficient interpolation technique. We propose to optimize the trade off between the environmental field mapping and the costs (energy consumed, time spent, and distance traveled) associated with sensing. We present an anytime algorithm for sampling the environment adaptively by following a multi-scale path to produce a variable resolution map of the spatial field. We compare our approach to a traditional exhaustive survey approach and show that we are able to effectively represent a spatial field spending minimum energy. We present results that indicate our sampling technique gathering most informative samples with least travel. We validate our approach through simulations and test the system on real robots in the open ocean.

We present a robust multi-robot convoying approach that relies on visual detection of the leading agent, thus enabling target following in unstructured 3-D environments. Our method is based on the idea of tracking-by-detection, which interleaves efficient model-based object detection with temporal filtering of image-based bounding box estimation. This approach has the important advantage of mitigating tracking drift (i.e. drifting away from the target object), which is a common symptom of model-free trackers and is detrimental to sustained convoying in practice. To illustrate our solution, we collected extensive footage of an underwater robot in ocean settings, and hand-annotated its location in each frame. Based on this dataset, we present an empirical comparison of multiple tracker variants, including the use of several convolutional neural networks, both with and without recurrent connections, as well as frequency-based model-free trackers. We also demonstrate the practicality of this tracking-by-detection strategy in real-world scenarios by successfully controlling a legged underwater robot in five degrees of freedom to follow another robot's independent motion.

We address the integrated planning and control problem that enables a single follower robot (the “photographer”) to maintain a moving target (the “subject”) in its field of view for as long as possible. We propose a real-time pursuit algorithm that seamlessly handles the often neglected, yet unavoidable, scenario in which the target escapes the follower's field of view; a scenario that simple, reactive controllers are ill-equipped to handle. Our algorithm aims to minimize the expected time until visual contact is re-established, which enables the photographer to track the subject for as long as possible, even in the presence of loss of visibility. At the core of our pursuit algorithm is an efficient method for sampling plausible trajectories from different homotopy classes. We do this by generating topologically distinct shortest paths by using the Voronoi diagram. We use these paths to make informed, model-based predictions of the likely future locations of the target, given a history of observations. Given these predictions, our algorithm produces pursuit trajectories that approximately minimize the expected time to recover visual contact. We show that constraining the predictive pursuit problem to the space of homotopy classes condenses the expanse of possibilities that our algorithm must consider, which enables target tracking in large occupancy grids, as opposed to many POMDP methods that are constrained to small environments. We benchmark the tracking behavior of our algorithm against the baseline of human subjects who performed the same set of pursuit tasks in simulation, as well as against two other pursuit algorithms that only take into account paths from a single homotopy class. We show that considering homotopy alternatives in 2D pursuit improves the tracking performance and that our algorithm does at least as well as humans in most pursuit scenarios.

This paper presents an approach to learn meaningful spatial relationships in an unsupervised fashion from the distribution of 3D object poses in the real world. Our approach begins by extracting an over-complete set of features to describe the relative geometry of two objects. Each relationship type is modeled using a relevance-weighted distance over this feature space. This effectively ignores irrelevant feature dimensions. Our algorithm RANSEM for determining subsets of data that share a relationship as well as the model to describe each relationship is based on robust sample-based clustering. This approach combines the search for consistent groups of data with the extraction of models that precisely capture the geometry of those groups. An iterative refinement scheme has shown to be an effective approach for finding concepts of differing degrees of geometric specificity. Our results show that the models learned by our approach correlate strongly with the English labels that have been given by a human annotator to a set of validation data drawn from the NYUv2 real-world Kinect dataset, demonstrating that these concepts can be automatically acquired given sufficient experience. Additionally, the results of our method significantly out-perform K-means, a standard baseline for unsupervised cluster extraction.

We address the problem of rendezvous between two agents in urban street networks. Specifically, we consider the case where the agents have variable speeds and they need to schedule a rendezvous or a meeting under uncertainty in their travel times. Examples of such a scenario range from everyday life where two people would like to coordinate a meeting while going from office to home; to a futuristic case where automated taxis would like to meet each other for load balancing passengers. The scheduling for such scenarios can easily become challenging with uncertainties such as delayed departures, road blocks due to construction or traffic congestion. Any solution for such a task is required to minimize the waiting time and the planning overhead. In this paper, we propose an algorithm that optimizes the total travel time and the waiting time for two agents to complete their respective paths from start to rendezvous and from rendezvous to goal locations subject to delays along their paths. We validate our approach with a street network database which has a cost associated with every query made to the database server. Thus our algorithm intelligently optimizes for rendezvous trajectories that effectively mitigate the scourge of traffic delays, while simultaneously limiting the number of queries through careful analysis of the informative value of each potential query.

In this paper, we examine multi-target search, where one or more targets must be found by a moving robot. Given the target's initial probability distribution or the expected search region, we present an analysis of three search strategies - Global maxima search, Local maxima search, and Spiral search. We aim at minimizing the mean-time-to-find and maximizing the total probability of finding the target. This leads to two types of illustrative performance metrics: minimum time capture and guaranteed capture. We validate the search strategies with respect to these two performance metrics. In addition, we study the effect of different target distributions on the performance of the search strategies. We also consider the practical realization of the proposed algorithms for multi-target search. The search strategies are analytically evaluated, through simulations and illustrative deployments, in open-water with an Autonomous Surface Vehicle (ASV) and drifting sensor targets.

In this work, we grant robot agents the capacity to sense and react to their human supervisor's changing trust state, as a means to maintain the efficiency of their collaboration. We propose the novel formulation of Trust-Aware Conservative Control (TACtiC), in which the agent alters its behaviors momentarily whenever the human loses trust. This trust-seeking robot framework builds upon an online trust inference engine and also incorporates an interactive behavior adaptation technique. We present end-to-end instantiations of trust-seeking robots for distinct task domains of aerial terrain coverage and interactive autonomous driving. Empirical assessments comprise a large-scale controlled interaction study and its extension into field evaluations with an autonomous car. These assessments substantiate the efficiency gains that trust-seeking agents bring to asymmetric human-robot teams.

We present OPTIMo: an Online Probabilistic Trust Inference Model for quantifying the degree of trust that a human supervisor has in an autonomous robot "worker". Represented as a Dynamic Bayesian Network, OPTIMo infers beliefs over the human's moment-to-moment latent trust states, based on the history of observed interaction experiences. A separate model instance is trained on each user's experiences, leading to an interpretable and personalized characterization of that operator's behaviors and attitudes. Using datasets collected from an interaction study with a large group of roboticists, we empirically assess OPTIMo's performance under a broad range of configurations. These evaluation results highlight OPTIMo's advances in both prediction accuracy and responsiveness over several existing trust models. This accurate and near real-time human-robot trust measure makes possible the development of autonomous robots that can adapt their behaviors dynamically, to actively seek greater trust and greater efficiency within future human-robot collaborations.

In this paper, we investigate the question of how a legged robot can walk efficiently by taking advantage of its ability to alter its gait as a function of statistical (large-scale) terrain properties. One of the contributions of this paper is the algorithm to achieve real-time terrain identification and autonomous gait adaptation on a legged robot. We approach this problem by first classifying the terrains based on their proprioceptive responses and identifying the terrain in real-time. Then we choose an optimal gait to best suit the identified terrain type. We exploit our recent findings regarding gaits, estimated from terrain-contact signatures, in order to obtain an optimized mapping between terrain signatures and terrain-specific gaits. We evaluate our algorithm on synthetic data, and real robot data collected on different terrains and naturally occurring terrain transitions. Another key contribution of this work is the statistical verification that precise gait selection can lead to energy savings in practice in legged robots. This assessment of energy efficiency, achieved by gait adaptation, is among the firsts of its kind in gait adaptation literature. We also present an analysis of the effect of terrain transition frequency on our gait adaptation algorithm. Our results are supported by validation using both synthetic data and field testing.

We present an end-to-end framework for realizing fully automated gait learning for a complex underwater legged robot. Using this framework, we demonstrate that a hexapod flipper-propelled robot can learn task-specific control policies purely from experience data. Our method couples a state-of-the-art policy search technique with a family of periodic low-level controls that are well suited for underwater propulsion. We demonstrate the practical efficacy of tabula rasa learning, that is, learning without the use of any prior knowledge, of policies for a six-legged swimmer to carry out a variety of acrobatic maneuvers in three dimensional space. We also demonstrate informed learning that relies on simulated experience from a realistic simulator. In numerous cases, novel emergent gait behaviors have arisen from learning, such as the use of one stationary flipper to create drag while another oscillates to create thrust. Similar effective results have been demonstrated in under-actuated configurations, where as few as two flippers are used to maneuver the robot to a desired pose, or through an acrobatic motion such as a corkscrew. The success of our learning framework is assessed both in simulation and in the field using an underwater swimming robot.

We consider how to directly extract a road map (also known as a topological representation) of an initially-unknown 2-dimensional environment via an on-line procedure which robustly computes a retraction of its boundaries. While such approaches are well known for their theoretical elegance, computing such representations in practice is complicated when the data is sparse and noisy. In this paper we present the online construction of a topological map and the implementation of a control law for guiding the robot to the nearest unexplored area. The proposed method operates by allowing the robot to localize itself on a partially constructed map, calculate a path to unexplored parts of the environment (frontiers), compute a robust terminating condition when the robot has fully explored the environment, and achieve loop closure detection. The proposed algorithm results in smooth safe paths for the robot's navigation needs. The presented approach is an any-time-algorithm which allows for the active creation of topological maps from laser-scan data, as it is being acquired. The resulting map is stable under variations to noise and the initial conditions. The key idea is the use of a flux-based skeletonization algorithm on the latest occupancy grid map. We also propose a navigation strategy based on a heuristic where the robot is directed towards nodes in the topological map that open to empty space. The method is evaluated on both synthetic data and in the context of active exploration using a Turtlebot 2. Our results demonstrate complete mapping of different environments with smooth topological abstraction without spurious edges.

We present a robotic exploration technique in which the goal is to learn a visual model that can be used to distinguish between different terrains and other visual components in an unknown environment. We use ROST, a realtime online spatiotemporal topic modeling framework to model these terrains using the observations made by the robot, and then use an information theoretic path planning technique to define the exploration path. We conduct experiments with aerial view and underwater datasets with millions of observations and varying path lengths, and find that paths that are biased towards locations with high topic perplexity produce better terrain models with high discriminative power.

In this paper we present an algorithm for finding a distance optimal rendezvous location with respect to both initial and target locations of the mobile agents. These agents can be humans or robots, who need to meet and split while performing a collaborative task. Our aim is to embed the meeting process within a background activity such that the agents travel through the rendezvous location while taking the shortest paths to their respective target locations. We analyze this problem in a street network scenario with two agents who are given their individual scheduled routes to complete with an underlying common goal. The agents are allowed to select any combination of the waypoints along their routes as long as they travel the shortest path and pass through the same potential rendezvous location. The total number of path combinations that the agents need to evaluate for the shortest path increases rapidly with the number of waypoints along their routes. We address this computational cost by proposing a combination of Euclidean and street network distances for a trade-off between the number of queries and a distance optimal solution.

In this paper we address the issue of coordinating the trajectories of two collaborating robots in environments with obstacles so that visibility between them is maximized in the presence of competing constraints. Specifically, we examine the problem of allowing one robot (the “photographer”) to follow another robot (“the subject”) through a planar environment while maintaining visual contact to the maximum degree consistent with an efficient traversal. This problem has numerous applications, for instance in scenarios where communication between robots requires line-of-sight. We formalize this problem in the context of centralized kinodynamic planning and we present solutions based on the asymptotically optimal sampling-based RRT* planner. We discuss connections to the traditional formulation of pursuit-evasion games where the analysis typically ends the moment the evader manages to escape the pursuer's visibility region. We also illustrate types of environments and other conditions under which allowing the pair of robots to break the line-of-sight is a better option than always requiring the presence of visual contact.

The problem of Adaptation from Participation (AfP) aims to improve the efficiency of a human-robot team by adapting a robot's autonomous systems and behaviors based on command-level input from a human supervisor. As a solution to AfP, the Adaptive Parameter EXploration (APEX) algorithm continuously explores the space of all possible parameter configurations for the robot's autonomous system in an online and anytime manner. Guided by information deduced from the human's latest intervening commands, APEX is capable of adapting an arbitrary robot system to dynamic changes in task objectives and conditions during a session. We explore this framework within visual navigation contexts where the humanrobot team is tasked with covering or patrolling over multiple terrain boundaries such as coastlines and roads. We present empirical evaluations of two separate APEX-enabled systems: the first, deployed on an aerial robot within a controlled environment, and the second, on a wheeled robot operating within a challenging university campus setting.

Inspection and exploration of complex underwater structures requires the development of agile and easy to program platforms. In this paper, we describe a system that enables the deployment of an autonomous underwater vehicle in 3D environments proximal to the ocean bottom. Unlike many previous approaches, our solution: uses oscillating hydrofoil propulsion; allows for stable control of the robot's motion and sensor directions; allows human operators to specify detailed trajectories in a natural fashion; and has been successfully demonstrated as a holistic system in the open ocean near both coral reefs and a sunken cargo ship. A key component of our system is the 3D control of a hexapod swimming robot, which can move the vehicle through agile sequences of orientations despite challenging marine conditions. We present two methods to easily generate robot trajectories appropriate for deployments in close proximity to challenging contours of the sea floor. Both offline recording of trajectories using augmented reality and online placement of fiducial tags in the marine environment are shown to have desirable properties, with complementary strengths and weaknesses. Finally, qualitative and quantitative results of the 3D control system are presented.

We study the problem of distributing a single global task between a group of heterogeneous robots. We view this problem as a fair division game. In this setting, every robot defines a preference function over parts of the task according to its sensing and motion capabilities. These preferences are described by density functions over the task. With such interpretation, we want to find an allocation of the global task that maximizes the probability of task completion. We first formulate the task distribution problem as a fair subdivision problem and provide a centralized algorithm to compute the allocations for each robot. We provide a complexity analysis and computational results of the algorithm.

We discuss the problem of automatically discovering different acoustic regions in the world, and then labeling the trajectory of a robot using these region labels. We use quantized Mel Frequency Cepstral Coefficients (MFCC) as low level features, and a temporally smoothed variant of Latent Dirichlet Allocation (LDA) to compute both the region models, and most likely region labels associated with each time step in the robot's trajectory. We validate our technique by showing results from two datasets containing sound recorded from 51 and 43 minute long trajectories through downtown Montreal and the McGill University campus. Our preliminary experiments indicate that the regions discovered by the proposed technique correlate well with ground truth, labeled by a human expert.

In this paper we examine pursuit-evasion games in which the pursuer has higher speed than the evader. This scenario is motivated by visibility-based pursuit-evasion problems, particularly by the question of what happens when the pursuer loses visual track of the moving evader. In these cases the pursuer has two options for recovering visual contact with the evader: to perform search over the possible locations where the evader might be moving, or to clear the environment, in other words to progressively search it without allowing the evader to move into locations that have already been cleared. It has been shown that in sufficiently complex environments a single pursuer having the same speed as the evader cannot clear the environment. In this work we prove that computing the minimum speed which enables a faster pursuer to clear a graph environment is NP-hard. In light of this result we provide an experimental comparison of randomized and deterministic search strategies on planar graphs, which has practical significance in search and rescue settings.

In this paper we propose a design of a class of robotic legs (known as “Ninja legs”) that enable amphibious operation, both walking and swimming, for use on a class of hexapod robots. Amphibious legs equip the robot with a capability to explore diverse locations in the world encompassing both those that are on the ground as well as underwater. In this paper we work with a hexapod robot of the Aqua vehicle family (based on a body plan first developed by Buehler et al. [1]), which is an amphibious robot that employs legs for amphibious locomotion. Many different leg designs have been previously developed for Aqua-class vehicles, including both robust all-terrain legs for walking, and efficient flippers for swimming. But the walking legs have extremely poor thrust for swimming and the flippers are completely unsuitable for terrestrial operations. In this work we propose a single leg design with the advantages of both the walking legs and the swimming flippers. We design a cage-like circular enclosure for the flippers in order to protect the flippers during terrestrial operations. The enclosing structure also plays the role of the walking legs for terrestrial locomotion. The circular shape of the enclosure, as well, has the advantages of an offset wheel. We evaluate the performance of our design for terrestrial mobility by comparing the power efficiency and the physical speed of the robot equipped with the newly designed legs against that with the walking legs which are semi-circular in shape. The swimming performance is examined by measuring the thrust generated by newly designed legs and comparing the same with the thrust generated by the swimming flippers. In the field, we also verified that these legs are suitable for swimming through moderate surf, walking through the breakers on a beach (and thus through slurry), and onto wet and dry sand.

We are interested in the task of online summarization of the data observed by a mobile robot, with the goal that these summaries could be then be used for applications such as surveillance, identifying samples to be collected by a planetary rover, and site inspections to detect anomalies. In this paper, we pose the summarization problem as an instance of the well known k-center problem, where the goal is to identify k observations so that the maximum distance of any observation from a summary sample is minimized. We focus on the online version of the summarization problem, which requires that the decision to add an incoming observation to the summary be made instantaneously. Moreover, we add the constraint that only a finite number of observed samples can be saved at any time, which allows for applications where the selection of a sample is linked to a physical action such as rock sample collection by a planetary rover. We show that the proposed online algorithm has performance comparable to the offline algorithm when used with real world data.

We describe a model of “trust” in human-robot systems that is inferred from their interactions, and inspired by similar concepts relating to trust among humans. This computable quantity allows a robot to estimate the extent to which its performance is consistent with a human's expectations, with respect to task demands. Our trust model drives an adaptive mechanism that dynamically adjusts the robot's autonomous behaviors, in order to improve the efficiency of the collaborative team. We illustrate this trust-driven methodology through an interactive visual robot navigation system. This system is evaluated through controlled user experiments and a field demonstration using an aerial robot.

We address the problem of arranging a meeting (or rendezvous) between two or more robots in an unknown bounded topological environment, starting at unknown locations, without any communication. The goal is to rendezvous in minimum time such that the robots can share resources for performing any global task. We specifically consider a global exploration task executed by two or more robots. Each robot explores the environment simultaneously, for a specified time, then selects potential rendezvous locations, where it expects to find other robots, and visits them. We propose a ranking criterion for selecting the order in which potential rendezvous locations will be visited. This ranking criterion associates a cost for visiting a rendezvous location and gives an expected reward of finding other agents. We evaluate the time taken to rendezvous by varying a set of conditions including: world size, number of robots, starting location of each robot and the presence of sensor noise. We present simulation results to quantify the effect of the aforementioned factors on the rendezvous time.

In this paper we describe a heterogeneous multi-robot system for assisting scientists in environmental monitoring tasks, such as the inspection of marine ecosystems. This team of robots is comprised of a fixed-wing aerial vehicle, an autonomous airboat, and an agile legged underwater robot. These robots interact with off-site scientists and operate in a hierarchical structure to autonomously collect visual footage of interesting underwater regions, from multiple scales and mediums. We discuss organizational and scheduling complexities associated with multi-robot experiments in a field robotics setting. We also present results from our field trials, where we demonstrated the use of this heterogeneous robot team to achieve multi-domain monitoring of coral reefs, based on real-time interaction with a remotely-located marine biologist.

In this paper we focus on the task of summarizing observations made by a mobile robot on a trajectory. A navigation summary is the synopsis of these observations. We pose the problem of generating navigation summaries as a sampling problem. The goal is to select a few samples from the set of all observations, which are characteristic of the environment, and capture its mean properties and surprises. We define the surprise score of an observation as its distance to the closest sample in the summary. Hence, an ideal summary is defined to have a low mean and a low max surprise score, measured over all the observations. We present three different strategies for solving this sampling problem. Of these, we show that the kCover sampling algorithm produces summaries with low mean and max surprise scores; even in the presence of noise. These results are demonstrated on datasets acquired in different robotics context.

We describe a framework that combines a software development paradigm, a software visualization technique, and a tool for robot programming. This infrastructure is called "Graphical State Space Programming" (GSSP), and allows robot application programs to be decomposed and visualized within state-dependent views. Our approach simplifies and expedites the programming process for robot routines and behaviors, and we examine the performance improvement that ensues through a set of controlled user studies. The usability and effectiveness of GSSP are also illustrated using a field demonstration with an aerial robotic vehicle.

We present a technique for robust human-robot interaction taking into consideration uncertainty in input and task execution costs incurred by the robot. Specifically, this research aims to quantitatively model confirmation feedback, as required by a robot while communicating with a human operator to perform a particular task. Our goal is to model human-robot interaction from the perspective of risk minimization, taking into account errors in communication, "risk" involved in performing the required task, and task execution costs. Given an input modality with non-trivial uncertainty, we calculate the cost associated with performing the task specified by the user, and if deemed necessary, ask the user for confirmation. The estimated task cost and the uncertainty measure are given as input to a Decision Function, the output of which is then used to decide whether to execute the task, or request clarification from the user. We test our system through human-interface experiments, based on a framework custom-designed for our family of amphibious robots, and demonstrate the utility of the framework in the presence of large task costs and uncertainties. We also present qualitative results of our algorithm from field trials of our robots in both open-and closed-water environments.

We present MARE, an autonomous airboat robot that is suitable for exploration-oriented tasks, such as inspection of coral reefs and shallow seabeds. The combination of this platform's particular mechanical properties and its powerful software framework enables it to function in a multitude of potential capacities, including autonomous surveillance, mapping, and search operations. In this paper we describe two different exploration strategies and their implementation using the MARE platform. First, we discuss the application of an efficient coverage algorithm, for the purpose of achieving systematic exploration of a known and bounded environment. Second, we present an exploration strategy driven by surprise, which steers the robot on a path that might lead to potentially surprising observations.

This paper presents an adaptation of a vision and inertial-based state estimation algorithm for use in an underwater robot. The proposed approach combines information from an Inertial Measurement Unit (IMU) in the form of linear accelerations and angular velocities, depth data from a pressure sensor, and feature tracking from a monocular downward facing camera to estimate the 6DOF pose of the vehicle. To validate the approach, we present extensive experimental results from field trials conducted in underwater environments with varying lighting and visibility conditions, and we demonstrate successful application of the technique underwater.

This paper describes a tactile probe designed for surface identification in a context of all-terrain low-velocity mobile robotics. The proposed tactile probe is made of a small metallic rod with a single-axis accelerometer attached near its tip. Surface identification is based on analyzing acceleration patterns induced at the tip of this mechanically robust tactile probe, while it is passively dragged along a surface. A training dataset was collected over ten different indoor and outdoor surfaces. Classification results for an artificial neural network were positive, with an 89.9% and 94.6% success rate for 1- and 4-s time windows of data, respectively. We also demonstrated that the same tactile probe can be used for unsupervised learning of terrains. For 1-s time windows of data, the classification success rate was only reduced to 74.1%. Finally, a blind mobile robot, performing real-time classification of surfaces, demonstrated the feasibility of this tactile probe as a guidance mechanism.

Our objective is to find a small set of images that summarize a robot's visual experience along a path. We present a novel on-line algorithm for this task. This algorithm is based on a new extension to the classical Secretaries Problem. We also present an extension to the idea of Bayesian Surprise, which we then use to measure the fitness of an image as a summary image.

We present an interface for controlling mobile robots that combines aspects of graphical trajectory specification and state-based programming. This work is motivated by common tasks executed by our underwater vehicles, although we illustrate a mode of interaction that is applicable to mobile robotics in general. The key aspect of our approach is to provide an intuitive linkage between the graphical visualization of regions of interest in the environment, and activities relevant to these regions. In addition to introducing this novel programming paradigm, we also describe the associated system architecture developed on-board our amphibious robot. We then present a user interaction study that illustrates the benefits in usability of our graphical interface, compared to conventionally established programming techniques.

We propose an algorithm for generating navigation summaries. Navigation summaries are a specialization of video summaries, where the focus is on video collected by a mobile robot, on a specified trajectory. We are interested in finding a few images that epitomize the visual experience of a robot as it traverses a terrain. This paper presents a novel approach to generating summaries in form of a set of images, where the decision to include the image in the summary set is made online. Our focus is on the case where the number of observations is infinite or unknown, but the size of the desired summary is known. Our strategy is to consider the images in the summary set as the prior hypothesis of the appearance of the world, and then use Set Theoretic Surprise to compute the novelty of an observed image. If the novelty is above a threshold, then we accept the image. We discuss different criterion for setting this threshold. Online nature of our approach allows for several interesting applications such as coral reef inspection, surveying, and surveillance.

We present an integration of classical computer vision techniques to achieve real-time autonomous steering of an unmanned aircraft along the boundary of different regions. Using an unified conceptual framework, we illustrate solutions for tracking coastlines and for following roads surrounded by forests. In particular, we exploit color and texture properties to differentiate between region types in the aforementioned domains. The performance of our system is evaluated using different experimental approaches, which includes a fully automated in-field flight over a 1km coastline trajectory.

In this paper, we investigate a pure form of the topological mapping problem in mobile robotics. We consider the mapping ability of a robot navigating a graph-like world in which it is able to assign a relative ordering to the edges, leaving a vertex with reference to the edge by which it arrived but is unable to associate a unique label with any vertex or edge. Our work extends and builds upon earlier approaches in this problem domain, which are based on construction of exploration tree of plausible world models. The main contributions of the paper are improved exploration strategies that reduce model ambiguity, a new method of search through consistent models in the exploration tree that maintains a bounded set of likely hypotheses based on the principle of Occam's Razor, the incorporation of arbitrary feature vectors into the problem formulation, and an investigation of various aspects of this problem through numerical simulations.

This paper describes an approach to surface identification in the context of mobile robotics, applicable to supervised and unsupervised learning. The identification is based on analyzing the tip acceleration patterns induced in a metallic rod, dragged along a surface that is to be identified. Eight features in time and frequency domains are used for classification. Results show that for ten type of indoor and outdoor surfaces, reliable identification can be achieved (90.0 and 94.6 percent for a 1 and 4 seconds time-window, respectively), using a non-sophisticated classifier (artificial neural network). Demonstration is done on how such a sensor and a simple control strategy can be used to guide a blind robot, using a simulation and a real differential drive robot.

In this paper we present an approach for localizing a sensor network augmented with a mobile robot which is capable of providing inter-sensor pose estimates through its odometry measurements. We present a stochastic algorithm that samples efficiently from the probability distribution for the pose of the sensor network by employing Rao-Blackwellization and a proposal scheme which exploits the sequential nature of odometry measurements. Our algorithm automatically tunes itself to the problem instance and includes a principled stopping mechanism based on convergence analysis. We demonstrate the favourable performance of our approach compared to that of established methods via simulations and experiments on hardware.

We present an application of machine learning to the semi-automatic synthesis of robust servo-trackers for underwater robotics. In particular, we investigate an approach based on the use of Boosting for robust visual tracking of color objects in an underwater environment. To this end, we use AdaBoost, the most common variant of the Boosting algorithm, to select a number of low-complexity but moderately accurate color feature trackers and we combine their outputs. The novelty of our approach lies in the design of this family of weak trackers, which enhances a straightforward color segmentation tracker in multiple ways. From a large and diverse family of possible filters, we select a small subset that optimizes the performance of our trackers. The tracking process applies these trackers on the input video frames, and the final tracker output is chosen based on the weights of the final array of trackers. By using computationally inexpensive, but somewhat accurate trackers as members of the ensemble, the system is able to run at quasi real-time, and thus, is deployable on-board our underwater robot. We present quantitative cross-validation results of our spatio-chromatic visual tracker, and conclude by pointing out some difficulties faced and subsequent shortcomings in the experiments we performed, along with directions of future research in the area of ensemble tracking in real-time.

We present a bimodal information analysis system for automatic emotion recognition. Our approach is based on the analysis of video sequences which combines facial expressions observed visually with acoustic features to automatically recognize five universal emotion classes: anger, disgust, happiness, sadness and surprise. We address the challenges posed during the temporal analysis of the bimodal data and introduce a novel technique for combining the best features of instantaneous and temporal based visual recognition systems. We obtain robust appearance-based visual features which we classify instantaneously and aggregate it temporally to improve the recognition rates when compared to single-frame based instantaneous classification. The performance of the system is further boosted by using the complementary audio information for the bimodal emotion recognition. We combine the two modalities at both feature and score level to compare the respective joint emotion recognition rates. The emotions are instantaneously classified using a support vector machine and sequentially aggregated based on their classification probabilities. This approach is validated on a posed audio-visual database and a natural interactive database. The experiments performed on these databases provide encouraging results with the best combined recognition rate being 82%.

A gesture-based interaction framework is presented for controlling mobile robots. This natural interaction paradigm has few physical requirements, and thus can be deployed in many restrictive and challenging environments. We present an implementation of this scheme in the control of an underwater robot by an on-site human operator. The operator performs discrete gestures using engineered visual targets, which are interpreted by the robot as parametrized actionable commands. By combining the symbolic alphabets resulting from several visual cues, a large vocabulary of statements can be produced. An iterative closest point algorithm is used to detect these observed motions, by comparing them with an established database of gestures. Finally, we present quantitative data collected from human participants indicating accuracy and performance of our proposed scheme.

The path followed by a mobile robot while mapping an environment (i.e. an exploration trajectory) plays a large role in determining the efficiency of the mapping process and the accuracy of any resulting metric map of the environment. This paper examines some important aspects of path planning in this context: the trade-offs between the speed of the exploration process versus the accuracy of resulting maps; and alternating between exploration of new territory and planning through known maps. The resulting motion planning strategy and associated heuristic are targeted to a robot building a map of an environment assisted by a Sensor Network composed of uncalibrated monocular cameras. An adaptive heuristic exploration strategy based onA* search over a combined distance and uncertainty cost function allows for adaptation to the environment and improvement in mapping accuracy. We assess the technique using an illustrative experiment in a real environment and a set of simulations in a parametric family of idealized environments.

Underwater operations present unique challenges and opportunities for robotic applications. These can be attributed in part to limited sensing capabilities, and to locomotion behaviours requiring control schemes adapted to specific tasks or changes in the environment. From enhancing teleoperation procedures, to providing high-level instruction, all the way to fully autonomous operations, enabling autonomous capabilities is fundamental for the successful deployment of underwater robots. This paper presents an overview of the approaches used during underwater sea trials in the coral reefs of Barbados, for two amphibious mobile robots and a set of underwater sensor nodes. We present control mechanisms used for maintaining a preset trajectory during enhanced teleoperations and discuss their experimental results. This is followed by a discussion on amphibious data gathering experiments conducted on the beach. We then present a tetherless underwater communication approach based on pure vision for high-level control of an underwater vehicle. Finally the construction details together with preliminary results from a set of distributed underwater sensor nodes are outlined.

We present a method for inferring the topology of a sensor network given nondiscriminating observations of activity in the monitored region. This is accomplished based on no prior knowledge of the relative locations of the sensors and weak assumptions regarding environmental conditions. Our approach employs a two-level reasoning system made up of a stochastic expectation maximization algorithm and a higher level search strategy employing the principle of Occam's Razor to look for the simplest solution explaining the data. The result of the algorithm is a Markov model describing the behavior of agents in the system and the underlying traffic patterns. Numerical simulations and experimental assessment conducted on a real sensor network suggest that the technique could have promising real-world applications in the area of sensor network self-configuration.

We describe an interaction paradigm for controlling a robot using hand gestures. In particular, we are interested in the control of an underwater robot by an on-site human operator. Under this context, vision-based control is very attractive, and we propose a robot control and programming mechanism based on visual symbols. A human operator presents engineered visual targets to the robotic system, which recognizes and interprets them. This paper describes the approach and proposes a specific gesture language called "RoboChat". RoboChat allows an operator to control a robot and even express complex programming concepts, using a sequence of visually presented symbols, encoded into fiducial markers. We evaluate the efficiency and robustness of this symbolic communication scheme by comparing it to traditional gesture-based interaction involving a remote human operator

We present an algorithm for underwater robots to track mobile targets, and specifically human divers, by detecting periodic motion. Periodic motion is typically associated with propulsion underwater and specifically with the kicking of human swimmers. By computing local amplitude spectra in a video sequence, we find the location of a diver in the robot's field of view. We use the Fourier transform to extract the responses of varying intensities in the image space over time to detect characteristic low frequency oscillations to identify an undulating flipper motion associated with typical gaits. In case of detecting multiple locations that exhibit large low-frequency energy responses, we combine the gait detector with other methods to eliminate false detections. We present results of our algorithm on open-ocean video footage of swimming divers, and also discuss possible extensions and enhancements of the proposed approach for tracking other objects that exhibit low- frequency oscillatory motion.

When a network of robots or static sensors is emplaced in an environment, the spatial relationships between the sensing units must be inferred or computed for most key applications. In this paper we present a Monte Carlo expectation maximization algorithm for recovering the connectivity information (i.e. topological map) of a network using only detection events from deployed sensors. The technique is based on stochastically reconstructing samples of plausible agent trajectories allowing for the possibility of transitions to and from sources and sinks in the environment. We demonstrate robustness to sensor error and non-trivial patterns of agent motion. The result of the algorithm is a probabilistic model of the sensor network connectivity graph and the underlying traffic trends. We conclude with results from numerical simulations and an experiment conducted with a heterogeneous sensor network

We consider the use of visual target tracking for autonomous steering of an underwater robot. In this context, we consider a performance comparison for three key visual tracking algorithms used for servo control. We present a comparative study of the performance in underwater environments of three tracking algorithms that are widely used in vision applications. Variations in illumination, suspended particles and a resulting reduction in visibility hinders vision systems from performing satisfactorily in marine environments; at least not as well as they do in terrestrial (Le. non-underwater) surroundings. Our work focuses on quantitatively measuring the performance of three color-based tracking algorithms- color blob tracker, color histogram tracker and mean-shift tracker, in tracking objects underwater in different levels lighting and visibility. We also present results demonstrating the effect of suspended particles underwater, and in conclusion we summarize the three tracking algorithms by comparing their pros and cons

In order to better understand the behavior of the underwater robot developed at our laboratory, a simple but relatively good model of the underwater behavior of the robot had to be developed. In order to be useful for model-based control techniques onboard the robot, the model had to have low computing requirements, yet be complex enough to capture the transient response of the robot. To achieve this, a system identification approach was taken by first capturing the robot response to various inputs, and then matching them to a simple model

We consider the problem of inferring sensor positions and a topological (i.e. qualitative) map of an environment given a set of cameras with non-overlapping fields of view. In this way, without prior knowledge of the environment nor the exact position of sensors within the environment, one can infer the topology of the environment, and common traffic patterns within it. In particular, we consider sensors stationed at the junctions of the hallways of a large building. We infer the sensor connectivity graph and the travel times between sensors (and hence the hallway topology) from the sequence of events caused by unlabeled agents (i.e. people) passing within view of the different sensors. We do this based on a first-order semi-Markov model of the agent's behavior. The paper describes a problem formulation and proposes a stochastic algorithm for its solution. The result of the algorithm is a probabilistic model of the sensor network connectivity graph and the underlying traffic patterns. We conclude with results from numerical simulations

Minimum distance localization is the problem of finding the shortest possible path for a robot to eliminate ambiguity regarding its position in the environment. We consider the problem of minimum distance localization in self-similar environments, where the robot's sensor has limited visibility, and describe two randomized algorithms that solve the problem. Our algorithms reduce the risk of requiring impractical observations and solve the problem without excessive computation. Our results are validated using numerical simulations.

We describe recent results obtained with AQUA, a mobile robot capable of swimming, walking and amphibious operation. Designed to rely primarily on visual sensors, the AQUA robot uses vision to navigate underwater using servo-based guidance, and also to obtain high-resolution range scans of its local environment. This paper describes some of the pragmatic and logistic obstacles encountered, and provides an overview of some of the basic capabilities of the vehicle and its associated sensors. Moreover, this paper presents the first ever amphibious transition from walking to swimming.

In this paper we present a new approach for the online calibration of a camera sensor network. This is the first step towards fully exploiting the potential for collaboration between mobile robots and static sensors sharing the same network. In particular we propose an approach for extracting the 3D pose of each camera in a common reference frame, with the help of a mobile robot. The camera poses can then be used to further refine the robot pose or to perform other tracking tasks. The analytical formulation of the problem of pose recovery is presented together with experimental results of a six node sensor network in different configurations.

This paper describes a visual servoing system for an underwater legged robotic system named AQUA and initial experiments with the system performed in the open sea. A large class of significant applications can be leveraged by allowing such a robot to follow a diver or some other moving target. The robot uses a suite of sensing technologies, primarily based on computer vision, to allow it to navigate in shallow-water environments. The visual servoing system described here allows the robot to track and follow a given target underwater. The servo package is made up of two distinct parts: a tracker and a feedback controller. The system has been evaluated in the sea water and under natural lighting conditions. The servo system has been tested underwater, and with minor modifications, the system can be used while the robot is walking on the ground as well.

Tremendous progress has been made recently in simultaneous localization and mapping of unknown environments. Using sensor and odometry data from an exploring mobile robot, it has become much easier to build high-quality globally consistent maps of many large, real-world environments. To date, however, relatively little attention has been paid to the controllers used to build these maps. Existing exploration strategies usually attempt to cover the largest amount of unknown space as quickly as possible. Few strategies exist for building the most reliable map possible, but the particular control strategy can have a substantial impact on the quality of the resulting map. In this paper, we devise a control algorithm for exploring unknown space that explicitly tries to build as large a map as possible while maintaining as accurate a map as possible. We make use of a parameterized class of spiral trajectory policies, choosing a new parameter setting at every time step to maximize the expected reward of the policy. We do this in the context of building a visual map of an unknown environment, and show that our strategy leads to a higher accuracy map faster than other candidate controllers, including any single choice in our policy class.

This paper describes an underwater walking robotic system being developed under the name AQUA, the goals of the AQUA project, the overall hardware and software design, the basic hardware and sensor packages that have been developed, and some initial experiments. The robot is based on the RHex hexapod robot and uses a suite of sensing technologies, primarily based on computer vision and INS, to allow it to navigate and map clear shallow-water environments. The sensor-based navigation and mapping algorithms are based on the use of both artificial floating visual and acoustic landmarks as well as on naturally occurring underwater landmarks and trinocular stereo.

We present a method for learning a set of models of visual features which are invariant to scale and translation in the image domain. The models are constructed by first applying the scale-invariant feature transform (SIFT) to a set of training images, and matching the extracted features across the images, followed by learning the pose-dependent behavior of the features. The modeling process avoids assumptions with respect to scene and imaging geometry, but rather learns the direct mapping from camera pose to feature observation. Such models are useful for applications to robotic tasks, such as localization, as well as visualization tasks. We present the model learning framework, and experimental results illustrating the success of the method for learning models that are useful for robot localization.

We address the problem of computing dense range maps of indoor locations using only intensity images and partial depth. We allow a mobile robot to navigate the environment, take some pictures and few range data. Our method is based on interpolating the existing range data using statistical inferences learned from the available intensity image and from those (sparse) regions where both range and intensity information is present. The spatial relationships between the variations in intensity and range can be efficiently captured by the neighborhood system of a Markov random field (MRF). In contrast to classical approaches to depth recovery (i.e. stereo, shape from shading), we can afford to make only weak assumptions regarding specific surface geometries or surface reflectance functions since we compute the relationship between existing range data and the images we started with. Experimental results show the feasibility of our method.

Abstract We present a technique for creating a smoothly varying sequence of procedural textures that interpolates between arbitrary input samples of texture. This texture transformation uses a library of procedural shaders and selects the correct shaders and associated parameters to accomplish the task. In general, selecting a procedural texture from a library, or finding the correct parameters to produce a smooth texture transition can be complex and time consuming. We propose a strategy for automating this process. While superficially this problem appears intractable for both humans and computational systems, its natural characteristics make a computational solution feasible. We present an algorithm and experimental results demonstrating this approach. Transformation between two textures can then be achieved procedurally, while enforcing perceptual similarity constraints between adjacent texture frames. We describe a technique for efficiently sampling the parameter domain of a shader based on a texture similarity function to create a smooth path through its texture range. In the case of evolving between several shaders, a method is described to obtain the best jump‐points which can be used to connect different shaders smoothly in texture space. Several examples of the technique are shown, and future directions as well as potential problems are discussed. Categories and Subject Descriptors (according to ACM CCS): I.3.7 [Computer Graphics]: Texture

In this paper we present a novel method for robot path planning based on learning motion patterns. A motion pattern is defined as the path that results from applying a set of probabilistic constraints to a "raw" input path. For example, a user can sketch an approximate path for a robot without considered issues such as bounded radius of curvature and our system would then elaborate it to include such a constraint. In our approach, the constraints that generate a path are learned by capturing the statistical properties of a set of training examples using supervised learning. Each training example consists of a pair of paths: an unconstrained (raw) path and an associated preferred path. Using a Hidden Markov Model in combination with multi-scale methods, we compute a probability distribution for successive path segments as a function of their context within the path and the raw path that guides them. This learned distribution is then used to synthesize a preferred path from an arbitrary input path by choosing some mixture of the training set biases that produce the maximum likelihood estimate. We present our method and applications for robot control and non-holonomic path planning.

We discuss a software environment for multi-robot, multi-platform mobile robot control and simulation. Like others, we have observed that mobile robotics research is greatly facilitated by the availability of a suitable simulator for both vehicle kinematics as well as sensing, and have created an environment that permits this while allowing a large measure of device independence. By using a multiprocessor internet-based architecture, our platform permits multiple users to use a variety of programming interfaces (visual, script-based or various application programming interfaces (API's)) to rapidly prototype methods to control multiple heterogeneous robots both in simulation and in real-world settings. We present an overview of our architecture and discuss its future directions.

In this paper we present a probabilistic framework for the reduction in the uncertainty of a moving robot pose during exploration by using a second robot to assist. A Monte Carlo Simulation technique (specifically, a Particle Filter) is employed in order to model and reduce the accumulated odometric error. Furthermore, we study the requirements to obtain an accurate yet timely pose estimate. A team of two robots is employed to explore an indoor environment in this paper, although several aspects of the approach have been extended to larger groups. The concept behind our exploration strategy has been presented previously and is based on having one robot carry a sensor that acts as a "robot tracker" to estimate the position of the other robot. By suitable use of the tracker as an appropriate motion-control mechanism we can sweep areas of free space between the stationary and the moving robot and generate an accurate graph-based description of the environment. This graph is used to guide the exploration process. Complete exploration without any overlaps is guaranteed as a result of the guidance provided by the dual graph of the spatial decomposition (triangulation) of the environment. We present experimental results from indoor experiments in our laboratory and from more complex simulated experiments.

We present a system to beautify curves: i.e. to take curves that roughly depict some property of interest and make them look more like what experts would draw. The focus of our work is in applications for artistic drawings, but our system can also be applied to various other domains where curves and trajectories play a dominant role such as robot path planning, animation or edge-deblurring. Our approach consists of learning properties from a database of ideal example, which could be comic sketches or robot trajectories, and transform a coarse input curve to make it look like those in the database. The key scientific issue is: in what sense are these curves 'like' one another? In our work, this likeness is expressed statistically. Using hidden Markov models in combination with multi-scale methods and mixture models, we synthesize a new curve as a statistically consistent mixture of the training set that best describes the input. Additionally, our approach also allows us to easily include predefined application specific models that can further bias the system.

This paper presents a first detailed case study of collaborative exploration of a substantial environment. We use a pair of cooperating robots to test multi-robot environment mapping algorithms based on triangulation of free space. The robots observe one another using a robot tracking sensor based on laser range sensing (LIDAR). The environment mapping itself is accomplished using sonar sensing. The results of this mapping are compared to those obtained using scanning laser range sensing and the scan matching algorithm. We show that with appropriate outlier rejection policies, the sonar-based map obtained using collaborative localization can be as good or, in fact, better than that obtained using what is typically considered to be a superior sensing technology.

We consider the effect of exploration policy in the context of the autonomous construction of a visual map of an unknown environment. Like other concurrent mapping and localization (CML) tasks, odometric uncertainty poses the problem of introducing distortions into the map which are difficult to correct without costly on-line or post-processing algorithms. Our problem is further compounded by the implicit nature of the visual map representation, which is designed to accommodate a wide variety of visual phenomena without assuming a particular imaging platform, thereby precluding the inference of scene geometry. Such a representation presents a requirement for a relatively dense sampling of observations of the environment in order to produce reliable models. Our goal is to develop an online policy for exploring an unknown environment which minimizes map distortion while maximizing coverage. We do not depend on costly post-hoc expectation maximization approaches to improve the output, but rather employ extended Kalman filter (EKF) methods to localize each observation once, and rely on the exploration policy to ensure that sufficient information is available to localize the successive observations. We present an experimental analysis of a variety of exploratory policies, in both simulated and real environments, and demonstrate that with an effective policy an accurate map can be constructed.

This paper examines a novel method we have developed for computing range data in the context of mobile robotics. Our objective is to compute dense range maps of locations in the environment, but to do this using intensity images and very limited range data as input. We develop a statistical learning method for inferring and extrapolating range data from a combination of a single video intensity image and a limited amount of input range data. Our methodology is to compute the relationship between the observed range data and the variations in the intensity image, and use this to extrapolate new range values. These variations can be efficiently captured by the neighborhood system of a Markov random field (MRF) without making any strong assumptions about the kind of surfaces in the world. Experimental results show the feasibility of our method.

This paper examines the tradeoffs between different classes of sensing strategy and motion control strategy in the context of terrain mapping with multiple robots. We consider a larger group of robots that can mutually estimate one another's position (in 2D or 3D) and uncertainty using a sample-based (particle filter) model of uncertainty. Our prior work has dealt with a pair of robots that estimate one another's position using visual tracking and coordinated motion. Here we extend these results and consider a richer set of sensing and motion options. In particular, we focus on issues related to confidence estimation for groups of more than two robots.

In this paper a range synthesis algorithm is proposed as an initial solution to the problem of 3D environment modeling from sparse data. We develop a statistical learning method for inferring and extrapolating range data from as little as one intensity image and from those (sparse) regions where both range and intensity information is available. Our work is related to methods for texture synthesis using Markov Random Field methods. We demonstrate that MRF methods can also be applied to general intensity images with little associated range information and used to estimate range values where needed without making any strong assumptions about the kind of surfaces in the world Experimental results show the feasibility of our method.

We present a method for learning a set of generative models which are suitable for representing variations of selected image-domain features of the scene as a function of changes in the camera viewpoint. Such models are important for robotic tasks, such as probabilistic position estimation (i.e. localization), as well as visualization. Our approach entails the selection of image-domain features, as well as the synthesis of models of their visual behavior. The model we propose is capable of generating maximum likelihood views of automatically selected features, as well as a measure of the likelihood of a particular view from a particular camera position. Training the models involves regularizing observations of the features from known camera locations. The uncertainty of the model is evaluated using cross validation. The features themselves are initially selected automatically as salient points by a measure of visual attention, and are tracked across multiple views. While the motivation for this work is for robot localization, the results have implications for image interpolation, virtual scene reconstruction and object recognition. The paper presents a formulation of the problem and illustrative experimental results.

We examine the problem of learning a visual map of the environment while maintaining an accurate pose estimate. Our approach is based on using two robots in a simple collaborative scheme. Without outside information, as a robot collects training images, its position estimate accumulates errors, thus corrupting its knowledge of the positions from which observations are taken. We address this problem by deploying a second robot to observe the first one as it explores, thereby establishing a virtual tether, and enabling an accurate estimate of the robot's position while it constructs the map. We refer to this process as cooperative localization. The images collected during this process are assembled into a representation that allows vision-based position estimation from a single image at a later date. In addition to developing a formalism and concept, we validate our results experimentally and present quantitative results demonstrating the performance of the method in over 90 trials.

We present an approach to appearance-based object recognition using single camera images. Our approach is based on using an attention mechanism to obtain visual features that are generic, robust and informative. The features themselves are recognized using principal components an the frequency domain. In this paper we show how the visual characteristics of only a small number of such features can be used for appearance-based object recognition that is not confounded by planar rotations or background clutter.

This paper describes an approach to the automated creation of virtual realities (or virtual maps) of an a priori unknown environment by using a mobile robot. The method we propose is aimed at the creation of an image-based or iconic map, rather than a representation in terms of 2D or 3D spatial occupancy. A key aspect of this is having a mobile robot automatically select points and views of interest that can be used to exemplify the appearance of the environment. This paper develops the use of alpha-backtracking as a technique to efficiency select these points of estimated globally maximum interest.

We present an approach to the automatic recognition of locations or landmarks using single camera images. Our approach is to learn visual features in the appearance domain that can be used to characterize an object or a location. These features are defined statistically and then are recognized using principal components in the frequency domain. We show that this technique can be used to recognize specific objects on varying backgrounds, as well as environmental features.

This paper presents a new sensing modality and stratagem for multirobot exploration. The approach is based on using pairs of robots that observe each other's behavior, acting in concert to reduce odometry errors. We assume the robots can both directly sense nearby obstacles and see each other. This allows the robots to obtain a map of higher accuracy than would be possible with robots acting independently by reducing inaccuracies that occur over time from dead reckoning errors. Furthermore, by exploiting the ability of the robots to see each other, we can detect opaque obstacles in the environment independently of their surface reflectance properties. Two different algorithms, based on the size of the environment, are introduced with a complexity analysis, and experimental results in simulation and with real robots.

Multiple mobile robots, or robot collectives, have been proposed as solutions to various tasks in which distributed sensing and action are required. Here we consider applying a collective of robots to the paparazzi problem - the problem of providing sensor coverage of a target robot. We demonstrate how the computational task of the collective can be formulated as a global energy minimization task over the entire collective, and show how individual members of the collective can solve the task in a distributed fashion so that the entire collective meets its goal. This result is then extended to consider unbounded communication delays between members and complete failure of individual members of the collective.

We present a method for learning a set of visual landmarks which are useful for pose estimation. The landmark learning mechanism is designed to be applicable to a wide range of environments, and generalized for different approaches to computing a pose estimate. Initially, each landmark is detected as a focal extremum of a measure of distinctiveness and represented by a principal components encoding which is exploited for matching. Attributes of the observed landmarks can be parameterized using a generic parameterization method and then evaluated in terms of their utility for pose estimation. We present experimental evidence that demonstrates the utility of the method.

We consider the robot exploration of a planar graph-like world. The robot's goal is to build a complete map of its environment. The environment is modeled as an arbitrary undirected planar graph which is initially unknown to the robot. The robot cannot distinguish vertices and edges that it has explored from the unexplored ones. The robot is assumed to be able to autonomously traverse graph edges, recognize when it has reached a vertex, and enumerate edges incident upon the current vertex. The robot cannot measure distances nor does it have a compass, but it is equipped with a single marker that it can leave at a vertex and sense if the marker is present at a newly visited vertex. The total number of edges traversed while constructing a map of a graph is used as a measure of performance. We present an efficient algorithm for learning an unknown, undirected planar graph by a robot equipped with one marker. Experimental results obtained by running a large collection of example worlds are presented.

We present an approach to vision-based mobile robot localization, even without an a-priori pose estimate. This is accomplished by learning a set of visual features called image-domain landmarks. The landmark learning mechanism is designed to be applicable to a wide range of environments. Each landmark is detected as a focal extremum of a measure of uniqueness and represented by an appearance-based encoding. Localization is performed using a method that matches observed landmarks to learned prototypes and generates independent position estimates for each match. The independent estimates are then combined to obtain a final position estimate, with an associated uncertainty. Quantitative experimental evidence is presented that demonstrates that accurate pose estimates can be obtained, despite changes to the environment.

This paper presents an approach to the registration of individual images to one another to produce a larger composite mosaic. The approach is based on the use of the moments of Zernike orthogonal polynomials to compute the relative scale, rotation and translation between the images. A preliminary stage involves the use of an attention-like operation to estimate potential approximate correspondence points between the images based on extrema of local edge element density. Experimental results illustrate that the technique is effective in a range of environments and over a broad range of image registration parameters. In particular, our method makes few assumptions regarding the image content and yet, unlike several alternative approaches, can perform registration for images with only a limited amount of overlap.

In this paper we discuss new results on the Shape From Darkness problem: using the motion of cast shadows to recover scene structure. Our approach is based on collecting a set of images from a fixed viewpoint as a known light source mover; "across the sky". Previously published solutions to this problem have performed the reconstruction only for cross sections of the scene. In this paper, we present a reconstruction algorithm and discuss the reconstruction of an entire 3-D scene under various light source trajectories. We also consider the constraints on reconstruction. We conclude with experimental results that illustrate the convergence properties of the solution process and its robustness properties.

This paper describes the fully automatic creation of an environment's description using an image-based representation. This representation is a collection of cylindrical sample images combined into an "image-based virtual reality". The locations at which the environment will be sampled are chosen automatically using an operator inspired by models of human visual attention and saccadic motion. The image acquisition is performed by a mobile robot. The selection of vantage points is based on an analysis of the edge structure of sampled panoramic images. In order to trade off the optimality of the generated description of the navigation effort required in solving the online problem, a concept referred to as alpha-backtracking is introduced. The paper illustrates sample data acquired by the procedure.

Addresses the problem of seeking out parts of the environment that provide adequate features in order to perform robot localization. The objective is to choose good regions in which local metric maps can be established. A distinctiveness measure is defined as a measure of how well the environment allows the robot to accomplish a task, in our case the task being localization. The distinctiveness measure is evaluated as a function of both the localization strategy and the environment. Areas in the environment are considered to have high distinctiveness measures if they exhibit both sufficient spatial structure and good sensor feedback. The problem is treated as defining an evaluation criterion based on the usefulness of gathered information.

Describes an integrated system for the automatic construction of image-based virtual realities to describe a real environment. A mobile robot autonomously navigates through the environment and uses a camera to make observations. At locations that are deemed sufficiently interesting, panoramic images are collected that are used to construct a multi-node VR movie. Images of the environment are classified in terms of two features related to human attention: edge element density and edge orientation. The system deems locations interesting if they are sufficiently different from the surrounding environment. The parameterization of the surrounding environment is computed either in a pre-computation pass, or online using a technique termed alpha-backtracking. The panoramic images that describe the environment are automatically joined together in a navigable movie that simulates motion in the real environment.

Presents an approach to vision-based mobile robot localization. In an attempt to capitalize on the benefits of both image and landmark-based methods, we describe a method that combines their strengths. Images are encoded as a set of visual features called landmarks. Potential landmarks are detected using an attention mechanism implemented as a measure of uniqueness. They are then selected and represented by an appearance-based encoding. Localization is performed using a landmark tracking and interpolation method which obtains an estimate accurate to a fraction of the environment sampling density. Experimental results are shown to confirm the feasibility and accuracy of the method.

We describe a method of mapping large scale static environments using a hybrid topological-metric model. A global map is formed from a set of local maps organized in a topological structure. Each local map contains quantitative environment information using a local reference frame. They are denoted as islands of reliability because they provide accurate metric information of the environment. The mapping problem then becomes where to place the islands of reliability and to what extent should they cover the environment. This is accomplished by defining the placement criteria in terms of the task the islands of reliability portray.

Many methods are available for image mosaicing most of which are not useful because they either (1) require a great deal of overlap between images, or (2) if they work for restricted sub-problem (translation, rotation or zooming) they would not work for the others. While numerous methods exist for accurately calculating translational shifts, no method was found that could handle rotations of angles greater than 15 degrees, or for scaling. In this paper, a new method using Zernike moments is presented to solve for the translational registration, rotational registration (2D and 3D) and zooming all at the same time. This method was tested on different sets of images with translational shifts, rotational shifts and zooming. The method is very fast, efficient and does not require any human interaction, and the results proved to be very accurate.

We consider the problem of localizing a robot in a known environment modeled by a simple polygon P. We assume that the robot has a map of P but is placed at an unknown location inside P. From its initial location, the robot sees a set of points called the visibility polygon V of its location. In general, sensing at a single point will not suffice to uniquely localize the robot, since the set H of points in P with visibility polygon V may have more than one element. Hence, the robot must move around and use range sensing and a compass to determine its position (i.e., localize itself). We seek a strategy that minimizes the distance the robot travels to determine its exact location. We show that the problem of localizing a robot with minimum travel is NP-hard. We then give a polynomial time approximation scheme that causes the robot to travel a distance of at most (k - 1)d, where k = |H|, which is no greater than the number of reflex vertices of P, and d is the length of a minimum length tour that would allow the robot to verify its true initial location by sensing. We also show that this bound is the best possible.

We are interested in inferring the sources of various types of sonar features typically observed by a mobile robot. After a brief discussion of terrestrial sonar sensing, we develop a set of operators that associates arc-shaped features extracted from sonar scans with real world primitives. Our classification scheme is probabilistic and is based on empirical data: the confidence of the association hypotheses produced by the operators is evaluated statistically. Some of our experimental results suggest that methods based on models of perfect sonar sensors may not be completely consistent with observed data. The management and merging of a collection of hypotheses concerning various sonar features allows the system to produce a coherent and mutually-compatible set of inferences for the entire observed environment.

We show how a neural network can be used to allow a mobile robot to derive an accurate estimate of its location from noisy sonar sensors and noisy motion information. The robot's model of its location is in the form of a probability distribution across a grid of possible locations. This distribution is updated using both the motion information and the predictions of a neural network that maps locations into likelihood distributions across possible sonar readings. By predicting sonar readings from locations, rather than vice versa, the robot can handle the very nongaussian noise in the sonar sensors. By using the constraint provided by the noisy motion information, the robot can use previous readings to improve its estimate of its current location. By treating the resulting estimates as if they were correct, the robot can learn the relationship between location and sonar readings without requiring an external supervision signal that specifies the actual location of the robot. It can learn to locate itself in a new environment with almost no supervision, and it can maintain its location ability even when the environment is nonstationary.

This paper deals with generic 3D shape modelling for the purposes of object recognition. Difficulties with many existing methods are that they either capture insufficient detailed structure or fail to provide sufficiently abstract descriptions. The approach presented here attempts to address this problem by building a composite representation of the data in terms of a superquadric augmented with multi-scale surface models. This is illustrated experimentally using laser range data. The superquadric that results in the best possible fit is expressed in terms of its position, size, shape and pose parameters. The residual of the fit is then modelled at several scales using multiple surface patches with uniform mean and Gaussian curvature. A hierarchical ranking of these patches is used to describe the residual based on geometric properties. These geometric properties are ranked according to criteria expressing their stability and utility. The most stable patches are selected as the description of the residual. The resulting representation can then be used for both pose estimation and object recognition.

This paper describes an approach to combining range data from both a set of sonar sensors as well as from a directional laser range finder to efficiently take advantage of the characteristics of both types of devices when exploring and mapping unknown worlds. The authors call their approach "just in time sensing" because it uses the more accurate but constrained laser range sensor only as needed, based upon a preliminary interpretation of sonar data. In this respect, it resembles "just in time" inventory control which attempts to judiciously obtain materials for industrial manufacturing only when and as needed. Experiments with a mobile robot equipped with sonar and a laser rangefinder demonstrate that by judiciously using the more accurate but more complex laser rangefinder to deal with the well-known ambiguity which arises in sonar data, the authors are able to obtain a much better map of an interior space at little additional cost (in terms of time and computational expense).

We consider the problem of locating a robot in an initially-unfamiliar environment from visual input. The robot is not given a map of the environment, but it does have access to a collection of training examples, each of which specifies the video image observed when the robot is at a particular location and orientation. We address two variants of this problem: how to estimate translation of a moving robot assuming the orientation is known, and how to estimate translation and orientation for a mobile robot. Performing scene reconstruction to construct a metric map of the environment using only video images is difficult. We avoid this by using an approach in which the robot learns to convert a set of image measurements into a representation of its pose (position and orientation). This provides a metric estimate of the robot's location within a region covered by the statistical map we build. Localization can be performed online without a prior location estimate, The conversion from visual data to camera pose is implemented using a multilayer neural network that is trained using backpropagation. An aspect of the approach is the use of an inconsistency measure to eliminate incorrect data and estimate components of the pose vector. The experimental data reported in this paper suggests that the accuracy and flexibility of the technique is good, while the online computational cost is very low.

Mobile robot navigation and localization is frequently aided by, or even dependent upon, a good estimate of the rate of dead-reckoning error accumulation. Sensor data can be used for position estimation, but this often involves overheads in acquiring and processing the data. By sensing and then classifying the surface type, an estimate of the rate of error accumulation for dead-reckoning allows one to estimate accurately how often localization, including sensor data acquisition, must be performed. The authors describe experiments in which a boom-mounted microphone is tapped on different floor materials, much as a blind man might tap his cane. The acoustic signature arising from the contact is then used to classify the floor type by comparing a windowed power spectrum of the acoustic signature with one of a family of prototypical signatures generated statistically from the same material. The technique is low-cost, involves limited computational expense, and performs very well.

This paper describes an approach to building a hierarchy of maplike descriptions of an initially unknown environment using a mobile robot. High-level tasks require a symbolic and qualitative description of space, yet this remains a stable and consistent method for defining specific regions of interest. The paper describes a hierarchy of spatial abstractions and a classification scheme for spatial regions to be used in navigation and mapping. In particular, the problem of maintaining a stable description of the environment that can be used for symbol extraction receives attention. By combining signal-domain analysis for initial processing of sonar and video data in conjunction with geometric processing for symbolic map data constructed from the sonar data, robust navigation becomes possible. Geometric map construction is based on range data, with sonar data used as an example. Recalibration of the robot's position, however, is critical to this task. Two different and complementary forms, of localization algorithm are described. One of these is based on simple geometric modeling of range data points, followed by data-to-object matching. The second method depends less strongly on a simple environment geometry and is based on image data.

In the domain of mobile robotic task execution under dialogue control, a primary goal is to identify the task target which is specified by a natural language description. A number of concepts are expressed in the user spoken language by vague terms like "the big box" and "very close to the door". We use fuzzy logic to map these vague terms onto the quantitative data collected by system sensors. Fuzziness may cause uncertainty in interpretation and, in particular, in understanding references. This uncertainty is abated by collecting additional information through queries to the user and autonomous sensing. Entropy is used to select the queries having the greatest discriminatory power among referent candidates. In addition, we examine the trade-off between querying, sensing and uncertainty. A framework to deal with each of these issues has been developed and is presented.

This paper deals with coordinating behaviour in a multi-autonomous robot system. When two or more autonomous robots must interact in order to accomplish some common goal, communication between the robots is essential. Different inter-robot communications strategies give rise to different overall system performance and reliability. After a brief consideration of some theoretical approaches to multiple robot collections, we present concrete implementations of different strategies for convoy-like behaviour. The convoy system is based around two RWI B12 mobile robots and uses only passive visual sensing for inter-robot communication. The issues related to different communication strategies are considered.

Addresses the problem of estimating 3D space occupancy using video imagery in the context of mobile robotics. A stationary robot observes a cluttered scene from a single viewpoint, and a second robot illuminates the scene from a sequence of directions thus producing a sequence of grey-level images. Differences of successive images are used to compute a sequence of shadowimages. The problem is to compute free space and occupied space from these shadowimages. Solutions to this problem are known for the special case of terrain scenes. The authors generalize these solutions to non-terrain scenes by making two key observations. First, there is a subset constraint on the shadowimages of a non-terrain scene, which allows the visible surfaces of a non-terrain scene to be recovered by a terrain-based technique. Second, the remaining regions of the shadowimages provide a conservative estimate of the occupied space hidden by these visible surfaces.

When a particular 3D point is fixated by a robotic stereo system different portions of the world are brought into interocular alignment. This region is known as the horoptor. Purposeful modifications to the binocular geometry can be used to bring different regions of three-space closer to the horoptor: camera pan and tilt define the rough structure of the horoptor, while camera torsion can be used to change its local shape. Theoretical and empirical results suggest that for binocular vision tasks: 1) it is important to understand the region of three space that contains the horoptor curve; and 2) it is possible to control this shape in an active way so as to simplify certain binocular tasks.

This paper addresses the coupled tasks of constructing a spatial representation of the environment with a mobile robot using noisy sensors (sonar) and using such a map to determine the robot's position. The map is not meant to represent the actual spatial structure of the environment so much as it is meant to represent the major structural components of what the robot "sees". This can, in turn, be used to construct a model of the physical objects in the environment. One problem with such an approach is that maintaining an absolute coordinate system for the map is difficult without periodically calibrating the robot's position. The authors demonstrate that in a suitable environment it is possible to use sonar data to correct position and orientation estimates on an ongoing basis. This is accomplished by incrementally constructing and updating a model-based description of the acquired data. Given coarse position estimates of the robot's location and orientation, these can be refined to high accuracy using the stored map and a set of sonar readings from a single position. This approach is then generalized to allow global position estimation, where position and orientation estimates may not be available. The authors consider the accuracy of the method based on a single sonar reading and illustrate its region of convergence using empirical data.>

In many cases several mobile robots (autonomous agents) can be used together to accomplish tasks that would be either more difficult or impossible for a robot acting alone. Many different models have been suggested for the makeup of such collections of robots. In this paper the authors present a taxonomy of the different ways in which such a collection of autonomous robotic agents can be structured. It is shown that certain swarms provide little or no advantage over having a single robot, while other swarms can obtain better than linear speedup over a single robot. There exist both trivial and non-trivial problems for which a swarm of robots can succeed where a single robot will fail. Swarms are more than just networks of independent processors - they are potentially reconfigurable networks of communicating agents capable of coordinated sensing and interaction with the environment.

An approach for describing objects for the purpose of recognition is developed. The authors deal with two key issues: building natural descriptions of curved objects, and making these descriptions compact and abstract. Typical examples of the types of curve one is able to describe as both qualitatively similar, yet discriminably different, are shown. Methods based on curvature extrema alone are likely to find three of the four of these shapes indistinguishable, while methods based on approaches such as shape templates may be oblivious to their similarity. The central ideas of the approach are outlined.>

Addressed is the problem of robotic exploration of a graphlike world, where no distance or orientation metric is assumed of the world. The robot is assumed to be able to autonomously traverse graph edges, recognize when it has reached a vertex, and enumerate edges incident upon the current vertex relative to the edge via which it entered the current vertex. The robot cannot measure distances, and it does not have a compass. It is demonstrated that this exploration problem is unsolvable in general without markers, and, to solve it, the robot is equipped with one or more distinct markers that can be put down or picked up at will and that can be recognized by the robot if they are at the same vertex as the robot. An exploration algorithm is developed and proven correct. Its performance is shown on several example worlds, and heuristics for improving its performance are discussed.>

The authors present a technique for both the smoothing and decomposition of planar curves. This technique, called curvature-tuned smoothing, provides for robust rotation- and translation-invariant smoothing of planar curves. As the smoothing is performed, parts are extracted robustly and at multiple scales. These parts can subsequently be used for object recognition. The parts extracted correspond to regions of roughly uniform curvature and constitute a rich description of the original data. This representation describes some regions at multiple scales, since multiple structures may occur.>