Robot navigation methods allow mobile robots to operate in applications such as warehouses or hospitals. While the environment in which the robot operates imposes requirements on its navigation behavior, most existing methods do not allow the end-user to configure the robot's behavior and priorities, possibly leading to undesirable behavior (e.g., fast driving in a hospital). We propose a novel approach to adapt robot motion behavior based on natural language instructions provided by the end-user. Our zero-shot method uses an existing Visual Language Model to interpret a user text query or an image of the environment. This information is used to generate the cost function and reconfigure the parameters of a Model Predictive Controller, translating the user's instruction to the robot's motion behavior. This allows our method to safely and effectively navigate in dynamic and challenging environments. We extensively evaluate our method's individual components and demonstrate the effectiveness of our method on a ground robot in simulation and real-world experiments, and across a variety of environments and user specifications.

We present AdaptiVe Optimal Collision Avoidance Driven by Opinion (AVOCADO), a novel navigation approach to address holonomic robot collision avoidance when the robot does not know how cooperative the other agents in the environment are. AVOCADO departs from a velocity obstacle's (VO) formulation akin to the optimal reciprocal collision avoidance method. However, instead of assuming reciprocity, it poses an adaptive control problem to adapt to the cooperation level of other robots and agents in real time. This is achieved through a novel nonlinear opinion dynamics design that relies solely on sensor observations. As a by-product, we leverage tools from the opinion dynamics formulation to naturally avoid the deadlocks in geometrically symmetric scenarios that typically suffer VO-based planners. Extensive numerical simulations show that AVOCADO surpasses existing motion planners in mixed cooperative/noncooperative navigation environments in terms of success rate, time to goal and computational time. In addition, we conduct multiple real experiments that verify that AVOCADO is able to avoid collisions in environments crowded with other robots and humans.

Autonomous navigation in crowded environments is an open problem with many applications, essential for the coexistence of robots and humans in the smart cities of the future. In recent years, deep reinforcement learning approaches have proven to outperform model-based algorithms. Nevertheless, even though the results provided are promising, the works are not able to take advantage of the capabilities that their models offer. They usually get trapped in local optima in the training process, that prevent them from learning the optimal policy. They are not able to visit and interact with every possible state appropriately, such as with the states near the goal or near the dynamic obstacles. In this work, we propose using intrinsic rewards to balance between exploration and exploitation and explore depending on the uncertainty of the states instead of on the time the agent has been trained, encouraging the agent to get more curious about unknown states. We explain the benefits of the approach and compare it with other exploration algorithms that may be used for crowd navigation. Many simulation experiments are performed modifying several algorithms of the state-of-the-art, showing that the use of intrinsic rewards makes the robot learn faster and reach higher rewards and success rates (fewer collisions) in shorter navigation times, outperforming the state-of-the-art.

The work describes the research advances by the Robotics, Perception and Real-Time Group of University of Zaragoza (Spain) in the field of ground robotics in confined environments. This type of environment presents several special characteristics that often make well-established techniques fail. The aim is to present the main challenges, the developed solutions, and the lessons learned.

Leveraging Manhattan assumption we generate metrically rectified novel views from a single image, even for non-box scenarios. Our novel views enable the already trained classifiers to handle training data missing views (blind spots) without additional training. We demonstrate this on end-to-end scene text spotting under perspective. Additionally, utilizing our fronto-parallel views, we discover unsuspended invariant mid-level patches given a few widely separated training examples (small data domain). These invariant patches outperform various baselines on small data image retrieval challenge.

Nowadays real time visual Simultaneous Localization And Mapping (SLAM) algorithms exist and rely on consistent measurements across multiple views. In indoor environments, where majority of robot's activity takes place, severe occlusions can occur, e.g., when turning around a corner or moving from one room to another. In these situations, SLAM algorithms can not establish correspondences across views, which leads to failures in camera localization or map construction. This work takes advantage of the recent scene box layout descriptor to make the above mentioned SLAM systems occlusion aware. This room box reasoning helps the sequential tracker to reason about possible occlusions and therefore look for matches in only potentially visible features instead of the entire map. This increases the life of the tracker, as it does not consider itself lost under the occlusion state. Additionally, focusing on the potentially visible portion of the map, i.e., the current room features, it improves the computational efficiency without compromising the accuracy. Finally, this room level reasoning helps in better image selection for bundle adjustment. The image bundle coming from the same room has little occlusion, which leads to better dense reconstruction. We demonstrate the superior performance of layout aware SLAM on several long monocular sequences acquired in difficult indoor situations, specifically in a room-room transition and turning around a corner.

The vision of the RoboEarth project is to design a knowledge-based system to provide web and cloud services that can transform a simple robot into an intelligent one. In this work, we describe the RoboEarth semantic mapping system. The semantic map is composed of: 1) an ontology to code the concepts and relations in maps and objects and 2) a SLAM map providing the scene geometry and the object locations with respect to the robot. We propose to ground the terminological knowledge in the robot perceptions by means of the SLAM map of objects. RoboEarth boosts mapping by providing: 1) a subdatabase of object models relevant for the task at hand, obtained by semantic reasoning, which improves recognition by reducing computation and the false positive rate; 2) the sharing of semantic maps between robots; and 3) software as a service to externalize in the cloud the more intensive mapping computations, while meeting the mandatory hard real time constraints of the robot. To demonstrate the RoboEarth cloud mapping system, we investigate two action recipes that embody semantic map building in a simple mobile robot. The first recipe enables semantic map building for a novel environment while exploiting available prior information about the environment. The second recipe searches for a novel object, with the efficiency boosted thanks to the reasoning on a semantically annotated map. Our experimental results demonstrate that, by using RoboEarth cloud services, a simple robot can reliably and efficiently build the semantic maps needed to perform its quotidian tasks. In addition, we show the synergetic relation of the SLAM map of objects that grounds the terminological knowledge coded in the ontology.

Extracting the 3D geometry plays an important part in scene understanding. Recently, robust visual descriptors are proposed for extracting the indoor scene layout from a passive agent’s perspective, specifically from a single image. Their robustness is mainly due to modelling the physical interaction of the underlying room geometry with the objects and the humans present in the room. In this work we add the physical constraints coming from acoustic echoes, generated by an audio source, to this visual model. Our audio-visual 3D geometry descriptor improves over the state of the art in passive perception models as we show in our experiments.

In this paper we address robot-human interactions in a multi-robot and people group framework. The objective is to develop and evaluate techniques for missions in which several robots cooperate among themselves, interacting with a group of people. The robots detect people behaviors and act consequently adopting different strategies. Probabilistic techniques for robust cooperative detection of group and individual behavior are developed from the range finder information. Environment perception and cooperative motion planning techniques adaptable to the situation observed are developed and jointly used with the reactions to the people's behavior. In order to evaluate the system, we propose a scenario where the robots guide a group of people to a goal position. This scenario has been tested in simulations and in real experiments, to validate the techniques developed.

In previous work we presented a multi-robot strategy for routing missions in large scenarios where network connectivity must be explicitly preserved. This strategy is founded on the traversal of path trees in such a way that connectivity to a static control center is always maintained, while ensuring that any target that is reachable by a chain consisting of all robots is eventually visited. In this work we improve the strategy performance by extending its sequential one-task-at-a-time execution approach with concurrent execution of tasks. We demonstrate that the general problem is NP-hard and offer several heuristic approaches to tackle it. We study the improvements that these heuristics can offer in regard to several important variables like network range and clustering of targets, and finally compare their performance over the optimal solutions for small problem instances. In summary, we offer a complete characterization of the new concurrent capabilities of the CONNECTTREE strategy.

This paper addresses a flexible cooperative navigation system for small robot teams. The members of these groups move in formation, but with enough flexibility to be able to avoid obstacles and adapt to the environment shape. This approach is based on an analysis of the environment on a grid-based model that permits each robot to continuously evaluate the situation in which it is, in order to decide the best of the three developed navigation strategies, one for each situation type. Moreover, by using this model the robot cooperation for a coordinate navigation is improved, allowing each robot to share the observations with the other robots. So the field of view of each robot of the team on which it makes the decisions is enlarged. The system is implemented in a decentralized way so the number of robots is not a main issue, thus being scalable. Eventually, initial formation topology can be modified, in order to comply with the environment while the mission is accomplished. The system and the strategies are evaluated by means of simulations, showing the robustness against possible blocking situations.

Most collision avoidance methods do not consider the vehicle shape and its kinematic and dynamic constraints, assuming the robot to be point-like and omnidirectional with no acceleration constraints. The contribution of this paper is a methodology to consider the exact shape and kinematics, as well as the effects of dynamics in the collision avoidance layer, since the original avoidance method does not address them. This is achievable by abstracting the constraints from the avoidance methods in such a way that when the method is applied, the constraints already have been considered. This study is a starting point to extend the domain of applicability to a wide range of collision avoidance methods.

Teams of mobile robots have been recently proposed as effective means of completing complex missions involving multiple tasks spatially distributed over a large area. A central problem in such domains is multi-robot routing, namely the problem of coordinating a team of robots in terms of the locations they should visit and the routes they should follow in order to accomplish their common mission. A typical assumption made in prior work on multi-robot routing is that robots are able to communicate uninterruptedly at all times independently of their locations. In this paper, we investigate the multi-robot routing problem under communication constraints, reflecting on the fact that real mobile robots have a limited range of communication and the requirement that connectivity must remain intact (even through relaying) during the entire mission. We propose four algorithms for this problem, all based on the same reactive framework, ranging from greedy to deliberative approaches. All algorithms are tested in various scenarios implemented using the Player-Stage robot simulation environment. Our results demonstrate that effective multi-robot routing can be achieved even under limited communication range with moderate loss compared to the case of infinite communication range.

This paper reports an autonomous cooperative navigation system for robot formations in realistic scenarios. The formation movement control is based on a virtual structure composed by spring-dampers elements, which allows the formation to comply with the environment shape. A different navigation strategy is applied to the leader of the formation and to the rest of robots of the team. The leader plans the trajectories by using a two-level path planner with obstacle avoidance capabilities. The motion of the follower robots is controlled by the virtual structure, which adapts to the environment while the leader is tracked, taking into account the kinodynamic constraints of the vehicles. The system is evaluated in experiments carried out in simulations, some of them made in a realistic and complex urban scenario, and with real robots.

Auction based techniques are a highly successful tool used for multi-robot task allocation. However, theoretical performance and a proper taxonomy of optimization objectives have remained scarce until recent studies. Implementations from different authors have not been compared in common grounds and in light of these recent findings. In this paper we address this lack of comparative experimentation, providing simulation results on a large real life based scenario and in random worlds. Two intuitive optimization objectives, minimum total resource usage and minimum total time, are evaluated in object searching missions. A method for flexible tailoring of the bidding rules is presented and new insight is gained on the effect of using hybrid criteria for the optimization objective.

In this paper, we describe the adaptation of an autonomous robotic wheelchair for cognitive disabled children. The constraints imposed by these users require developing specific human-machine interfaces adapted to their limitations. In most cases it is necessary to develop additional tools to teach the children the spatial relations between the wheelchair, its motion and the environment. In addition to this, it is important to interact closely with the children and their educators. The paper describes the whole process followed to make the children use the autonomous wheelchair and the lessons learnt during the validation phase with the wheelchair and the children

This paper addresses a method to optimize the robot motion planning in dynamic environments, avoiding the moving and static obstacles while the robot drives towards the goal. The method maps the dynamic environment into a model in the velocity space, computing the times to potential collision and potential escape and the associated robot velocities. The problem of finding a trajectory to the goal is stated as a constrained nonlinear optimization problem. The initial seed trajectory for the optimization is directly generated in the velocity space using the model built. The method is applied to robots which are subject to both kinematic constraints (i.e. involving the configuration parameters of the robot and their derivatives), and dynamic constraints, (i.e. the constraints imposed by the acceleration/deceleration capabilities). Some experimental results are discussed

This paper proposes an approach allowing indoor environment supervised learning to recognize relevant features for environment understanding. Stochastic preprocessing methods in combination with either of usual pattern recognition schemes are used. Preprocessing method treated is a combination of the principal components analysis and the Fisher linear discriminant analysis well adapted to the sensorial information and to the kind of environments considered. The supervised method is applied to the raw range data obtained from typical indoor environments, obtaining good recognition performances without geometrical feature extraction, allowing its real time implementation. Our work focuses on the preprocessing method, giving a geometrical interpretation of their main components, and analyzing their robustness to shape distortions and scale changes

This paper addresses the modeling of the static and dynamic parts of the scenario and how to use this information within a real sensor-based navigation system. The contribution in the modeling aspect is a formulation of the Detection and Tracking of Mobile Objects and the Simultaneous Localization and Map Building in such a way that the nature (static/dynamic) of the observations is included in the estimation process. This is achieved by a set of filters tracking the moving objects and a map of the static structure constructed on line. In addition, this paper discusses how this modeling module is integrated in a real sensor-based navigation system taking advantage selectively of the dynamic and static information. The experimental results confirm that the complete navigation system is able to move a vehicle in unknown and dynamic scenarios. Furthermore, the system overcomes many of the limitations of previous systems associated to the ability to distinguish the nature of the parts of the scenario.

This paper presents a method to cooperatively localize pairs of robots fusing bearing-only information provided by cameras and the motion of the vehicles. The algorithm uses the robots as landmarks to estimate their relative location. Bearings are the simplest measurements directly obtained from the cameras, as opposed to measuring depths which would require knowledge or reconstruction of the world structure. We present the general recursive Bayes estimator and three different implementations based on an extended Kalman filter, a particle filter and a combination of both techniques. We have compared the performance of the different implementations using real data acquired with two platforms equipped with omnidirectional cameras and simulated data.

This paper presents a probabilistic scan matching algorithm to estimate the robot planar displacement by matching dense two-dimensional range scans. The general framework follows an iterative process of two steps: (i) computation of correspondences between scans, and (ii) estimation of the relative displacement. The contribution is a probabilistic modelling of this process that takes into account all the uncertainties involved: the uncertainty of the displacement of the sensor and the measurement noises. Furthermore, it also considers all the possible correspondences resulting from these uncertainties. This technique has been implemented and tested on a real vehicle. The experiments illustrate how the performances of this method are better than previous geometric ones in terms of robustness, accuracy and convergence.

This paper addresses a method for robot motion planning in dynamic environments, avoiding the moving and static obstacles while the robot drives towards the goal. The method maps the dynamic environment into a velocity space, using the concept of estimated arriving time to compute the times to potential collision and potential escape. The problem of finding the best motion command is directly treated in the velocity space, providing the trajectory which satisfies an optimization criterium (typically the minimum time or the shortest path). In this work the method is applied to robots which are subject to both kinematic constraints (i.e. involving the configuration parameters of the robot and their derivatives), and dynamic constraints, (i.e. the constraints imposed by the dynamics of the robot and the limits of its actuators). Some experimental results are discussed.

This paper addresses reactive collision avoidance for robots that move in arduous environments (i.e., very dense, complex and cluttered). To achieve this goal, the technique simplifies the difficulty of the navigation by a divide and conquer strategy, which is based on identifying navigational situations and applying the corresponding motion laws. The state of the art in reactive navigation still presents classic limitations such as trap situations due to U-shape obstacles, difficulty to achieve motion among very close obstacles, to obtain oscillation-free and stable motion, to move over directions far from the goal direction or towards the obstacles, or to tune the heuristic or internal parameters. This paper presents a method that overcomes all these limitations. As a result, navigation with this method is successfully achieved in scenarios where existing techniques present a high degree of difficulty to navigate. Outstanding navigation results are reported using a wheelchair vehicle. A discussion and comparison with existing techniques is provided.

We address here a sensor-based navigation system to safely drive vehicles in realistic scenarios. The system is composed of three modules with the following functionalities: model builder, planning and reactive motion. These modules are integrated within a planner-reactor architecture that supervises and coordinates them in order to carry out the motion task. The advantage of our system is to achieve a robust and trustworthy navigation in difficult scenarios, which remain troublesome for many of the existing systems. In order to validate the system, we present experiments with a wheelchair vehicle transporting a human among locations in an office type scenario.

This paper presents a new method for relative localization of a pair of robots based on the trajectories described by unidentifiable moving objects. Our approach uses a Rao-Blackwellized particle filter to estimate both the relative location of the robots and the data associations between the moving objects around the robots. We describe our implementation on real robots and present experiments illustrating the robustness of our algorithm.

This paper addresses the reactive collision avoidance for vehicles that move in very dense, cluttered, and complex scenarios. First, we describe the design of a reactive navigation method that uses a "divide and conquer" strategy based on situations to simplify the difficulty of the navigation. Many techniques could be used to implement this design (since it is described at symbolic level), leading to new reactive methods that must be able to navigate in arduous environments (as the difficulty of the navigation is simplified). We also propose a geometry-based implementation of our design called the nearness diagram navigation. The advantage of this reactive method is to successfully move robots in troublesome scenarios, where other methods present a high degree of difficulty in navigating. We show experimental results on a real vehicle to validate this research, and a discussion about the advantages and limitations of this new approach.

This paper presents a framework to use collision avoidance methods in the majority of existing mobile robots (that have any shape, kinematic and dynamic constraints). The solution proposed is a vehicle abstraction layer based on transforming the space where these methods work, onto another one in which the constraints are implicitly represented. This space incorporates the vehicle kinematics and dynamics in such a way that when the reactive navigation methods are used, the motions computed comply with the motion constraints. We validate the utility of this framework by applying a classic reactive method in a real vehicle with motion constraints (whereas the original method does not address the kinematics and dynamics).

This paper describes a method for the identification of moving objects by a team of robots based on kinematic information. The objective is to be able to identify moving objects observed by different robots without using specific landmarks. Our method uses a Bayesian approach and is based on the matching of maps of dynamic objects built by the members of the team. These maps contain the relative position of moving objects and their velocity at a given time. Experimental results using data from a real environment carried out to validate the method are presented and discussed.

We address the problem of applying reactive navigation methods to non-holonomic robots. Rather than embedding the motion constraints when designing a navigation method, we propose to introduce the robot's kinematic constraints directly in the spatial representation. In this space - the ego-kinematic space - the robot moves as a "free-flying object". Hence, standard reactive navigation methods applied to this space will automatically take into account the robot's kinematic constraints, without additional modifications. This methodology can be used with a large class of constrained mobile platforms (e.g. differential-driven robots, car-like robots, tri-cycle robots). We show experiments involving non-holonomic robots with two reactive navigation methods whose original formulation does not take the robot kinematic constraints into account (the Nearness Diagram Navigation and a Potential Field method).

We address the problem of applying reactive navigation methods for collision avoidance to systems where the dynamics cannot be neglected: mobile robots with slow dynamic capabilities, or systems working at high speeds. Rather than embedding the motion constraints when designing a navigation method, we propose to introduce the robot dynamic constraints directly into the spatial representation. In this space the dynamic capabilities of the robot are implicitly represented. With minor modifications, standard reactive navigation methods can be used in this space implicitly taking into account the robot dynamic constraints. To validate this framework, we show experimental results using two reactive navigation methods whose original formulation do not take the robot dynamic constraints into account (the nearness diagram navigation and the potential field method).

Presents the global nearness diagram navigation system for mobile robots. The GND generates motion commands to drive a robot safely between locations, whilst avoiding collisions. This system has all the advantages of using the reactive scheme nearness diagram (ND), while having the ability to reason and plan globally (reaching global convergence to the navigation problem). This framework has been extensively tested using a holonomic mobile base equipped with a laser range-finder. Experiments in unknown, unstructured, dynamic and complex environments are reported to validate the system.

This paper presents a new real-time collision avoidance approach for mobile robots. The nearness diagram method (ND) performs a high level information extraction and interpretation of the environment. Subsequently, this information is used to generate the motion commands. The proposed approach is well-suited to deal with unknown, unstructured and dynamic environments, where problems of other approaches are avoided. Some experimental results are shown using an holonomic mobile base to demonstrate the usefulness of the method.

This paper presents a method to perform a goal directed reactive navigation in unknown indoor environments. Two sensors cooperate to accomplish this task: trinocular vision and 3D laser rangefinder. Trinocular vision selects the initial goal location for the navigation task. Laser is used to accomplish a reactive navigation to avoid the obstacles and to periodically relocate the goal with respect to the robot, so the dead-reckoning drift is compensated. An extended Kalman filter is used to solve the data association problem and to perform the goal relocation while the robot navigates. Experimental results involving a real mobile robot are presented, validating the proposed method.

In this paper a technique for real-time robot navigation is presented. Off-line planned trajectories and motions are modified in real-time to avoid obstacles, using a reactive behaviour. The information about the environment is provided to the control system of the robot by a rotating 3D laser sensor which have two degrees of freedom. Using this sensor, three dimensional information can be obtained, and can be used simultaneously for obstacle avoidance, robot self-localization and for 3D local map building. In this work we focus the attention in the obstacle avoidance problem with this kind of sensor. Experimental results in different indoor environments are reported, which show the versatility of the proposed technique to carry out typical tasks such as wall following, door crossing and motion in a room with several obstacles.