We present AdaptiVe Optimal Collision Avoidance Driven by Opinion (AVOCADO), a novel navigation approach to address holonomic robot collision avoidance when the robot does not know how cooperative the other agents in the environment are. AVOCADO departs from a velocity obstacle's (VO) formulation akin to the optimal reciprocal collision avoidance method. However, instead of assuming reciprocity, it poses an adaptive control problem to adapt to the cooperation level of other robots and agents in real time. This is achieved through a novel nonlinear opinion dynamics design that relies solely on sensor observations. As a by-product, we leverage tools from the opinion dynamics formulation to naturally avoid the deadlocks in geometrically symmetric scenarios that typically suffer VO-based planners. Extensive numerical simulations show that AVOCADO surpasses existing motion planners in mixed cooperative/noncooperative navigation environments in terms of success rate, time to goal and computational time. In addition, we conduct multiple real experiments that verify that AVOCADO is able to avoid collisions in environments crowded with other robots and humans.

The networked nature of multi-robot systems presents challenges in the context of multi-agent reinforcement learning. Centralized control policies do not scale with increasing numbers of robots, whereas independent control policies do not exploit the information provided by other robots, exhibiting poor performance in cooperative-competitive tasks. In this work we propose a physics-informed reinforcement learning approach able to learn distributed multi-robot control policies that are both scalable and make use of all the available information to each robot. Our approach has three key characteristics. First, it imposes a port-Hamiltonian structure on the policy representation, respecting energy conservation properties of physical robot systems and the networked nature of robot team interactions. Second, it uses self-attention to ensure a sparse policy representation able to handle time-varying information at each robot from the interaction graph. Third, we present a soft actor-critic reinforcement learning algorithm parameterized by our self-attention port-Hamiltonian control policy, which accounts for the correlation among robots during training while overcoming the need of value function factorization. Extensive simulations in different multi-robot scenarios demonstrate the success of the proposed approach, surpassing previous multi-robot reinforcement learning solutions in scalability, while achieving similar or superior performance (with averaged cumulative reward up to$\times 2$greater than the state-of-the-art with robot teams$\times 6$larger than the number of robots at training time). We also validate our approach on multiple real robots in the Georgia Tech Robotarium under imperfect communication, demonstrating zero-shot sim-to-real transfer and scalability across number of robots.

In the stochastic estimation context, the absence of measurement information at the state estimator during large intervals can cause a divergence in the uncertainty of the estimates. This issue is aggravated when strategies to reduce communication, such as event-triggering mechanisms (ETMs), are used if an appropriate design is not made. Particularly, dynamic ETMs (DETMs) may exhibit this problem, since they are designed to further reduce the number of communication instants. Motivated by this problem, we propose a novel state estimator that integrates discrete transmitted measurements and implicit information between events provided by the proposed DETM. Our proposal guarantees a uniformly bounded mean-squared error in the stochastic context, regardless of transmission instants. Moreover, compared to static ETMs, our proposal adaptively reduces the number of transmissions according to the behavior of the measured signal. Our proposal’s advantages are verified formally and through several numerical experiments.

A well–known drawback in distributed average consensus of multi–agent systems is that the exchanged information is usually delayed due to the time elapsed during the data transmission process. Using classical dynamic average consensus, delays may lead to poor performance or even instability. In this paper, we propose a novel dynamic consensus method that counteracts the negative effects of delays by means of delay compensation techniques. The interest of our dynamic consensus method with delay compensation is that it converges under mild conditions on graph connectivity and bounded reference signals, no matter how large the delays are, as long as delays are fixed and known. We also provide a formal characterization of the convergence speed of our method. Additionally, our results apply to fixed directed strongly connected, and undirected topologies.

This paper presents LEMURS, an algorithm for learning scalable multi-robot control policies from cooperative task demonstrations. We propose a port-Hamiltonian description of the multi-robot system to exploit universal physical constraints in interconnected systems and achieve closed-loop stability. We represent a multi-robot control policy using an architecture that combines self-attention mechanisms and neural ordinary differential equations. The former handles time-varying communication in the robot team, while the latter respects the continuous-time robot dynamics. Our representation is distributed by construction, enabling the learned control policies to be deployed in robot teams of different sizes. We demonstrate that LEMURS can learn interactions and cooperative behaviors from demonstrations of multi-agent navigation and flocking tasks.

In this paper, we propose a formation control system for deforming and transporting simultaneously a de-formable object with a team of robots, modeled with double-integrator dynamics. The goal is to reach a target configuration, defined as a combination of shape, scale, orientation and position of the formation. We augment this controller with a set of control barrier functions (CBFs). The CBFs allow us to satisfy fundamental constraints for the success of the task: avoidance of agent-to-agent, agent-to-obstacle and object-to-obstacle collisions, and of excessive stretching. We test the performance of our proposal in different simulation scenarios.

This article presents a novel control strategy to herd groups of noncooperative evaders by means of a team of robotic herders. In herding problems, the motion of the evaders is typically determined bystrongly nonlinearandheterogeneous reactivedynamics, which makes the development of flexible control solutions a challenging problem. In this context, we propose Implicit Control, an approach that leverages numerical analysis theory to find suitable herding inputs even when the nonlinearities in the evaders’ dynamics yieldimplicit equations. The intuition behind this methodology consists in driving the input, rather than computing it, toward theunknownvalue that achieves the desired dynamic behavior of the herd. The same idea is exploited to develop an adaptation law, with stability guarantees, that copes with uncertainties in the herd’s models. Moreover, our solution is completed with a novel caging technique based on uncertainty models and control barrier functions, together with a distributed estimator to overcome the need of complete perfect measurements. Different simulations and experiments validate the generality and flexibility of the proposal.

In this article, we consider a scenario of cooperative task servicing, with a team of heterogeneous robots with different maximum speeds and communication radii, in charge of keeping the network intermittently connected. We abstract the task locations into a one-dimensional cycle graph that is traversed by the communicating robots, and we discuss intermittent communication strategies so that each task location is periodically visited, with a worst-case revisiting time. Robots move forward and backward along the cycle graph, exchanging data with their previous and next neighbors when they meet, and updating their region boundaries. Asymptotically, each robot is in charge of a region of the cycle graph, depending on its capabilities. The method is distributed, and robots only exchange data when they meet.

Autonomous manipulation of deformable objects is a research topic of increasing interest due to the variety of current processes and applications that include this type of tasks. It is a complex problem that involves aspects such as modeling, control, perception, planning, grasping, estimation, etc. A single robot may be unable to perform the manipulation when the deformable object is too big, too heavy or difficult to grasp. Then, using multiple robots working together naturally arises as a solution to perform coordinately the manipulation task. In this paper, we contribute a survey of relevant state-of-the-art approaches concerning manipulation of deformable objects by multiple robots, which includes a specific classification with different criteria and a subsequent analysis of the leading methods, the main challenges and the future research directions.

Persistent coverage aims to maintain a certain coverage level over time in an environment where such level deteriorates. This level can be associated to temperature, dust or sensor information. We propose an algorithmic solution in which each robot locally finds the best paths and coverage actions to keep the desired coverage level over the whole environment. Using Fast Marching Methods, optimal paths are computed in terms of coverage quality, while keeping a safety distance to obstacles. Additionally, our solution enables a computationally efficient evaluation of a list of potential trajectories, allowing us to choose the one that mostly improves the coverage along the whole path. The combination of this algorithm with a Dynamic Window navigation makes our approach competitive in terms of flexibility and robustness in changing environments with existing solutions. Finally, we also propose a coverage action controller, locally computed and optimal, that makes the robots maintain the coverage level of the environment significantly close to the objective. Simulations and real experiments validate the whole approach.

In this paper we consider the problem of controlling a team of non-holonomic robots to reach a desired formation. The formation is described in terms of the desired relative positions and orientations the robots need to keep with respect to each other, and it is assumed that the robots do not have a common shared reference frame. In other words, the robots can use only on-board sensing to achieve the formation. We first consider a holonomic framework, using a well known distance-based approach to reach a formation for the positions. We then include a control law for the orientations. We further discuss the problem of mirror configurations that appear when different desired relative orientations can satisfy the same distance-based constraints through different formations. Exploiting the concept of chirality, we present a relabeling strategy to reassign the robots' roles to reach the desired pattern when a mirror configuration occurs. The distance-based holonomic control is then transformed to cope with the non-holonomic constraints using a piecewise-smooth function. Simulation results, as well as hardware experiments with five m3pi robots demonstrate the applicability of our approach.

In this paper, we present a fully distributed solution to drive a team of robots to reach a desired formation in the absence of an external positioning system that localizes them. Our solution addresses two fundamental problems that appear in this context. First, we propose a 3-D distributed control law, designed at a kinematic level, that uses two simultaneous consensus controllers: one to control the relative orientations between robots, and another for the relative positions. The convergence to the desired configuration is shown by comparing the system with time-varying orientations against the equivalent approach with fixed orientations, showing that their difference vanishes as time goes to infinity. Second, in order to apply this controller to a group of aerial robots, we combine this idea with a novel sensor fusion algorithm to estimate the relative pose of the robots by using onboard cameras and information from the inertial measurement unit. The algorithm removes the influence of roll and pitch from the camera images and estimates the relative pose between robots by using a structure from the motion approach. Simulation results, as well as hardware experiments with a team of three quadrotors, demonstrate the effectiveness of the controller and the vision system working together.

In this paper, we address the problem of persistently covering an environment with a group of mobile robots. In contrast to traditional coverage, in our scenario the coverage level of the environment is always changing. For this reason, the robots have to continually move to maintain a desired coverage level. In this context, our contribution is a complete approach to the problem, including distributed estimation of the coverage and control of the motion of the robots. First, we present an algorithm that allows every robot to estimate the global coverage function only with local information. We pay special attention to the characterization of the algorithm, establishing bounds on the estimation error, and we demonstrate that the algorithm guarantees a perfect estimation in particular areas. Second, we introduce a new function to determine the possible improvement of the coverage at each point of the environment. Upon this metric, we build a motion control strategy that drives the robots to the points of the highest improvement while following the direction of the gradient of the function. Finally, we simulate the proposal to test its correctness and performance.

This paper describes a new vision-based control method to drive a set of robots moving on the ground plane to a desired formation. As the main contribution, we propose to use multiple camera-equipped unmanned aerial vehicles (UAVs) as control units. Each camera views, and is used to control, a subset of the ground team. Thus, the method is partially distributed, combining the simplicity of centralized schemes with the scalability and robustness of distributed strategies. Relying on a homography computed for each UAV-mounted camera, our approach is purely image-based and has low computational cost. In the control strategy we propose, if a robot is seen by multiple cameras, it computes its motion by combining the commands it receives. Then, if the intersections between the sets of robots viewed by the different cameras satisfy certain conditions, we formally guarantee the stabilization of the formation, considering unicycle robots. We also propose a distributed algorithm to control the camera motions that preserves these required overlaps, using communications. The effectiveness of the presented control scheme is illustrated via simulations and experiments with real robots.

This paper presents a novel method that enables a team of aerial robots to enclose a target in 3D space by attaining a desired geometric formation around it. We propose an approach in which each robot obtains its motion commands using measurements of the relative position of the other agents and of the target, without the need for a central coordinator. As contribution, our method permits any desired 3D target enclosing configuration to be defined, in contrast with the planar circular patterns commonly encountered in the literature. The proposed control strategy relies on the minimization of a cost function that captures the collective motion objective. In our method, the robots do not need to use a common reference frame. This coordinate independence is achieved through the introduction in the cost function of a rotation matrix computed locally by each robot. We prove that our motion controller is exponentially stable, and illustrate its performance through simulations.

We study the feature-based map merging problem in robot networks. Each robot observes the environment and builds a local map. Simultaneously, robots communicate and compute the global map of the environment; this communication is range-limited. We propose a dynamic strategy based on consensus algorithms that is fully distributed and does not rely on any particular communication topology. Robots reach consensus on the latest global map, using the increments between their previous and current local maps. Under mild connectivity conditions, our merging algorithm asymptotically converges to the global map. We give proofs of unbiasedness of this global map, at each step and robot. Our approach has been validated using real RGB-D images.

We present a method for visual control of a set of robots moving on the ground plane. The goal of the control task is for the team to reach a desired geometric configuration. Each robot carries an omnidirectional camera and can communicate with a number of the other robots. The approach relies on the computation of the planar motion between two views, by means of 1D homographies. This knowledge, obtained by each robot from its own images and the visual information received from neighboring robots, allows it to define a desired position on the plane. Then, we propose a novel control scheme based on computing a particular 2D transformation to drive each robot towards its goal position. A contribution of this work is the use of 1D homography in a multirobot control framework. This tool allows to deal with purely angular visual information, which is precise and requires no calibration. The approach we present is completely distributed. Each robot uses only information from its formation neighbors and the global centroid to obtain its motion commands. These individual behaviors naturally result in the complete team of robots reaching the desired global configuration.

We address the data association problem of features that are observed by a robotic network. Every robot in the network has limited communication capabilities and can only exchange local matches with its neighbors. We propose a distributed algorithm that takes these local matches and, by their propagation in the network, computes global correspondences. When the algorithm finishes, each robot knows the correspondences between its features and the features of all the other robots, even if they cannot directly communicate. The presence of spurious local correspondences may produce inconsistent global correspondences, which are association paths between features observed by the same robot. The contributions of this study are the propagation of the local matches and the detection and resolution of these inconsistencies. We formally prove that after executing the algorithm, all the robots finish with a data association that is free of inconsistencies. We provide a fully decentralized solution to the problem that is valid for any fixed communication topology and with bounded communications between the robots. Simulations and experimental results with real images show the performance of the method considering different features, matching functions, and robotic applications.

In this paper, we give a distributed solution to the problem of making a team of nonholonomic robots reach consensus about their orientations using monocular cameras. We consider a scheme where the motions of the robots are decided using nearest-neighbor rules. Each robot is equipped with a camera and can only exchange visual information with a subset of the other robots. The main contribution of this paper is a new controller that uses the epipoles that are computed from the images provided by neighboring robots, eventually reaching consensus in their orientations without the necessity of directly observing each other. In addition, the controller only requires a partial knowledge of the calibration of the cameras in order to achieve the desired configuration. We also demonstrate that the controller is robust to changes in the topology of the network and we use this robustness to propose strategies to reduce the computational load of the robots. Finally, we test our controller in simulations using a virtual environment and with real robots moving in indoor and outdoor scenarios.

This paper addresses the estimation of planar camera motion using 1D homographies. As contributions, we show analytically that, contrary to what occurs with the 2D homography, there is a family of infinite solutions to the 1D homography decomposition, and therefore infinite possible motion reconstructions. In addition, we propose a new method to compute the planar motion between two images from the information provided by two different 1D homographies, employing their associated homology transformations. Therefore, our approach computes a general planar camera motion from only two 1D views, when previous works needed three 1D views for this task. The use of 1D information makes the method particularly suitable for omnidirectional cameras, due to the wide field of view and precise angular information provided by this kind of sensors. The performance of our proposal is illustrated through simulations and experiments on real images.

In this paper, we study different solutions of flexible cooking zones for induction hobs. Particularly, we focus on the usage of mobile inductors in two dimensions. The main contribution is the development of a mobile induction system which allows to heat circular pots with any size, in any position, with a homogeneous heat distribution. The good performance of this solution is tested experimentally. We present a theoretical model of the induction heating process which we use to analyze the heat distribution in a pot, and the main parameters of the problem. We also evaluate different heating strategies with a prototype with one double mobile inductor. From the obtained results, we can establish the best activation strategy of the inductor rings and the motion strategy as a function of the size of the heated pot.

This paper is focused on dynamic coverage control with a team of robots. In this framework, decentralized control algorithms have been investigated to deal with the efficient coordination of the resources. The main contribution is a novel global strategy based on a hierarchical grid decomposition of the domain. This decomposition allows an ordered coverage of the domain that combined with a gradient based control law of the local error, achieves a better performance than previous approaches of dynamic coverage. The total coverage of the domain is proven, and the good performance of the approach is supported with simulations.

In this paper, we study the feature-based map merging problem in robot networks. While in operation, each robot observes the environment and builds and maintains a local map. Simultaneously, each robot communicates and computes the global map of the environment. Communication between robots is range-limited. We propose a dynamic strategy, based on consensus algorithms, that is fully distributed and does not rely on any particular communication topology. Under mild connectivity conditions on the communication graph, our merging algorithm, asymptotically, converges to the global map. We present a formal analysis of its convergence rate and provide accurate characterizations of the errors as a function of the timestep. The proposed approach has been experimentally validated using real visual information.

This paper addresses the problem of visual control of a set of mobile robots. In our framework, the perception system consists of an uncalibrated flying camera performing an unknown general motion. The robots are assumed to undergo planar motion considering nonholonomic constraints. The goal of the control task is to drive the multirobot system to a desired rendezvous configuration relying solely on visual information given by the flying camera. The desired multirobot configuration is defined with an image of the set of robots in that configuration without any additional information. We propose a homography-based framework relying on the homography induced by the multirobot system that gives a desired homography to be used to define the reference target, and a new image-based control law that drives the robots to the desired configuration by imposing a rigidity constraint. This paper extends our previous work, and the main contributions are that the motion constraints on the flying camera are removed, the control law is improved by reducing the number of required steps, the stability of the new control law is proved, and real experiments are provided to validate the proposal.

In this paper we address the problem of estimating the poses of a team of agents when they do not share any common reference frame. Each agent is capable of measuring the relative position and orientation of its neighboring agents, however these measurements are not exact but they are corrupted with noises. The goal is to compute the pose of each agent relative to an anchor node. We present a strategy where, first of all, the agents compute their orientations relative to the anchor. After that, they update the relative position measurements according to these orientations, to finally compute their positions. As contribution we discuss the proposed strategy, that has the interesting property that can be executed in a distributed fashion. The distributed implementation allows each agent to recover its pose using exclusively local information and local interactions with its neighbors. This algorithm has a low memory load, since it only requires each node to maintain an estimate of its own orientation and position.

In this paper, we propose to exploit the epipolar geometry for dynamic pose-estimation of mobile robots. This is performed using a filtering approach with measurements given by the epipoles. The contribution of the paper is a novel observability analysis using nonlinear and linear tools that leads to achieve an efficient position-based visual servoing (PBVS) approach. Additionally, the visibility constraint problem is solved for this type of approach by using any visual sensor obeying a central projection model. This scheme does not need a target model neither scene reconstruction nor the 3D structure. The effectiveness of the proposed estimation scheme is evaluated via simulations in servoing tasks including obstacles and using omnidirectional vision.

This paper addresses the problem of visual control of a set of mobile robots. In our framework, the perception system consists of a calibrated flying camera looking downward to the mobile robots. The robots are assumed to undergo planar motion considering nonholonomic constraints. The goal of the task is to control the multi-robot system to a desired configuration relying solely on visual information given by the flying camera. The desired multi-robot configuration is defined with an image of the set of robots in that configuration. Then, any arbitrary configuration can be easily defined by this image without any additional information. As contribution, a new image-based control scheme is presented relying on the homography induced by the multi-robot system to lead the robots to the desired configuration. The stability of the control law is analyzed and simulations are provided to illustrate the proposal.

Data association mistakes, estimation and measurement errors are some of the factors that can contribute to incorrect observations in robotic sensor networks. In order to act reliably, a robotic network must be able to fuse and correct its perception of the world by discarding any outlier information. This is a difficult task if the network is to be deployed remotely and the robots do not have access to ground-truth sites or manual calibration. In this paper, we present a novel, distributed scheme for robust data fusion in autonomous robotic networks. The proposed method adapts the RANSAC algorithm to exploit measurement redundancy, and enables robots determine an inlier observation with local communications. Different hypotheses are generated and voted for using a dynamic consensus algorithm. As the hypotheses are computed, the robots can change their opinion making the voting process dynamic. Assuming that at least one hypothesis is initialized with only inliers, we show that the method converges to the maximum likelihood of all the inlier observations in a general instance. Several simulations exhibit the good performance of the algorithm, which also gives acceptable results in situations where the conditions to guarantee convergence do not hold.

Driving mobile robots to precise locations is of recognized interest, and using vision sensors in this context supplies many advantages. We propose a novel control law based on sliding-mode theory in order to drive mobile robots to a target location, which is specified by a previously acquired reference image. The control scheme exploits the piecewise epipolar geometry of three views on the basis of image-based visual servoing, in such a way that no 3-D scene information is required. The contribution of the paper is a new control law that achieves convergence to the target with no auxiliary images and without changing to any approach other than epipolar-based control. Additionally, the use of sliding-mode control deals with singularities, thus allowing the robot to move directly toward the target as well as avoiding the need of a precise camera calibration. The effectiveness of our approach is tested with simulations and real-world experiments.

Catadioptric systems consist of the combination of lenses and mirrors. From them, central panoramic systems stand out because they provide a unique effective viewpoint, leading to the well-known unifying theory for central catadioptric systems. This paper considers catadioptric systems consisting of a conic mirror and a projective camera. Although a system with conic mirror does not possess a single projection point, it has some advantages as the cone is a very simple shape to produce, it has higher resolution in the peripheral, and adds less optical distortion to the images. The contributions of this work are the model of this non-central system by means of projective mappings from a torus to a plane, and the definition of the conic fundamental matrix with a role similar to that of perspective cameras. Additionally, a procedure to compute the relative motion between two views from the conic fundamental matrix is presented.

In this paper we present an algorithm for merging visual maps in a robot network. Along the operation, each robot observes the environment and builds and maintains its local map. Simultaneously, the robots communicate and build a global map of the environment. The communication between the robots is limited, and, at every time instant, each robot can only exchange data with its neighboring robots. We provide a distributed solution to the problem which does not rely on any particular communication topology and is robust to changes in the topology. Each robot computes and tracks the global map based on local interactions with its neighbors. Our contribution is the extension of distributed sensor fusion ideas to the problem of dynamic map merging. Under mild connectivity conditions on the communication graph, this algorithm asymptotically converges to the global map. The real experiments have been carried out with visual information, which is of special interest in robotics.

This paper presents a new method for visual homing to be used on a robot moving on the ground plane. A relevant issue in vision-based navigation is the field-of-view constraints of conventional cameras. We overcome this problem by means of omnidirectional vision and we propose a vision-based homing control scheme that relies on the 1D trifocal tensor. The technique employs a reference set of images of the environment previously acquired at different locations and the images taken by the robot during its motion. In order to take advantage of the qualities of omnidirectional vision, we define a purely angle-based approach, without requiring any distance information. This approach, taking the planar motion constraint into account, motivates the use of the 1D trifocal tensor. In particular, the additional geometric constraints enforced by the tensor improve the robustness of the method in the presence of mismatches. The interest of our proposal is that the designed control scheme computes the robot velocities only from angular information, being this very precise information; in addition, we present a procedure that computes the angular relations between all the views even if they are not directly related by feature matches. The feasibility of the proposed approach is supported by the stability analysis and the results from simulations and experiments with real images.

In this paper, we propose a visual path following control scheme for wheeled mobile robots based on the epipolar geometry. The control law only requires the position of the epipole computed between the current and target views along the sequence of a visual memory. The proposed approach has two main advantages: explicit pose parameters decomposition is not required and the rotational velocity is smooth or eventually piece-wise constant avoiding discontinuities that generally appear when the target image changes. The translational velocity is adapted as required for the path and the approach is independent of this velocity. Furthermore, our approach is valid for all cameras obeying the unified model, including conventional, central catadioptric and some fisheye cameras. Simulations as well as real-world experiments with a robot illustrate the validity of our approach.

In this paper, we present a visual servo controller that effects optimal paths for a nonholonomic differential drive robot with field-of-view constraints imposed by the vision system. The control scheme relies on the computation of homographies between current and goal images, but unlike previous homography-based methods, it does not use the homography to compute estimates of pose parameters. Instead, the control laws are directly expressed in terms of individual entries in the homography matrix. In particular, we develop individual control laws for the three path classes that define the language of optimal paths: rotations, straight-line segments, and logarithmic spirals. These control laws, as well as the switching conditions that define how to sequence path segments, are defined in terms of the entries of homography matrices. The selection of the corresponding control law requires the homography decomposition before starting the navigation. We provide a controllability and stability analysis for our system and give experimental results.

This paper presents an image-based approach to perform visual control for differential-drive robots. We use for the first time the elements of the 1D trifocal tensor directly in the control law. The visual control utilizes the usual teach-by showing strategy without requiring any a prior knowledge of the scene and does not need any auxiliary image. The main contribution of the paper is that the proposed two-steps control law ensures total correction of both position and orientation without switching to any other visual constraint rather than the 1D trifocal tensor. The paper exploits the sliding mode control technique in a square system, ensuring stability and robustness for the closed loop. The good performance of the control system is proven via simulations.

This paper addresses the problem of visual control of a mobile robot. The system consists of a calibrated camera fixed onboard a robot with nonholonomic motion constraints. The parking task is defined by a reference image taken at the target location. The proposed control law is based on the essential matrix, but unlike traditional methods, it is not used to compute pose parameters. Instead, the control law is defined directly in terms of individual entries of the essential matrix by means of the input-output linearization of the system. Here we solve the problem of degeneracies due to short baseline by taking advantage of the planar motion constraint of the robot. Thus, a virtual target is defined providing a stable estimation of the essential matrix without degeneracies despite short baseline.

This paper addresses the problem of vision-based closed-loop control for undulatory robots. We present an image-based visual servoing scheme, which drives the robot to a desired location specified by a target image, without explicitly estimating its pose. Instead, the control relies on the computation of the epipolar geometry between the current and target images. We analyze controllability and stability of the proposed control scheme, which is validated by simulation studies using the SIMUUN computational tools. Preliminary experiments, involving the Nereisbot undulatory robotic prototype, are also presented.

This paper describes a strategy to select optimal motions of multi robot systems equipped with cameras in such a way that they can successively improve the observation of the environment. We present a solution designed for omnidirectional cameras, although the results can be extended to conventional cameras. The key idea is the selection of a finite set of candidate next positions for every robot within their local landmark-based stochastic maps. In this way, the cost function measuring the perception improvement when a robot moves to a new position can be easily evaluated on the finite set of candidate positions. Then, the robots in the team can coordinate based on these small pieces of information. The proposed strategy is designed to be integrated with a map merging algorithm where robots fuse their maps to get a more precise knowledge of the environment. The interest of the proposed strategy for uncertainty reduction is that it is suitable for visual sensing, allows an efficient information exchange, presents a low computational cost and makes the robot coordination easier.

A pose-estimation-based approach to perform visual control for differential-drive robots is presented in this paper. Our scheme recovers the camera location (position and orientation) using an extended kalman filter (EKF) algorithm with the 1D trifocal tensor (TT) as measurement model. This new visual servoing scheme allows knowing the real world path performed by the robot without the computational load introduced by position-based approaches. A state-estimated feedback control law is designed to solve a tracking problem for the lateral and longitudinal robot coordinates. The desired trajectories to be tracked ensure total correction of both position and orientation using a single control law, even though the orientation is a DOF in the control system. The effectiveness of our approach is tested via simulations.

Topological visual maps contain different abstraction levels of information that can be used by robots to carry out different activities. We propose here a new hierarchical structure in which landmarks extracted from conventional images are grouped creating a graph of planar regions. The new hierarchy improves previous approaches based on images reducing both, the size of the graph and its complexity. In order to segment and group the planar regions of a sequence of images a new approach based on the simultaneous matching of two images and the previously extracted planar regions is proposed. We also consider multi-plane restrictions so that the method is robust to the appearance of new planes. The paper presents two contributions. First the triple matching approach to extract all the planes seen in the set of images and second a new topological map construction based on a graph of planar regions which can be used by mobile robots to localize and move in the environment. Experiments with real images in both indoor and outdoor environments show good performance of our proposal.

In this paper we propose a new algorithm for relative pose estimation between two images based on a new decomposition for an homography matrix faster than the classical solutions. We introduce in our method approximate information about the planes in the reference images but this additional information allows the decomposition to avoid the multiplicity of solutions. An exhaustive analysis of the error propagation through the method is provided. The new decomposition can be used for visual navigation of robots moving on a planar surface. The approach is based in the well known teaching-by-doing scheme with the route defined by a set of images. The parameters of the planes can be computed automatically from the reference images before the navigation. The contributions are the easy method to extract the parameters of all the planes and the use of this information in navigation tasks using a fast homography decomposition. The experimental results show the performance of the proposal.

Nowadays we frequently find big amounts of data to work with, what facilitates many robotic tasks and helps to solve perception problems. At the same time, this fact origins an interesting ongoing research problem: how to organize and arrange big sets of information to be useful in later uses. Topological mapping is a very useful tool to arrange and deal with big amounts of reference images for robotic tasks. There are many previous works on topological mapping and many others use this kind of maps for topological localization, planning and navigation. This work is focused on the problem of carefully design topological map building processes that facilitate the posterior robot tasks that use them and make them safer. We propose a new hierarchy of topological maps focused on this aspect. The experiments included in this paper were run outdoors using omnidirectional images and GPS information, and show the good topological maps obtained and how they allow robust and safer localization and navigation tasks.

In this paper we address the problem of visual navigation of a mobile robot which simultaneously obtains metric localization and scene reconstruction using homographies. Initially, the robot is guided by a human and some scenes during the trip are stored from known reference locations. The interest of this paper consist in the possibility of getting real and precise data of the robot motion and the scene, which presents some advantages over other existing approaches. For example, it allows the robot to carry out other trajectories than the executed during the teaching phase. We show an extensive analysis of the output in presence of errors in some of the inputs.

In this paper, a robust control technique (sliding mode control) is proposed to be used in order to perform visual servoing for differential-drive mobile robots using the classical teach-by-showing strategy. We propose a commuted sliding mode control law that exploits the epipolar geometry. The major contribution of the paper is the design of a control law that solves the problem of passing through a singularity induced by the epipoles maintaining bounded inputs. Moreover, the designed control is able to drive the robot to the target even when it just starts on the singularity. The proposed approach does not need a precise camera calibration due to the robustness of the control system under uncertainty in parameters. It also ensures entire correction of both orientation and lateral error even with noise in the image. The effectiveness of our approach is tested via simulations.

This paper addresses the robot and landmark localization problem from bearing-only data in three views, simultaneously to the robust association of this data. The localization algorithm is based on the 1-D trifocal tensor, which relates linearly the observed data and the robot localization parameters. The aim of this work is to bring this useful geometric construction from computer vision closer to robotic applications. One contribution is the evaluation of two linear approaches of estimating the 1-D tensor: the commonly used approach that needs seven bearing-only correspondences and another one that uses only five correspondences plus two calibration constraints. The results in this paper show that the inclusion of these constraints provides a simpler and faster solution and better estimation of robot and landmark locations in the presence of noise. Moreover, a new method that makes use of scene planes and requires only four correspondences is presented. This proposal improves the performance of the two previously mentioned methods in typical man-made scenarios with dominant planes, while it gives similar results in other cases. The three methods are evaluated with simulation tests as well as with experiments that perform automatic real data matching in conventional and omnidirectional images. The results show sufficient accuracy and stability to be used in robotic tasks such as navigation, global localization or initialization of simultaneous localization and mapping (SLAM) algorithms.

This paper presents a switched homography-based visual control for differential drive vehicles. The goal is defined by an image taken at the desired position, which is the only previous information needed from the scene. The control takes into account the field-of-view constraints of the vision system through the specific design of the paths with optimality criteria. The optimal paths consist of straight lines and curves that saturate the sensor viewing angle. We present the controls that move the robot along these paths based on the convergence of the elements of the homography matrix. Our contribution is the design of the switched homography-based control, following optimal paths guaranteeing the visibility of the target.

This paper presents a new visual control approach based on homography. The method is intended for nonholonomic vehicles with a fixed monocular system on board. The idea of visual control used here is the usual approach where the desired position of the robot is given by a target image taken at that position. This target image is the only previous information needed by the control law to perform the navigation from the initial position to the target. The control law is designed by the input-output linearization of the system using elements of the homography as output. The contribution is a controller that deals with the nonholonomic constraints of the mobile platform needing neither decomposition of the homography nor depth estimation to the target.

Many robotic applications work with visual reference maps, which usually consist of sets of more or less organized images. In these applications, there is a compromise between the density of reference data stored and the capacity to identify later the robot localization, when it is not exactly in the same position as one of the reference views. Here we propose the use of a recently developed feature, SURF, to improve the performance of appearance-based localization methods that perform image retrieval in large data sets. This feature is integrated with a vision-based algorithm that allows both topological and metric localization using omnidirectional images in a hierarchical approach. It uses pyramidal kernels for the topological localization and three-view geometric constraints for the metric one. Experiments with several omnidirectional images sets are shown, including comparisons with other typically used features (radial lines and SIFT). The advantages of this approach are proved, showing the use of SURF as the best compromise between efficiency and accuracy in the results.

A significant amount of work has been reported in the area of visual servoing during the last decade. However, most of the contributions are applied in cases of holonomic robots. More recently, the use of visual feedback for control of nonholonomic vehicles has been reported. Some of the examples are docking and parallel parking maneuvers of cars or vision-based stabilization of a mobile manipulator to a desired pose with respect to a target of interest. Still, many of the approaches are mostly interested in the control part of visual servoing loop considering very simple vision algorithms based on artificial markers. In this paper, we present an approach for nonholonomic visual servoing based on epipolar geometry. The method facilitates a classical teach-by-showing approach where a reference image is used to define the desired pose (position and orientation) of the robot. The major contribution of the paper is the design of the control law that considers nonholonomic constraints of the robot as well as the robust feature detection and matching process based on scale and rotation invariant image features. An extensive experimental evaluation has been performed in a realistic indoor setting and the results are summarized in the paper

In this paper we present a technique to linearly recover 2D structure and motion in man made environments from three uncalibrated omnidirectional views. We use vertical lines from the scene which are projected as radial lines in the images and are automatically matched. The algorithm is based on a 1D radial trifocal tensor which encodes the relations of the three views and the projected lines. We include experiments with real images, which demonstrate the good performance of the method and its application to robotic tasks, such as robot localization based in a database of reference images or to obtain the initial values of robot and landmarks localization in SLAM algorithms

This paper presents a method for robot and landmarks 2D localization, in man made environments, taking profit of scene planes. The method uses bearing-only measurements that are robustly matched in three views. In our experiments we obtain them from vertical lines corresponding to natural landmarks. With these three view line-matches a trifocal tensor can be computed. This tensor contains the three views geometry and is used to estimate the aforementioned localization. As it is very usual to find a planar surface, we use the homography corresponding to that plane to obtain the tensor with one match less than the general case method. This implies lower computational complexity, mainly when trying a robust estimation, where we see a reduction in the number of iterations needed. Another advantage of obtaining an homography during the process is that it can help to automatically detect singular situations, such us totally planar scenes. It is shown that our proposal performs similarly to the general case method in a general scenario and better in case that we have some dominant plane in the scene. This paper includes simulated results proving this, as well as examples with real images, both with conventional and omnidirectional cameras