We address the problem of computing the optimal Q-function in Markov decision problems with infinite state-space. We analyze the convergence properties of several variations of Q-learning when combined with function approximation, extending the analysis of TD-learning in (Tsitsiklis & Van Roy, 1996a) to stochastic control settings. We identify conditions under which such approximate methods converge with probability 1. We conclude with a brief discussion on the general applicability of our results and compare them with several related works.

In this paper, we propose a reinforcement learning approach to address multi-robot cooperative navigation tasks in infinite settings. We propose an algorithm to simultaneously address the problems of learning and coordination in multi-robot problems. The proposed algorithm extends those existing in the literature, allowing to address simultaneous learning and coordination in problems with an infinite state-space. We also present the results obtained in several test scenarios featuring multi-robot navigation situations with partial observability.

This work describes a semi-autonomous robot for rescue operations, nicknamed RAPOSA (FOX in English). The robot was designed and built to operate in outdoor environments hostile to the human presence, such as debris resulting from the collapse of built structures, and is targeted to the tele-operated detection of potential survivors using a set of specific sensors whose information is transmitted to a remote human operator. RAPOSA's mechanical structure is composed of a main body and a front body, whose locomotion is supported on tracked wheels, allowing motion even when the robot is upside down. The front body has variable tilting capabilities, providing means to overcome edges higher than the robot main body (e.g., when climbing a stair) and is also useful to grab the lower ground when only the main body has ground contact. This front body has one thermal camera and two web cameras installed. Additional sensors include gas, temperature and humidity sensors, Web cams, light diodes, microphone and loudspeaker. The robot uses wireless communications, with an option for tethered operation. The tether carries both power and communications, with an access point on its end, and can also be used to suspend the robot inside a deep hole. Docking and undocking the robot to the tether is accomplished remotely by the operator with the help of a camera located inside the robot, and represents the most innovative feature of RAPOSA

This paper presents a hybrid architecture for robot control supported on basic concepts from the geometry of Hilbert spaces and nonsmooth calculus. The architecture develops in two classic layers. A supervising layer chooses which motion strategies to apply from a set contained in an execution layer. The paper focuses mainly in the lower execution layer, motivated by the control of semi-autonomous robots. A set of experiments on the control of unicycle robots is presented.

This paper describes a multi-loop, modular navigation architecture for mobile robots whose structure allows the execution of most types of navigation tasks in a highly robust manner. The modularity allows the clear separation of functions according to their complexity and priority. We propose a clear separation between local and global navigation in such a way that both can run independently, but an adequate alternation/competition between them allows accomplishment of trajectory execution including avoidance of unknown obstacles. The multi-loop nature of the architecture ensures adequate stability at different levels yielding safe navigation and accomplishment of higher level tasks. Examples may range from goal reaching in some point of the environment to a 3D-environment mapping application, as this is the case in this work. 1. Introduction Navigation is a major issue when addressing mobile robotics because the concept is so wide that it includes all aspects of directing a robot's ...

Presents an active scheme for the localization of a mobile robot based on the detection of natural landmarks. Assuming that a partial 3D model of the environment is known a priori, an optimal and active choice of the landmark is supported on a new strategy for 3D map partition, yielding high updating rates of the location estimates. Good localization accuracy and robustness are achieved by combining laser range data with odometry, using Kalman filtering schemes.

This paper presents a new 3D scene analysis system that automatically reconstructs the 3D model of real-world scenes from multiple range images acquired by a laser range finder on board of a mobile robot. The reconstruction is achieved through an integrated procedure including range data acquisition, geometrical feature extraction, planning the next view, registration and integration of multiple views. The system relies only on the acquired data. Experimental results on real range images are presented in the paper. Direct applications of this technique include 3D reconstruction and/or update of architectural or industrial plans into a CAD model, design verification of buildings, navigation of autonomous robots, and input to virtual reality systems.

This paper presents an active sensing process for the efficient reconstruction of a real world environment using multiple views of a scene captured by a laser range finder onboard a mobile robot. To resolve the ambiguities caused by occlusions there is the need to acquire multiple range views at different viewpoints. A strategy for optimising the selection of the next sensing position and orientation based on previously detected occlusions is proposed. No a priori knowledge about the scene is used for planning the next views. The system evaluates the set of capture points from which all the necessary data can be acquired aiming at minimising its number. Among these points it then selects the one(s) maximising the occluded areas to be resolved.

Two alternative ARMA (autoregressive moving average) estimators are compared both theoretically and through simulation analysis. The first is a dual algorithm that estimates the MA and the AR components as the solution of two linear and independent systems of equations. For the second estimator, the AR coefficients result from a system of linear equations, while the MA component is obtained from a fast filtering algorithm initialized with the previous AR estimated coefficients.>

The authors study, for an ARMA (autoregressive moving-average) (p/sub 0/, q/sub 0/) process, the joint determination from a finite data sample of its structural parameters p/sub 0/ and q/sub 0/, its AR and MA components, and its innovation power sigma /sup 2/. The order estimation algorithm is based on the minimization of a functional d that measures the mismatch of the assumed model ARMA (p, q) to the data. The functional is evaluated from the estimated reflection coefficient sequence associated with the process. When the orders are decided, the proposed technique simultaneously provides the estimates of the AR and the MA coefficients as well as sigma /sup 2/.>

The present work describes an ARMA estimation aIgorithm that differs from the known available techniques. It substitutes the autocorrelation estimation sequence by the sequence of estimated reflection coefficients. These are reliably provided by the Burg technique [1]. Then it fits to the process both a sequence of higher order linear predictors (e.g., Levinson algorithm), and a sequence of higher order linear innovations filters (e.g., by recursive inversion). Finally, it obtains the MA coefficients from the linear relations satisfied by the corresponding coefficients of the successive higher order linear predictors, and likewise obtains the AR coefficients from the linear relations satisfied by the corresponding coefficients of the successive higher order innovation filters. We stress that the procedure does not use the sample autocorrelation lags; it uses instead the sequence of sample reflection coefficients, from which it estimates independently of each other and in a dual way, the MA and the AR components of the process.