One of the primary challenges in building a General Purpose Service Robot (GPSR), i.e. a robot capable of executing generic human commands, lies in acting upon natural language instructions. These instructions often contain speech recognition errors and incomplete information, complicating the extraction of clear goals and the formulation of an efficient and effective action plan. This work presents a pipeline that leverages a Large Language Model to directly translate instruction transcripts into coherent action plans. The pipeline also integrates environmental context into the model’s input, allowing for the generation of more efficient and context-aware plans. The system’s performance was evaluated using a simulator based on generalized stochastic Petri Nets, achieving a success rate of around 55% on the ALFRED dataset, even in unseen environments. The entire pipeline was also successfully deployed at RoboCup 2024 in Eindhoven, where it secured second place in the GPSR task. The code, dataset, and models are available at https://github.com/socrob/llm_gpsr.

Efficient exploration of large-scale environments remains a critical challenge in robotics, with applications ranging from environmental monitoring to search and rescue operations. This article proposes Frontier Shepherding (FroShe), a bio-inspired multi-robot framework for large-scale exploration. The framework heuristically models frontier exploration based on the shepherding behavior of herding dogs, where frontiers are treated as a swarm of sheep reacting to robots modeled as shepherding dogs. FroShe is robust across varying environment sizes and obstacle densities, requiring minimal parameter tuning for deployment across multiple agents. Simulation results demonstrate that the proposed method performs consistently, regardless of environment complexity, and outperforms state-of-the-art exploration strategies by an average of 20% with three UAVs. The approach was further validated in real-world experiments using single-and dual-drone deployments in a forest-like environment.

Navigating in environments alongside humans requires agents to reason under uncertainty and account for the beliefs and intentions of those around them. Under a sequential decision-making framework, egocentric navigation can naturally be represented as a Markov Decision Process (MDP). However, social navigation additionally requires reasoning about the hidden beliefs of others, inherently leading to a Partially Observable Markov Decision Process (POMDP), where agents lack direct access to others’ mental states. Inspired by Theory of Mind and Epistemic Planning, we propose (1) a neuro-symbolic model-based reinforcement learning architecture for social navigation, addressing the challenge of belief tracking in partially observable environments; and (2) a perspective-shift operator for belief estimation, leveraging recent work on Influence-based Abstractions (IBA) in structured multi-agent settings.

Advancements in Automated Driving Systems (ADSs) have enabled the achievement of a certain level of autonomy while commuting in a car. However, emergency and high-speed maneuvers still arise as significant challenges for ADSs due to the intrinsic nonlinearity and fast-paced behavior of such events. These maneuvers are a distinctive feature within the recently established motorsport discipline of Autonomous Racing (AR). In this work, we explore the use of Learning-based Model Predictive Control (LMPC) to address possible model mismatches of the first principles model in high-speed racing. To this end, a Model Predictive Contouring Control (MPCC) (a specific formulation of the standard Model Predictive Control, MPC) is formulated, and a Neural Network (NN) that leverages the use of Feedforward and Recurrent layers is employed to learn the errors of the first principles model. By combining the NN with the first principles model, the LMPC is born, capable of accurately predicting the future with a computational effort compatible with real-time feasibility, effectively handling the vehicle at its limits. Furthermore, the controller can adapt to changing environments by training the NN during the race. The MPCC (formulation without the NN) is deployed on a real autonomous formula student racing car showing an improvement of 16 % in mean lap times across the same track between a common geometric controller. The LMPC is analyzed in a high-fidelity simulator, achieving an improvement of 8.9 % in mean lap times when compared to the MPCC.

Previous incremental estimation methods consider estimating a single line, requiring as many observers as the number of lines to be mapped. This leads to the need for having at least 4N state variables, with N being the number of lines. This paper presents the first approach for multi-line incremental estimation. Since lines are common in structured environments, we aim to exploit that structure to reduce the state space. The modeling of structured environments proposed in this paper reduces the state space to 3N + 3 and is also less susceptible to singular configurations. An assumption the previous methods make is that the camera velocity is available at all times. However, the velocity is usually retrieved from odometry, which is noisy. With this in mind, we propose coupling the camera with an Inertial Measurement Unit (IMU) and an observer cascade. A first observer retrieves the scale of the linear velocity and a second observer for the lines mapping. The stability of the entire system is analyzed. The cascade is shown to be asymptotically stable and shown to converge in experiments with simulated data.

Humans tend to build environments with structure, which consists of mainly planar surfaces. From the intersection of planar surfaces arise straight lines. Lines have more degrees of freedom than points. Thus, line-based structure-from-motion (SfM) provides more information about the environment. In this article, we present solutions for SfM using lines, namely, incremental SfM. These approaches consist of designing state observers for a camera’s dynamical visual system looking at a 3-D line. We start by presenting a model that uses spherical coordinates for representing the line’s moment vector. We show that this parameterization has singularities, and, therefore, we introduce a more suitable model that considers the line’s moment and shortest viewing ray. Concerning the observers, we present two different methodologies. The first uses a memory-less state-of-the-art framework for dynamic visual systems. Since the previous states of the robotic agent are accessible—while performing the 3-D mapping of the environment—the second approach aims at exploiting the use of memory to improve the estimation accuracy and convergence speed. The two models and the two observers are evaluated in simulation and real data, where mobile and manipulator robots are used.

This paper presents an approach for multi-robot long-term planning under uncertainty over the duration of actions. The proposed methodology takes advantage of generalized stochastic Petri nets with rewards (GSPNR) to model multi-robot problems. A GSPNR allows for unified modeling of action selection, uncertainty on the duration of action execution, and for goal specification through the use of transition rewards and rewards per time unit. Our approach relies on the interpretation of the GSPNR model as an equivalent embedded Markov reward automaton (MRA). We then build on a state-of-the-art method to compute the long-run average reward over MRAs, extending it to enable the extraction of the optimal policy. We provide an empirical evaluation of the proposed approach on a simulated multi-robot monitoring problem, evaluating its performance and scalability. The results show that the synthesized policy outperforms a policy obtained from an infinite horizon discounted reward formulation as well as a carefully hand-crafted policy.

We consider the problem of communication planning for human-machine cooperation in stochastic and partially observable environments. Partially Observable Markov Decision Processes with Information Rewards (POMDPs-IR) form a powerful framework for information-gathering tasks in such environments. We propose an extension of the POMDP-IR model, called a Communicating POMDP-IR (com-POMDP-IR), that allows an agent to proactively plan its communication actions by using an approximation of the human's beliefs. We experimentally demonstrate the capability of our com-POMDPIR agent to limit its communication to relevant information and its robustness to lost messages.

Problem benchmarks are used in experimental science as a reference against which results of experiments using distinct approaches to solve the problem are compared and evaluated in relative terms. In Robotics, just formulating a general performance assessment problem is difficult per se, as robot systems are composed of very diverse subsystems (e.g., localisation, human-robot interaction, task planning, motion planning). This paper introduces a probabilistic approach to benchmarking and evaluating performance of robot systems, which uses probability theory as the common language to quantify the performance of distinct functionalities of a robot system and their impact on the performance of a task carried out by that system. The approach can be used to analyse the performance of a task plan from the performances if its composing functionalities, or to (re)plan when a performance degradation in functionality is predicted to cause performance degradation of the task plan beyond acceptable limits.

Social norms are the understandings that govern the behavior of members of a society. As such, they regulate communication, cooperation and other social interactions. Robots capable of reasoning about social norms are more likely to be recognized as an extension of our human society. However, norms stated in a form of the human language are inherently vague and abstract. This allows for applying norms in a variety of situations, but if the robots are to adhere to social norms, they must be capable of translating abstract norms to the robotic language. In this paper we use a notion of institution to realize social norms in real robotic systems. We illustrate our approach in a case study, where we translate abstract norms into concrete constraints on cooperative behaviors of humans and robots. We investigate the feasibility of our approach and quantitatively evaluate the performance of our framework in 30 real experiments with user-based evaluation with 40 participants.

This paper proposes a Decision-Theoretic approach to problems involving interaction between robot systems and human users, which takes into account the latent aspects of Human-Robot interaction, e.g., the user's status. The presented approach is based on the Partially Observable Markov Decision Process framework, which handles uncertainty in planning problems, extended with information rewards to optimize the information-gathering capabilities of the system. The approach is formalized into a framework which considers: observable and latent state variables; gesture and speech observations; and action factors which are related to the agent's actuators or to the information gain goals (Information-Reward actions). Under the proposed framework, the robot system is able to: actively gain information and react according to latent states, inherent to Human-Robot interaction settings; effectively achieve the goals of the task in which the robot is employed; and follow a socially appealing behavior. Finally, the framework was thoroughly tested in a socially assistive scenario, in a realistic apartment testbed and resorting to an autonomous mobile social robot. The experiments' results validate the proposed approach for problems involving robot systems in HumanRobot interaction scenarios.

In this paper, we present a unified approach for multi-robot cooperative simultaneous localization and object tracking based on particle filters. Our approach is scalable with respect to the number of robots in the team. We introduce a method that reduces, from an exponential to a linear growth, the space and computation time requirements with respect to the number of robots in order to maintain a given level of accuracy in the full-state estimation. Our method requires no increase in the number of particles with respect to the number of robots. However, in our method, each particle represents a full-state hypothesis, leading to the linear dependency on the number of robots of both space and time complexity. The derivation of the algorithm implementing our approach from a standard particle filter algorithm and its complexity analysis are presented. Through an extensive set of simulation experiments on a large number of randomized datasets, we demonstrate the correctness and efficacy of our approach. Through real robot experiments on a standardized open dataset of a team of four soccer-playing robots tracking a ball, we evaluate our method's estimation accuracy with respect to the ground truth values. Through comparisons with other methods based on 1) nonlinear least squares minimization and 2) joint extended Kalman filter, we further highlight our method's advantages. Finally, we also present a robustness test for our approach by evaluating it under scenarios of communication and vision failure in teammate robots.

This work proposes a novel relative localization system, based on active markers and an on-board camera, for tracking multiple quadrotors in a limited field of view. The system extracts the 3D poses of the markers including one that, by pulsating at a predefined frequency, provides an unique platform ID. We discuss how the camera field of view can be explored in presence of multiple targets, and what are the conditions on the system visibility that lead to the establishment of bidirectional sensing between robots with similar sensing capabilities. A visibility analysis is conducted to show that the developed relative localization system meets such requirements, and a closed-loop experiment is used to validate its performance under these conditions. Finally, its performance is compared with other results from the literature, and a metric is established with the intent of mapping different design solutions, facilitating design choices in presence of different requirements.

In this work we study the efficient search of objects in domestic environments, using probabilistic logic to represent uncertainty about object location and partially observable Markov decision processes (POMDP) for the decision-making process regarding the movements to be carried out by the robot to improve its belief about the object locations. We propose the use of a semantic map that stores information about the knowledge in the system and updates it, by an inference process, with sensor information received from the object recognition module. However, semantic maps are not capable of actively search for more information in the environment. For that reason a decision-making module, based on a POMDP framework, is integrated in the system. Several experiments were made in a realistic apartment test bed using every day objects and a mobile robot, showing that this hybrid solution makes the search process more efficient.

Multi-robot cooperative navigation in real-world environments is essential in many applications, including surveillance and search-and-rescue missions. State-of-the-art methods for cooperative navigation are often tested in ideal laboratory conditions and not ready to be deployed in real-world environments, which are often cluttered with static and dynamic obstacles. In this work, we explore a graph-based framework to achieve control of real robot formations moving in a world cluttered with a variety of obstacles by introducing a new distributed algorithm for reconfiguring the formation shape. We systematically validate the reconfiguration algorithm using three real robots in scenarios of increasing complexity.

Target tracking with bearing-only sensors is a challenging problem when the target moves dynamically in complex scenarios. Besides the partial observability of such sensors, they have limited field of views, occlusions can occur, etc. In those cases, cooperative approaches with multiple tracking robots are interesting, but the different sources of uncertain information need to be considered appropriately in order to achieve better estimates. Even though there exist probabilistic filters that can estimate the position of a target dealing with uncertainties, bearing-only measurements bring usually additional problems with initialization and data association. In this paper, we propose a multi-robot triangulation method with a dynamic baseline that can triangulate bearing-only measurements in a probabilistic manner to produce 3D observations. This method is combined with a decentralized stochastic filter and used to tackle those initialization and data association issues. The approach is validated with simulations and field experiments where a team of aerial and ground robots with cameras track a dynamic target.

In this paper we present a framework for the application of augmented reality to a mobile robot, using non-central camera systems. Considering a virtual object in the world with known local 3D coordinates, the goal is to project this object into the image of a non-central catadioptric imaging device. We propose a solution to this problem which allows us to project textured objects to the image in real-time (up to 20 fps): projection of 3D segments to the image; occlusions; and illumination. In addition, since we are considering that the imaging device is on a mobile robot, one needs to take into account the real-time localization of the robot. To the best of our knowledge this is the first time that this problem is addressed (all state-of-the-art methods are derived for central camera systems). To evaluate the proposed framework we test the solution using a mobile robot and a non-central catadioptric camera (using a spherical mirror).

Small teams of cooperating robots have been shown to benefit from the increased reliability of synchronised message exchanges provided by TDMA-based schemes. However, such schemes may also impose a long propagation delay to communications between non neighbour robots. Such negative impact is further increased by long TDMA rounds, which favour reduced medium utilisation and energy consumption, and by decentralised mesh topologies, which increase the team coverage and layout flexibility. For small size teams, say up to 10 units, it is feasible to set-up a global TDMA framework as well as tracking the instantaneous network topology, making it available to each robot. In this work we use this knowledge to forward packets along their path within each TDMA slot. To that end, we present a novel communication protocol that combines global TDMA and multi-hop routing. It maintains the reliability benefits of the TDMA schemes while strongly reducing the end-to-end propagation delay of interactions between non neighbour robots. We validate our protocol with simulation results using OMNET++. In a worst-case topology scenario we achieved an end-to-end delay that can be as low as 35% that of a traditional TDMA implementation that forwards the packets to the immediate one-hop neighbours, only. In a concrete audio streaming application scenario reported in the literature for an alternative real-time token-passing protocol, our proposal achieves similar delays with significantly less management bandwidth.

Partially observable Markov decision processes (POMDPs) provide a principled mathematical framework for modeling autonomous decision-making problems. A POMDP solution is often represented by a value function comprised of a set of vectors. In the case of factored models, the size of these vectors grows exponentially with the number of state factors, leading to scalability issues. We consider an approximate value function representation based on a linear combination of basis functions. In particular, we present a backup operator that can be used in any point-based POMDP solver. Furthermore, we show how under certain conditions independence between observation factors can be exploited for large computational gains. We experimentally verify our contributions and show that they have the potential to improve point-based methods in policy quality and solution size.

The engineering of fault-detection systems for multirobot systems (MRS) is a well-studied problem (e.g, Christensen et al. (2009)). Most fault-detection models are built on the assumption that normal behavior is known, and can be characterized in advance. The models are trained to recognize predefined normal behaviors, and behaviors not recognized are labeled abnormal. While such an approach does provide some interesting results of robust fault detection and fault tolerance, they may not be applicable when normal behavior can change as a result of unforeseen environmental conditions and online learning for instance. Furthermore, prior information required to characterize normal behaviors, may not always be available.

Localization is one of the key challenges that needs to be considered beforehand to design truly autonomous MAV teams. In this paper, we present a cooperative method to address the localization problem for a team of MAVs, where individuals obtain their position through perceiving a sound-emitting beacon MAV that is flying relative to a reference point in the environment. For this purpose, an on-board audio-based localization system is proposed that allows individuals to measure the relative bearing to the beacon robot and furthermore to localize themselves and the beacon robot simultaneously, without the need for a communication network. Our method is based on coherence testing among signals of a small on-board microphone array, to obtain the relative bearing measurements, and an estimator, to fuse these measurements with sensory information about the motion of the robot throughout time, to estimate robustly the MAV positions. The proposed method is evaluated both in simulation and in real world experiments.

The work described is part of a long term program of introducing institutional robotics, a novel framework for the coordination of robot teams that stems from institutional economics concepts. Under the framework, institutions are cumulative sets of persistent artificial modifications made to the environment or to the internal mechanisms of a subset of agents, thought to be functional for the collective order. In this article we introduce a formal model of institutional controllers based on Petri nets. We define executable Petri nets-an extension of Petri nets that takes into account robot actions and sensing-to design, program, and execute institutional controllers. We use a generalized stochastic Petri net view of the robot team controlled by the institutional controllers to model and analyze the stochastic performance of the resulting distributed robotic system. The ability of our formalism to replicate results obtained using other approaches is assessed through realistic simulations of up to 40 e-puck robots. In particular, we model a robot swarm and its institutional controller with the goal of maintaining wireless connectivity, and successfully compare our model predictions and simulation results with previously reported results, obtained by using finite state automaton models and controllers.

Markov Decision Processes (MDPs) provide an extensive theoretical background for problems of decision-making under uncertainty. In order to maintain computational tractability, however, real-world problems are typically discretized in states and actions as well as in time. Assuming synchronous state transitions and actions at fixed rates may result in models which are not strictly Markovian, or where agents are forced to idle between actions, losing their ability to react to sudden changes in the environment. In this work, we explore the application of Generalized Semi-Markov Decision Processes (GSMDPs) to a realistic multi-robot scenario. A case study will be presented in the domain of cooperative robotics, where real-time reactivity must be preserved, and synchronous discrete-time approaches are therefore sub-optimal. This case study is tested on a team of real robots, and also in realistic simulation. By allowing asynchronous events to be modeled over continuous time, the GSMDP approach is shown to provide greater solution quality than its discrete-time counterparts, while still being approximately solvable by existing methods.

Maximizing the performance of cooperative perception of a tracked target by a team of mobile robots while maintaining the team's formation is the core problem addressed in this work. We propose a solution by integrating the controller and the estimator modules in a formation control loop. The controller module is a distributed non-linear model predictive controller and the estimator module is based on a particle filter for cooperative target tracking. A formal description of the integration followed by simulation and real robot results on two different teams of homogeneous robots are presented. The results highlight how our method successfully enables a team of homogeneous robots to minimize the total uncertainty of the tracked target's cooperative estimate while complying with the performance criteria such as keeping a pre-set distance between the team-mates and/or the target and obstacle avoidance.

In this paper we address the problem of cooperative localization and target tracking with a team of moving robots. We model the problem as a least squares minimization problem and show that this problem can be efficiently solved using sparse optimization methods. To achieve this, we represent the problem as a graph, where the nodes are robot and target poses at individual time-steps and the edges are their relative measurements. Static landmarks at known position are used to define a common reference frame for the robots and the targets. In this way, we mitigate the risk of using measurements and state estimates more than once, since all the relative measurements are i.i.d. and no marginalization is performed. Experiments performed using a set of real robots show higher accuracy compared to a Kalman filter.

This work is developed in the framework of Institutional Robotics (IR), an approach to cooperative distributed robotic systems that draws inspiration from the social sciences. We consider a case study concerned with a swarm of simple robots which has to maintain wireless connectivity and a certain degree of spatial compactness. Robots have local, bounded communication capabilities and have to execute the task (running an IR controller) using exclusively as information their current number of wireless connections to neighbors. For the very same case study, we previously introduced an IR-based macroscopic model for the behavior of a large number of robots, validated using a submicroscopic model implemented through a realistic simulator. In this work, we go a step further and validate our submicroscopic model with real world experiments, duplicating accurately the conditions used, including a large number of robots and noisy communication channels. The main conclusions of this paper are two-fold. First, the IR approach was able to maintain the wireless connectivity of a swarm of 40 real, resource-constrained robots. This speaks in favor of the robustness and scalability of such approach. Second, the submicroscopic model implemented is faithfully capturing the reality and can be used to further optimize the performances of distributed control strategies using an IR approach.

Top-down causation has been suggested to occur at all scales of biological organization as a mechanism for explaining the hierarchy of structure and causation in living systems. Here we propose that a transition from bottom-up to top-down causation -- mediated by a reversal in the flow of information from lower to higher levels of organization, to that from higher to lower levels of organization -- is a driving force for most major evolutionary transitions. We suggest that many major evolutionary transitions might therefore be marked by a transition in causal structure. We use logistic growth as a toy model for demonstrating how such a transition can drive the emergence of collective behavior in replicative systems. We then outline how this scenario may have played out in those major evolutionary transitions in which new, higher levels of organization emerged, and propose possible methods via which our hypothesis might be tested.

In search and rescue missions, Micro Air Vehicles (MAV's) can assist rescuers to faster locate victims inside a large search area and to coordinate their efforts. Acoustic signals play an important role in outdoor rescue operations. Emergency whistles, as found on most aircraft life vests, are commonly carried by people engaging in outdoor activities, and are also used by rescue teams, as they allow to signal reliably over long distances and far beyond visibility. For a MAV involved in such missions, the ability to locate the source of a distress sound signal, such as an emergency whistle blown by a person in need of help, is therefore significantly important and would allow the localization of victims and rescuers during night time, through foliage and in adverse conditions such as dust, fog and smoke. In this paper we present a sound source localization system for a MAV to locate narrowband sound sources on the ground, such as the sound of a whistle or personal alarm siren. We propose a method based on a particle filter to combine information from the cross correlation between signals of four spatially separated microphones mounted on the MAV, the dynamics of the aerial platform, and the doppler shift in frequency of the sound due to the motion of the MAV. Furthermore, we evaluate our proposed method in a real world experiment where a flying micro air vehicle is used to locate and track the position of a narrowband sound source on the ground.

We present a decentralized methodology to control multi-robot systems, where each robot behaviour is modelled as a Petri net (PN) and a set of coordination rules between the robots is given as linear temporal logic (LTL) formulas describing safety properties for the system. The LTL formulas are used to define the events and changes in state that must be communicated between robots and to augment the individual PN model of each robot so that it can handle the incoming communications. These augmented PNs are then used, in conjunction with the LTL formulas, to build PN realizations of local supervisors, based on discrete event system theory, that enforce the LTL specifications by construction. The methodology is illustrated through a simulated application example.

Factored Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) form a powerful framework for multiagent planning under uncertainty, but optimal solutions require a rigid history-based policy representation. In this paper we allow inter-agent communication which turns the problem in a centralized Multiagent POMDP (MPOMDP). We map belief distributions over state factors to an agent's local actions by exploiting structure in the joint MPOMDP policy. The key point is that when sparse dependencies between the agents' decisions exist, often the belief over its local state factors is sufficient for an agent to unequivocally identify the optimal action, and communication can be avoided. We formalize these notions by casting the problem into convex optimization form, and present experimental results illustrating the savings in communication that we can obtain.

In this work we focus on the problem of probabilistic sensor fusion in Multi-Robot Multi-Sensor Systems (MRMS), taking into account that some sensors might fail or produce erroneous information. We study fusion methods that can successfully cope with situations of agreement, partial agreement, and disagreement between sensors. We define a set of specifications for fusion methods appropriate for MRMS environments. In light of these specifications, we review two popular algorithms for probabilistic sensor fusion, Linear Opinion Pool (LOP) and Logarithmic Opinion Pool (LGP). To overcome difficulties of applying them to a MRMS setting, a new method is introduced, p-norm Opinion Pool (POP). Comparing to LOP and LGP, POP is more compatible with the specifications and more flexible, successfully handling situations of agreement and disagreement between sensors. Through simulation and real-world experiments, we check performance of the POP and compare it with LOP and LGP. We also implement a real-world experiment through which the performance of POP is examined.

Network robot systems (NRS) provide many scientific and technological challenges, given that robots interact with each other as well as with sensors present in the environment to accomplish certain tasks. In this work, we consider an essential problem in NRS, namely how to perform task planning given the limitations both in on-board sensing as well as in the environment's sensors. Partially observable Markov decisions processes (POMDPs) form an attractive framework to address planning in the uncertain environments that typify NRS. We show how to model a typical cooperative perception task in a NRS, namely tracking and classifying people, and we present experiments that show how the proposed approach results in an effective interplay between robot and environment sensors.

This paper introduces a testbed for sensor and robot network systems, currently composed of 10 cameras and 5 mobile wheeled robots equipped with several sensors for self-localization, obstacle avoidance and vision cameras, and wireless communications. The testbed includes a service-oriented middleware to enable fast prototyping and implementation of algorithms previously tested in simulation, as well as to simplify integration of subsystems developed by different partners. We survey an integrated approach to human-robot interaction that has been developed supported by the testbed under an European research project. The application integrates innovative methods and algorithms for people tracking and waving detection, cooperative perception among static and mobile cameras to improve people tracking accuracy, as well as decision-theoretical approaches to sensor selection and task allocation within the sensor network.

We consider the problem of sensor-aware path planning for a robot in a networked robot system, in particular in urban environments equipped with a network of surveillance cameras. A robot can use observations from the camera network to improve its own localization performance, but also needs to take into account the specifics of its local sensors. We model our problem in the Markov decision process framework, which forms a natural way to express concurrent and possibly conflicting objectives - such as reaching a goal quickly, keeping the robot localized, keeping the target in sight - each with their own priority. We show how we can successfully prioritize the different objectives in a flexible way by changing the reward function, based on the sensory needs of the system.

This paper introduces Petri net based models of robotic tasks, which can be used to analyse and synthesise task plans, taking into account a Petri net model that abstracts the relevant features from the robot environment as well. Logical analysis concerning deadlocks and resource conservation can be performed over the ordinary version of the model. A task plan modeled by a Petri net can be extracted from the generalised stochastic version of the model, representing the optimal plan given a probabilistic measure of uncertainty associated to the effects of its composing actions. The Petri net representing the model is suitable for being ran directly within the code, as well as for plan monitoring during execution time. Simulation results illustrating the methodology are presented for a robotic soccer scenario.

In this work we overview two popular algorithms for sensor fusion, linear opinion pool (LOP) and logarithmic opinion pool (LGP) and introduce a new method to overcome their difficulties: cooperative opinion pool (COP). COP considers all of the dependencies between observations such as LOP and reduces the uncertainty such as LGP. We check its performance on a simulated multi-robot environment, where a group of robots cooperate to reduce uncertainty of self- localization and object localization. Simulation results show that the entropy of cooperative localization is reduced as the number of cooperating robots grows.

Reinforcement learning addresses the question of programming an autonomous agent to execute tasks that are described as reinforcement functions. Then, the agent is responsible for discovering the best actions to fulfil such task. Most of the work on reinforcement learning considers that reinforcements are given by the environment, not addressing the problem of how to describe tasks as reinforcement functions. Preference elicitation addresses the problem of describing a human preference through utility functions, from which reinforcement functions are special cases. This paper proposes an approach where preference elicitation and reinforcement learning are handled in an integrated manner, providing an autonomous method of programming an agent. The agent is programmed through pairwise evaluations over observed behaviours of the agent, where the evaluations are summarised in the reinforcement function. In this paper we present an approach to solve such a problem based on evaluations over observed behaviours. We propose a new algorithm, PEOB-RS, that can be shown to converge towards an optimal policy, providing the number of trials for each behaviour tends to infinity. Experimental results from learning in a grid stochastic environment are used to obtain a reinforcement function, illustrating the effectiveness of PEOB-RS, even if requiring too many evaluations. Such reinforcement function is then transferred to a more real-like environment simulating a pioneer robot, showing the abstraction property of utility functions.

We present a model-based 3D tracking system, using wide angle perspective catadioptric sensors. These sensors acquire 360deg views of the environment and the projection from 3D world points to the image plane is approximated by a perspective model. This is a major advantage in structured environments because straight lines on specific surfaces are not deformed by the sensor, allowing the application of standard computer vision algorithms. Objects off the surface are distorted according to a complex projection model, but can be approximated by a simple wide angle perspective mapping. This is exploited here to develop a robust tracking system for autonomous robots using a 3D shape and color-based object model. The use of particle filters allows tracking to be done with 3D realistic motion models and tackling object occlusion, overlap and ambiguities. We show that the use of the perspective model is advantageous over more standard catadioptric projection models, since it renders a very good approximation to the true model, being simpler and more efficient to use, in particular with 3D particle filtering methods.

Reinforcement learning (RL) is a method that helps programming an autonomous agent through human-like objectives as reinforcements, where the agent is responsible for discovering the best actions to fulfil the objectives. Nevertheless, it is not easy to disentangle human objectives in reinforcement like objectives. Inverse reinforcement learning (IRL) determines the reinforcements that a given agent behaviour is fulfilling from the observation of the desired behaviour. In this paper we present a variant of IRL, which is called IRL with evaluation (IRLE) where instead of observing the desired agent behaviour, the relative evaluation between different behaviours is known by the access to an evaluator. We present also a solution for this problem under the assumption that a relative linear function that preserves the order assumed by the evaluator exists and that the evaluator evaluates policies instead of behaviours. This is posed as a linear feasibility problem, whose solution is well known. Results of simulations of a set of heterogeneous robots in a search and rescue scenario are presented to illustrate the method and the possibility to transfer the learned reinforcement function among robots

This work describes a semi-autonomous robot for rescue operations, nicknamed RAPOSA (FOX in English). The robot was designed and built to operate in outdoor environments hostile to the human presence, such as debris resulting from the collapse of built structures, and is targeted to the tele-operated detection of potential survivors using a set of specific sensors whose information is transmitted to a remote human operator. RAPOSA's mechanical structure is composed of a main body and a front body, whose locomotion is supported on tracked wheels, allowing motion even when the robot is upside down. The front body has variable tilting capabilities, providing means to overcome edges higher than the robot main body (e.g., when climbing a stair) and is also useful to grab the lower ground when only the main body has ground contact. This front body has one thermal camera and two web cameras installed. Additional sensors include gas, temperature and humidity sensors, Web cams, light diodes, microphone and loudspeaker. The robot uses wireless communications, with an option for tethered operation. The tether carries both power and communications, with an access point on its end, and can also be used to suspend the robot inside a deep hole. Docking and undocking the robot to the tether is accomplished remotely by the operator with the help of a camera located inside the robot, and represents the most innovative feature of RAPOSA

This paper describes an approach to the modeling and control of multiagent populations composed of a large number of agents. The complexity of population modeling is avoided by assuming a stochastic approach, under which the agent distribution over the state space is modeled. The dynamics of the state probability density functions is determined, and a control problem of maximizing the probability of robotic presence in a given region is introduced. The Minimum Principle for the optimal control of partial differential equations is exploited to solve this problem, and it is applied to the mission control of a simulated large robotic population

Formations of multi-agent systems, such as mobile robots, satellites and aircraft, require individual agents to satisfy their kinematic equations while constantly maintaining interagent constraints. In this paper, we develop a systematic framework for studying formation motion feasibility of multi-agent systems. In particular, we consider formations wherein all the agents cooperate to enforce the formation. We determine algebraic conditions that guarantee formation feasibility given the individual agent kinematics. Our framework also enables us to obtain lower dimensional control systems describing the group kinematics while maintaining all formation constraints.

Introduces a robotic task model (RTM) based on Petri nets, that establishes a framework for task evaluation from qualitative and quantitative viewpoints, as well as a methodology for the implementation of robotic task coordination. A testbed for the evaluation of the RTM and the details of its implementation over a network of distributed task executors is described.

In this paper, a method for robot self-localization based on a catadioptric omni-directional sensor is introduced. The method uses natural geometric landmarks of the environment. It is assumed that the robot moves on flat surfaces and straight lines can be identified in the surrounding environment image acquired by the catadioptric system. This omni-directional vision system is based on a camera plus a convex mirror designed to obtain (by hardware) the ground plane bird's eye view. Results from the application to a real robot moving on RoboCup soccer field and concerning the method's accuracy are presented.

This paper introduces a design methodology for intelligent controllers, based on a hierarchical linguistic model of command translation by tasks-primitive tasks-primitive actions, and on a two-stage hierarchical learning stochastic automaton that models the translation interfaces of a three-level hierarchical intelligent controller. The methodology relies on the designer's a priori knowledge on how to implement by primitive actions the different primitive tasks which define the intelligent controller. A cost function applicable to any primitive task is introduced and used to learn on-line the optimal choices from the corresponding predesigned sets of primitive actions. The same concept applies to the optimal tasks for each command, whose choice is based on conflict sets of stochastic grammar productions. Optional designs can be compared using this performance measure. A particular design evolves towards the command translation (by tasks-primitive tasks-primitive actions) that minimizes the cost function.

Petri nets have been widely used to model dynamic systems, namely manufacturing systems. In this paper we introduce the use of Petri nets to model robotic tasks. Different views of the robotic task model can be modeled by distinct Petri net types: interpreted Petri nets for task design and execution, generalized stochastic Petri nets for task quantitative performance evaluation and ordinary Petri nets for task qualitative performance evaluation. Quantitative performance evaluation and improvement based on reinforcement learning from feedback are detailed in the paper. Examples of applications to visual servoing and catching of moving objects by a robotic arm and to mobile robot tasks are presented.

The SocRob project was born as a challenge for multidisciplinary research on broad and generic approaches for the design of a cooperating society of robots, involving control, robotics and artificial intelligence researchers. In this paper we introduce some of the hardware options already taken by the group in the design of a robotic soccer team, chosen as our first case study. Each robot of the population is endowed with several sensors. The most important of them is vision. The others are linked to the main processing unit (a Pentium motherboard) by an i2c bus. Conceptual issues regarding the functional architecture of the team are also discussed. We propose a 3-level architecture, consisting of a set of context-switchable behaviors, each of them resulting of the composition of low-level task primitives.

A methodology for performance improvement of intelligent machines based on hierarchical reinforcement learning is introduced. Machine decision making and learning are based on a cost function which includes reliability and a computational cost of algorithms at the three levels of the hierarchy proposed by Saridis. Despite this particular formalization, the methodology intends to be sufficiently general to encompass different types of architectures and applications. Novel contributions of this work include the definition of a cost function combining reliability and complexity, recursively improved through feedback, a hierarchical reinforcement learning and decision making algorithm which uses that cost function, and a methodology supported on information-based complexity for joint measure of algorithm cost and reliability. Results of simulations show the application of the formalism to intelligent robotic systems.>