Adjustable Autonomy is gaining interest as it alleviates robot management costs, which often restrain non-routine applications. Whereas it seems straightforward to account for the availability of helpers when making plans that involve being granted for support in the future, no existing research covers this issue. As a solution, we formalize the first human-centric model that accounts for operator support dynamics when generating adjustable-autonomy plans. We formalize Restricted Autonomy Levels (RAL) within a Markov-based framework for representing when and what level of support the robot should ask for. This model is combined with a formalization of usual aspects of man-machine collaboration: operator availability, risk of denial and withdrawal, effect of teleoperation, risks and consequences for violating RAL restrictions and backup procedures, should violations occur. We empirically demonstrate, through a detailed example and the deployment on a professional-grade security robot, that the generated plans deeply combine the problem-solving activities of the robot with the management of requested human support, leading to improved performance and decreased operator effort. We also analyse the computational costs of computing policies that ensure a zero-chance of RAL violation.

This paper introduces Advice-MDPs, an expansion of Markov Decision Processes for generating policies that take into consideration advising on the desirability, undesirability, and prohibition of certain states and actions. AdviceMDPs enable the design of designing semi-autonomous systems (systems that require operator support for at least handling certain situations) that can efficiently handle unexpected complex environments. Operators, through advising, can augment the planning model for covering unexpected real-world irregularities. This advising can swiftly augment the degree of autonomy of the system, so it can work without subsequent human intervention. This paper details the Advice-MDP formalism, a fast AdviceMDP resolution algorithm, and its applicability for real-world tasks, via the design of a professional-class semi-autonomous robot system ready to be deployed in a wide range of unexpected environments and capable of efficiently integrating operator advising.

Most of existing coordination techniques for autonomous agents assume the knowledge or the estimation of the other agents' policy. However, this assumption is not valid in semi-autonomous agents because an external entity can take the control and modify the behavior of the agent. We face this problem in applications where an operator can take the control of the system (Robot/UAV). Many human factors may affect this behavior, such as stress, hesitations and preferences. Estimating the policy in such contexts is a difficult problem. Many existing algorithms using Inverse Reinforcement learning or imitation have been developed. However most of them have weak performance when non-optimal policy is followed. In this paper, we investigate techniques for estimating the followed policies of semi-autonomous agents that could be nonoptimal due to critical situations We extend some prediction methods and algorithms based on Factored MDPs and Inverse Reinforcement Learning to improve their stability and their efficiency during the execution of a mission. Then, we develop various experiments showing the performance on efficiency and stability of our approach in different conditions and comparing with it.

Recent works on multi-agent sequential decision making using decentralized partially observable Markov decision processes have been concerned with interaction-oriented resolution techniques and provide promising results. These techniques take advantage of local interactions and coordination. In this paper, we propose an approach based on an interaction-oriented resolution of decentralized decision makers. To this end, distributed value functions (DVF) have been used by decoupling the multi-agent problem into a set of individual agent problems. However existing DVF techniques assume permanent and free communication between the agents. In this paper, we extend the DVF methodology to address full local observability, limited share of information and communication breaks. We apply our new DVF in a real-world application consisting of multi-robot exploration where each robot computes locally a strategy that minimizes the interactions between the robots and maximizes the space coverage of the team even under communication constraints. Our technique has been implemented and evaluated in simulation and in real-world scenarios during a robotic challenge for the exploration and mapping of an unknown environment. Experimental results from real-world scenarios and from the challenge are given where our system was vice-champion.

This paper addresses the problem of exploring an unknown area with a team of autonomous robots using decentralized decision making techniques. The localization aspect is not considered and it is assumed the robots share their positions and have access to a map updated with all explored areas. A key problem is then the coordination of decentralized decision processes: each individual robot must choose appropriate exploration goals so that the team simultaneously explores different locations of the environment. We formalize this problem as a Decentralized Markov Decision Process (Dec-MDP) solved as a set of individual MDPs, where interactions between MDPs are considered in a distributed value function. Thus each robot computes locally a strategy that minimizes the interactions between the robots and maximizes the space coverage of the team. Our technique has been implemented and evaluated in real-world and simulated experiments.

Cooperative control can help overcome the limitations of autonomous systems (AS) by introducing a supervision unit (SU) (human or another system) into the control loop and creating adjustable autonomy. We present a decision-theoretic approach to accomplish this using Mixed Markov Decision Processes (MI-MDPs). The solution is an optimal plan that tells the AS what actions to perform as well as when to request SU attention or transfer control to the SU. This provides a varying degree of autonomy, particularly suitable for robots exploring a domain with regions that are too complex or risky for autonomous operation, or intelligent vehicles operating in heavy traffic.

We present a new framework for controlling a robot collaborating with a human to accomplish a common mission. Knowing that we are interested in collaboration domains where there is no shared plan between the human and the robot, the constraints on the decision process are more challenging. We study the decision process of a robot agent for a specific shared mission with a human considering the effect of the human presence, the planning flexibility according to human comfortability and achieving mission. We choose to formalize this problem with Partially Observable Markov Decision Process, then we describe a new domain example that represent human-robot collaboration with no shared plan and we show some preliminary results of solving the POMDP model with standard optimal algorithms as a base work to compare with state-of-the-art and future-work approximate algorithms.

A considerable amount of attention has been paid to the coalition formation problem to deal efficiently with tasks needing more than one agent. However, little attention has been paid to the problem of monitoring a coalition during the execution by modifying it according to the progress of the accomplishment of the task. In this paper, we consider a coalition of resource-bounded autonomous agents with anytime behavior solving a common complex task. There is no central control component. Agents can observe the effect of the other agents' actions. They can decide whether they should continue to contribute in solving the common task or to stop their contribution and to leave the coalition. This decision is made in a distributed way. The objective is to avoid the waste of resources and time by using the same coalition along the task accomplishment while some agents become not necessary to pursue the accomplishment of the task. We formalize this decentralized decision making problem as a DEC-MDP.

In this paper, we consider the problem of adapting the model of a diagnosis-helping module, which interacts with human experts. The approach consists of enforcing strong semantics in the model, so that this interaction may be as intuitive as possible. When learning the model, the problem consists in respecting these semantics while learning with few data. We addressed this problem through a semisupervised gradient descent algorithm applied to partially observable Markov models with fuzzy observations. This method optimizes several criteria at once, guiding the search to a compromise between the expert's directives and objective evaluations. This method has been successfully applied to a tele-medicine application where the system monitors dialyzed patients and alerts nephrologists.