Goal-driven autonomy is an agent model for managing a dynamic environment by reasoning about current and potential goals while planning and acting. Since unexpected events and conditions may cause an agent’s goals and plans to become invalid or infeasible, an agent with goal-driven autonomy should monitor the environment against its expectations. Designed for dynamic, open, and partially observable environments, such an agent can create new goals or change its existing goals as needed. We present a formalisation of expectations for agents operating in these kinds of environments. Our formalisation includes situations where agents have the capability to sense the environment with some associated costs. We examine agent choices and behaviour in these domains and evaluate multiple approaches for selecting a subset of the agent’s sensing actions to execute. The contributions of this work are (1) a specification of different approaches to generating expectations; (2) a formalisation of the autonomy problem that minimises sensing costs; (3) a complexity analysis of the problem; (4) new algorithms for deciding which sensing actions to perform; and (5) empirical results demonstrating the benefit and cost of these approaches.

Cognitive agents operating in complex and dynamic domains benefit from significant goal management. Operations on goals include formulation, selection, change, monitoring and delegation in addition to goal achievement. Here we model these operations as transformations on goals. An agent may observe events that affect the agent’s ability to achieve its goals. Hence goal transformations allow unachievable goals to be converted into similar achievable goals. This paper examines an implementation of goal change within a cognitive architecture. We introduce goal transformation at the metacognitive level as well as goal transformation in an automated planner and discuss the costs and benefits of each approach. We evaluate goal change in the MIDCA architecture using a resource-restricted planning domain, demonstrating a performance benefit due to goal operations.

We present a metacognitive, integrated, dual-cycle architecture whose function is to provide agents with a greater capacity for acting robustly in a dynamic environment and managing unexpected events. We present MIDCA 1.3, an implementation of this architecture which explores a novel approach to goal generation, planning and execution given surprising situations. We formally define the mechanism and report empirical results from this goal generation algorithm. Finally, we describe the similarity between its choices at the cognitive level with those at the metacognitive.

In this paper, we review recent advances in Reinforcement Learning (RL) in light of potential applications to robotics, introduce the basic concepts of RL and Markov Decision Process (MDP), and compare different RL algorithms such as Q-learning, Temporal Difference learning, the Actor Critic, and the Natural Actor Critic. We conclude that policy gradient methods are more suitable for solving continuous state/action MDP problems than RL with lookup tables or general function approximators. Further, natural policy gradient methods can efficiently converge to locally optimal solutions. Some simulation results are given to support our arguments. We also present a brief overview of our approach to developing an autonomous robot agent that can perceive, learn from and interact with the environment, and reason about and handle unexpected problems using its knowledge base.

Metacognitive architectures provide one solution to the brittleness problem for agents operating in complex, changing environments. The Metacognitive Loop, in which a system notes an anomaly, assesses the problem and guides a solution, is one form of such an architecture. This paper extends prior work on implementing the note phase of this process in symbolic planning domains using the A-distance. This extension uses a growing neural gas algorithm to construct a network which represents various normal and anomalous states. Testing shows that this technique allows for improved detection of anomalies in the note phase as well as categorization of anomalies by severity and type in the assess phase.

An intelligent multiagent system includes a number of software agents interacting to solve a problem. Various agents are responsible for planning different tasks. These tasks must be coordinated to solve complex problems. A number of reasons exist for which coordination among the agents is necessary, and numerous issues have to be tackled to achieve efficient coordination. The focus of this paper is resource coordination. When an agent lacks a resource to achieve a goal, it may choose one of two options. It can acquire another resource, or it can change the goal so that it can achieve it with the resource at hand. We present a cost-benefit analysis that can be performed to determine if the requesting agent receives the requested resource. We present empirical results that show the relative goal satisfaction achieved in the PRODIGY planner with and without resource coordination. We have implemented a multiagent system called COMAS that implements resource coordination.