We present a metacognitive, integrated, dual-cycle architecture whose function is to provide agents with a greater capacity for acting robustly in a dynamic environment and managing unexpected events. We present MIDCA 1.3, an implementation of this architecture which explores a novel approach to goal generation, planning and execution given surprising situations. We formally define the mechanism and report empirical results from this goal generation algorithm. Finally, we describe the similarity between its choices at the cognitive level with those at the metacognitive.

Reasoning does not work well when done in isolation from its significance, both to the needs and interests of an agent and with respect to the wider world. Moreover, those issues may best be handled with a new sort of data structure that goes beyond the knowledge base and incorporates aspects of perceptual knowledge and even more, in which a kind of anticipatory action may be key.

In this paper, we review recent advances in Reinforcement Learning (RL) in light of potential applications to robotics, introduce the basic concepts of RL and Markov Decision Process (MDP), and compare different RL algorithms such as Q-learning, Temporal Difference learning, the Actor Critic, and the Natural Actor Critic. We conclude that policy gradient methods are more suitable for solving continuous state/action MDP problems than RL with lookup tables or general function approximators. Further, natural policy gradient methods can efficiently converge to locally optimal solutions. Some simulation results are given to support our arguments. We also present a brief overview of our approach to developing an autonomous robot agent that can perceive, learn from and interact with the environment, and reason about and handle unexpected problems using its knowledge base.

Metacognitive architectures provide one solution to the brittleness problem for agents operating in complex, changing environments. The Metacognitive Loop, in which a system notes an anomaly, assesses the problem and guides a solution, is one form of such an architecture. This paper extends prior work on implementing the note phase of this process in symbolic planning domains using the A-distance. This extension uses a growing neural gas algorithm to construct a network which represents various normal and anomalous states. Testing shows that this technique allows for improved detection of anomalies in the note phase as well as categorization of anomalies by severity and type in the assess phase.

Maintaining adequate performance in dynamic and uncertain settings has been a perennial stumbling block for intelligent systems. Nevertheless, any system intended for real-world deployment must be able to accommodate unexpected change—that is, it must be perturbation tolerant. We have found that metacognitive monitoring and control—the ability of a system to self-monitor its own decision-making processes and ongoing performance, and to make targeted changes to its beliefs and action-determining components—can play an important role in helping intelligent systems cope with the perturbations that are the inevitable result of real-world deployment. In this article we present the results of several experiments demonstrating the efficacy of metacognition in improving the perturbation tolerance of reinforcement learners, and discuss a general theory of metacognitive monitoring and control, in a form we call the metacognitive loop. ||This research is supported in part by the AFOSR and ONR.

This essay describes a general approach to building perturbation-tolerant autonomous systems, based on the conviction that artificial agents should be able to notice when something is amiss, assess the anomaly, and guide a solution into place. This basic strategy of self-guided learning is termed the metacognitive loop; it involves system monitoring, reasoning about, and, when necessary, altering its own decision-making components. This paper (a) argues that equipping agents with a metacognitive loop can help to overcome the brittleness problem, (b) details the metacognitive loop and its relation to ongoing work on time-sensitive commonsense reasoning, (c) describes specific, implemented systems whose perturbation tolerance was improved by adding a metacognitive loop, and (d) outlines both short-term and long-term research agendas.

Presuppositionis a pervasive feature of human language. It involves many interesting interactions between the utterances of a discourse and the contextof the discourse. In this paper we focus on issues of logical form connected with the interaction of presupposition and discourse context, and illustrate our theory with some implementational work using the active logicframework. After reviewing some of the major issues in presupposition theory we turn to a largely successful unified approach of Heim. We show how the main principles of this theory can be implemented in active logic. But we also find two serious difficulties. These consist in (a) a straightforward counterexample and (b) a type of discourse that we call a garden‐path discourse. We maintain that both the counterexample and the garden‐path type of discourse can be handled by our active‐logic version of Heim's theory. This requires us to reformulate and extend Heim's theorey. Although this work is largely theoretical, both Heim's theory and ours have important things to say about the incremental processing of the utterances that make up discourse. And we present our theory as a specification of a processing device that takes logical form of a sentence along with current discourse context as input and delivers an updated discourse context as output. As an experiment, we have implemented portions of this device.

In planning situations involving tight deadlines, a commonsense reasoner may spend a substantial amount of the available time in reasoning towards and about the formulation of the (partial) plan. This reasoning involves, but is not limited to (partial) plan formulation, asking decisions about available and conceivable alternatives, plan sequecing, and also plan failure and revision. However, the time taken in reasoning about a plan brings the deadline closer. The reasoner should therefore take account of the passage of time during that same reasoning, and this accounting must continously affect every decision under time-pressure. Step-logics were introduced as a mechanism for reasoning situated in time. We employ an extension of them here, called ‘active logics’, to create a logic-based planner that lets a time-situated reasoner keep track of a approaching deadline as he/she makes (and enacts) his/her plan, thereby treatilng all facets of planning (including plan-formation and its simultaneous of subsequent execution) as deadline-coupled. While an agent under severe time-pressure may spend a substantial amount of the available time in reasoning towards and about a plan of action, in a realistic setting the same agent must also measure up to two other crucial resource limitations as well, namely space and computation bounds. We address these concerns and offer some solution by introducing a limited short-term memory combined with a primitive relevance mechanism, and a limited-capacity inference engine. We propose heuristics to maximize an agent's chances of meeting a dedline within these additional realistic constraints. We give examples from commonsense planning, including ones we have solved and implemented in Prolog.

Reason-based actions plunge the reasoner into temporal considerations from all angles. We see this not only when time enters explicitly into the problem statement, but also in formal robot blocks-world scenarios, in the Yale Shooting Problem, and other associated versions of the frame problem (e.g., Hanks and McDermott1), in various specialized actions (e.g., hiding, as in Allen2), and so on. In short, where there is action, there is time, and where there is time, there is a potential need for reasoning about time. Where, then, is the action? It certainly includes the usual overt physical acts of motion, and also certain covert behaviors such as hiding or watching. In these, of course, time is important. But there is another angle that is not usually noted, one that we have been exploring for the past several years3–6. Namely, action also occurs in the form of mere thinking or reasoning. Moreover, the very same temporal considerations apply to this reasoning behavior. This leads us to view reasoning itself as a kind of action, with the obvious yet nontrivial consequence that our reasoning goes on “as the world turns.” the present article offers various arguments in support of this position.

The needs of a real-time reasoner situated in an environment may make it appropriate to view error-correction and non-monotonicity as much the same thing. This has led us to formulate situated (or step) logic, an approach to reasoning in which the formalism has a kind of real-time self-reference that affects the course of deduction itself. Here we seek to motivate this as a useful vehicle for exploring certain issues in commonsense reasoning. In particular, a chief drawback of more traditional logics is avoided: from a contradiction we do not have all wffs swamping the (growing) conclusion set. Rather, we seek potentially inconsistent, but nevertheless useful, logics where the real-time self-referential feature allows a direct contradiction to be spotted and corrective action taken, as part of the same system of reasoning. Some specific inference mechanisms for real-time default reasoning are suggested, notably a form of introspection relevant to default reasoning. Special treatment of ‘now’ and of contradictions are the main technical devices here. We illustrate this with a computer-implemented real time solution to R. Moore's Brother Problem.

During incremental concept learning from examples, tentative hypotheses are formed and then modified to form new hypotheses. When there is a choice among hypotheses, bias is used to express a preference. Bias may be expressed by the choice of hypothesis language, it may be implemented as an evaluation function for selecting among hypotheses already generated, or it may consist of screening potential hypotheses prior to hypothesis generation. This paper describes the use of the third method. Bias is represented explicitly both as assumptions that reduce the space of potential hypotheses and as procedures for testing these assumptions. There are advantages gained by using explicit assumptions. One advantage is that the assumptions are meta‐level hypotheses that are used to generate future, as well as to select between current, inductive hypotheses. By testing these meta‐level hypotheses, a system gains the power to anticipate the form of future hypotheses. Furthermore, rigorous testing of these meta‐level hypotheses before using them to generate inductive hypotheses avoids consistency checks of the inductive hypotheses. A second advantage of using explicit assumptions is that bias can be tested using a variety of learning methods.

We discuss some points on which we agree, and others on which we disagree, with Cheeseman's target article. Particular agreements include the need for an eclectic approach; disagreements include the misleading distinction between probabilistic and logical reasoning regarding the notion of truth, and also some matters of nonmonotonicity.

We have suggested that a new kind of logical study that focuses on individual deductive steps is appropriate to agents that must do commonsense reasoning. In order to adequately study such reasoners, a formal description of such “steps” is necessary. Here we carry further this program for the propositional case. In particular we give a result on completeness for reasoning about agents.

Default reasoning is analyzed as consisting (implicitly) of at least three further aspects–oracles, jumps, and fixes‐which in turn are related to the notion of a belief. Beliefs are then discussed in terms of their use in a reasoning agent. Next an idea of Israel is embellished to show that certain desiderata regarding these aspects of default reasoning lead to inconsistent belief sets, and that as a consequence the handling of inconsistencies must be taken as central to commonsense reasoning. Finally, these results are applied to standard cases of default reasoning formalisms in the literature (circumscription, default logic, and nonmonotonic logic), where it turns out that even weaker hypotheses lead to failure to achieve commonsense default conclusions.

A technique for allowing numerical formulae in data is applied to the problem of facilitating the writing of a program to calculate definite integrals of functions input during execution without editing the program.

Ackermann's set theory [1], called here A, involves a schema where φ is an ∈-formula with free variables among y1, …, yn and w does not appear in φ. Variables are thought of as ranging over classes and V is intended as the class of all sets. S is a kind of comprehension principle, perhaps most simply motivated by the following idea: The familiar paradoxes seem to arise when the class CP of all P-sets is claimed to be a set, while there exists some P-object x not in CP such that x would have to be a set if CP were. Clearly this cannot happen if all P-objects are sets. Now, Levy [2] and Reinhardt [3] showed that A* (A with regularity) is in some sense equivalent to ZF. But the strong replacement axiom of Gödel-Bernays set theory intuitively ought to be a theorem of A* although in fact it is not (Levy's work shows this). Strong replacement can be formulated as This lack of A* can be remedied by replacing S above by where ψ and φ are ∈-formulas and x is not in ψ and w is not in φ. ψv is ψ with quantifiers relativized to V, and y and z stand for y1, …, yn and z1, …, zm.