Given a set of rules expressed in real-world regulatory language and an action one wants to perform, can state-of-the-art large language models determine whether the action is permissible? If not, why not, and what additional information is needed before they can perform this task well? These questions have been of significant interest recently, given the use of rules to constrain language models (LMs) behaviors and ensure that they align with human values. However, existing work studying how well LMs can interpret those rules focuses on a narrow set of domains, with little focus on cases where those rules have the complexity of real-world legal systems. We investigate these questions by constructing a benchmark dataset from a board game selected for the complexity of its rule set. We study the effects of shot size, function calls, and example rationales on the quality of reasoning about action permissibility. We then discuss the implications of these findings to work in automated legal reasoning.

Animesh Nighojkar, John Licato. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

Analogical reasoning, it has been argued, fundamentally underlies many cognitive processes and is an important marker of developmental cognition. This connection suggests that the clever use of analogical reasoning tasks can improve cognitive performance in specific ways, thus leading to clear educational applications, as recent psychological work has confirmed. However, currently there are no known methods to either solve or generate analogical word problems, at least to a degree of reliability that would be necessary before such educational applications are possible. To address these concerns we present work to both solve and generate analogy word problems: First, given an analogy word problem, our algorithm performs a parallel random walk through the semantic network ConceptNet to limit the number of choices that are then considered by a vector embedding. We achieve an improvement in accuracy beyond existing state-of-the-art. Second, we explore a method for automatically generating explainable n-step analogy word problems, and analyze the results.

The protean word `autonomous' has gained broad currency as a descriptive adjective for AI research projects, robotic and otherwise. Depending upon context, `autonomous' at present connotes anything from a shallow, purely reactive system to a sophisticated cognitive architecture reflective of much of human cognition; hence the term fails to pick out any specific set of constitutive functionality. However, philosophers and ethicists have something relatively well-defined in mind when they talk about the idea of autonomy. For them, an autonomous agent is often by definition potentially morally responsible for its actions. Moreover, as a prerequisite to correct ascription of `autonomous,' a certain capacity to choose freely is assumed - even if this freedom is understood to be semi-constrained by societal conventions, moral norms, and the like.

Self-consciousness would seem to be a sine qua non for moral competence in a social world. You and we are morally competent in no small part because you know what you ought to do, and we know what we ought to do. A mouse, in contrast, cannot say to itself: “I ought to share this cheese, even if my brother refuses to do so.” But can robots be self-conscious? Approaching this question from the standpoint of so-called Psychometric AI, we note that prior work by Govindarajulu and Bringsjord led to the engineering of a robot (Cogito) able to provably pass the famous mirror test of self-consciousness. But a more challenging test for robot self-consciousness has been provided by Floridi; this test is an ingenious and much-harder variant of the well-known-in-AI wise-man puzzle: Each of three robots is given one pill from a group of five, three of which are innocuous, but two of which, when taken, immediately render the recipient dumb. In point of fact, two robots (R1and R2) are given potent pills, but R3receives one of the three placebos. The human tester says: “Which pill did you receive? No answer is correct unless accompanied by a proof!” Given a formal regimentation of this test previously formulated by Bringsjord, it can be proved that, in theory, a future robot represented by R3can answer provably correctly (which for plausible reasons, explained by Floridi, entails that R3has satisfied some of the structural requirements for self-consciousness). In this paper we explain and demonstrate the engineering that now makes this theoretical possibility actual, both in the simulator known as ‘PAGI World’ (used for testing AIs), and in real (= physical) robots interacting with a human tester. These demonstrations involve scenarios that demand the passing of Floridi's test for self-consciousness, where for us, passing such a test is required for an agent to be regarded morally competent.

Psychologically and neurobiologically plausible models of knowledge often must make a difficult choice between distributed and localist representation. Distributed representation can be flexible and hold up well to noisy data, but localist models allow for structured knowledge to be represented unambiguously and reasoned over in rigorous, transparent fashion. We present a way of representing knowledge within the hybrid cognitive architecture CLARION. Our system allows both structured knowledge and distributed knowledge to synergistically coexist while remaining within the limits defined by CLARION'S dual-process framework. After showing how our system can allow more complex knowledge structures to arise, we describe algorithms that use such structures to model many types of reasoning, including: analogical reasoning, deductive reasoning, moral reasoning, and more. We place the structural knowledge afforded CLARION within a formal hierarchy of expressivity for such knowledge, and discuss implications of this work.