This paper addresses the issue of adversarial attacks on ethical AI systems. We investigate using moral axioms and rules of deontic logic in a norm learning framework to mitigate adversarial norm training. This model of moral intuition and construction provides AI systems with moral guard rails yet still allows for learning conventions. We evaluate our approach by drawing inspiration from a study commonly used in moral development research. This questionnaire aims to test an agent's ability to reason to moral conclusions despite opposed testimony. Our findings suggest that our model can still correctly evaluate moral situations and learn conventions in an adversarial training environment. We conclude that adding axiomatic moral prohibitions and deontic inference rules to a norm learning model makes it less vulnerable to adversarial attacks.

Research in AI & Law has sought to model common-law case-based reasoning by creating analogies from cases, extracting and applying rules from cases, or both. This paper presents a new approach to extracting legal information from cases and several methods to apply it to new cases, including by analogy and by conversion to logical rules. It evaluates the approaches on a dataset of real-world cases and compares the results to off-the-shelf machine-learning techniques. We conclude that abstract legal information can be extracted from similar cases through analogical generalization, and that the extracted legal schemas can be used to reason about and solve other cases both by analogy and by rules.

Sketch recognition has been studied for decades, but it is far from solved. Drawing styles are highly variable across people and adapting to idiosyncratic visual expressions requires data-efficient learning. Explainability also matters, so that users can see why a system got confused about something. This paper introduces a novel part-based approach for sketch recognition, based on hierarchical analogical learning, a new method to apply analogical learning to qualitative representations. Given a sketched object, our system automatically segments it into parts and constructs multi-level qualitative representations of them. Our approach performs analogical generalization at multiple levels of part descriptions and uses coarse-grained results to guide interpretation at finer levels. Experiments on the Berlin TU dataset and the Coloring Book Objects dataset show that the system can learn explainable models in a data-efficient manner.

We introduce the Illinois Intentional Tort Qualitative Dataset, a set of Illinois Common Law cases in Assault, Battery, Trespass, and Self-Defense, machine-translated into qualitative predicate representations. We discuss the cases involved, the natural language understanding system used to translate the cases into predicate logic, and validation measures that serve as performance baselines for future AI research using the dataset.

Visual Relation Detection is currently one of the most popular problems for visual understanding. Many deep-learning models are designed for relation detection on images and have achieved impressive results. However, deep-learning models have several serious problems, including poor training-efficiency and lack of understandability. Psychologists have ample evidence that analogy is central in human learning and reasoning, including visual reasoning. This paper introduces a new hybrid system for visual relation detection combining deep-learning models and analogical generalization. Object bounding boxes and masks are detected using deep-learning models and analogical generalization over qualitative representations is used for visual relation detection between object pairs. Experiments on the Visual Relation Detection dataset indicates that our hybrid system gets comparable results on the task and is more training-efficient and explainable than pure deep-learning models.

Analogy is core to human cognition. It allows us to solve problems based on prior experience, it governs the way we conceptualize new information, and it even influences our visual perception. The importance of analogy to humans has made it an active area of research in the broader field of artificial intelligence, resulting in data-efficient models that learn and reason in human-like ways. While cognitive perspectives of analogy and deep learning have generally been studied independently of one another, the integration of the two lines of research is a promising step towards more robust and efficient learning techniques. As part of a growing body of research on such an integration, we introduce the Analogical Matching Network: a neural architecture that learns to produce analogies between structured, symbolic representations that are largely consistent with the principles of Structure-Mapping Theory.

Generating formal-language programs represented by relational tuples, such as Lisp programs or mathematical operations, to solve problems stated in natural language is a challenging task because it requires explicitly capturing discrete symbolic structural information implicit in the input. However, most general neural sequence models do not explicitly capture such structural information, limiting their performance on these tasks. In this paper, we propose a new encoder-decoder model based on a structured neural representation, Tensor Product Representations (TPRs), for mapping Natural-language problems to Formal-language solutions, called TP-N2F. The encoder of TP-N2F employs TPR ‘binding’ to encode natural-language symbolic structure in vector space and the decoder uses TPR ‘unbinding’ to generate, in symbolic space, a sequential program represented by relational tuples, each consisting of a relation (or operation) and a number of arguments. TP-N2F considerably outperforms LSTM-based seq2seq models on two benchmarks and creates new state-of-the-art results. Ablation studies show that improvements can be attributed to the use of structured TPRs explicitly in both the encoder and decoder. Analysis of the learned structures shows how TPRs enhance the interpretability of TP-N2F.

Deep learning systems can perform well on some image recognition tasks. However, they have serious limitations, including requiring far more training data than humans do and being fooled by adversarial examples. By contrast, analogical learning over relational representations tends to be far more data-efficient, requiring only human-like amounts of training data. This paper introduces an approach that combines automatically constructed qualitative visual representations with analogical learning to tackle a hard computer vision problem, object recognition from sketches. Results from the MNIST dataset and a novel dataset, the Coloring Book Objects dataset, are provided. Comparison to existing approaches indicates that analogical generalization can be used to identify sketched objects from these datasets with several orders of magnitude fewer examples than deep learning systems require.

Human action recognition remains a difficult problem for AI. Traditional machine learning techniques can have high recognition accuracy, but they are typically black boxes whose internal models are not inspectable and whose results are not explainable. This paper describes a new pipeline for recognizing human actions from skeleton data via analogical generalization. Specifically, starting with Kinect data, we segment each human action by temporal regions where the motion is qualitatively uniform, creating a sketch graph that provides a form of qualitative representation of the behavior that is easy to visualize. Models are learned from sketch graphs via analogical generalization, which are then used for classification via analogical retrieval. The retrieval process also produces links between the new example and components of the model that provide explanations. To improve recognition accuracy, we implement dynamic feature selection to pick reasonable relational features. We show the explanation advantage of our approach by example, and results on three public datasets illustrate its utility.

Creating systems that can learn to answer natural language questions has been a longstanding challenge for artificial intelligence. Most prior approaches focused on producing a specialized language system for a particular domain and dataset, and they required training on a large corpus manually annotated with logical forms. This paper introduces an analogy-based approach that instead adapts an existing general purpose semantic parser to answer questions in a novel domain by jointly learning disambiguation heuristics and query construction templates from purely textual question-answer pairs. Our technique uses possible semantic interpretations of the natural language questions and answers to constrain a query-generation procedure, producing cases during training that are subsequently reused via analogical retrieval and composed to answer test questions. Bootstrapping an existing semantic parser in this way significantly reduces the number of training examples needed to accurately answer questions. We demonstrate the efficacy of our technique using the Geoquery corpus, on which it approaches state of the art performance using 10-fold cross validation, shows little decrease in performance with 2-folds, and achieves above 50% accuracy with as few as 10 examples.

Sketching can be a valuable tool for science education, but it is currently underutilized. Sketch worksheets were developed to help change this, by using AI technology to give students immediate feedback and to give instructors assistance in grading. Sketch worksheets use visual representations automatically computed by CogSketch, which are combined with conceptual information from the OpenCyc ontology. Feedback is provided to students by comparing an instructor’s sketch to a student’s sketch, using the Structure-Mapping Engine. This paper describes our experiences in deploying sketch worksheets in two types of classes: Geoscience and AI. Sketch worksheets for introductory geoscience classes were developed by geoscientists at University of Wisconsin-Madison, authored using CogSketch and used in classes at both Wisconsin and Northwestern University. Sketch worksheets were also developed and deployed for a knowledge representation and reasoning course at Northwestern. Our experience indicates that sketch worksheets can provide helpful on-the-spot feedback to students, and significantly improve grading efficiency, to the point where sketching assignments can be more practical to use broadly in STEM education.

Understanding commonsense reasoning is one of the core challenges of AI. We are exploring an approach inspired by cognitive science, called analogical chaining, to create cognitive systems that can perform commonsense reasoning. Just as rules are chained in deductive systems, multiple analogies build upon each other’s inferences in analogical chaining. The cases used in analogical chaining – called common sense units – are small, to provide inferential focus and broader transfer. Importantly, such common sense units can be learned via natural language instruction, thereby increasing the ease of extending such systems. This paper describes analogical chaining, natural language instruction via microstories, and some subtleties that arise in controlling reasoning. The utility of this technique is demonstrated by performance of an implemented system on problems from the Choice of Plausible Alternatives test of commonsense causal reasoning.

Harnessing the statistical power of neural networks to perform language understanding and symbolic reasoning is difficult, when it requires executing efficient discrete operations against a large knowledge-base.In this work, we introduce a Neural Symbolic Machine (NSM), which contains (a) a neural "programmer", i.e., a sequence-to-sequence model that maps language utterances to programs and utilizes a key-variable memory to handle compositionality (b) a symbolic "computer", i.e., a Lisp interpreter that performs program execution, and helps find good programs by pruning the search space.We apply REINFORCE to directly optimize the task reward of this structured prediction problem.To train with weak supervision and improve the stability of REINFORCE we augment it with an iterative maximum-likelihood training process.NSM outperforms the state-of-theart on the WEBQUESTIONSSP dataset when trained from question-answer pairs only, without requiring any feature engineering or domain-specific knowledge.

Moral reasoning is important to accurately model as AI systems become ever more integrated into our lives. Moral reasoning is rapid and unconscious; analogical reasoning, which can be unconscious, is a promising approach to model moral reasoning. This paper explores the use of analogical generalizations to improve moral reasoning. Analogical reasoning has already been used to successfully model moral reasoning in the MoralDM model, but it exhaustively matches across all known cases, which is computationally intractable and cognitively implausible for human-scale knowledge bases. We investigate the performance of an extension of MoralDM to use the MAC/FAC model of analogical retrieval over three conditions, across a set of highly confusable moral scenarios.

Fast and efficient learning over large bodies of commonsense knowledge is a key requirement for cognitive systems. Semantic web knowledge bases provide an important new resource of ground facts from which plausible inferences can be learned. This paper applies structured logistic regression with analogical generalization (SLogAn) to make use of structural as well as statistical information to achieve rapid and robust learning. SLogAn achieves state-of-the-art performance in a standard triplet classification task on two data sets and, in addition, can provide understandable explanations for its answers.

Concept learning is a central problem for cognitive systems. Generalization techniques can help organize examples by their commonalities, but comparisons with non-examples, near-misses, can provide discrimination. Early work on near-misses required hand-selected examples by a teacher who understood the learner’s internal representations. This paper introduces Analogical Learning by Integrating Generalization and Near-misses (ALIGN) and describes three key advances. First, domain-general cognitive models of analogical processes are used to handle a wider range of examples. Second, ALIGN’s analogical generalization process constructs multiple probabilistic representations per concept via clustering, and hence can learn disjunctive concepts. Finally, ALIGN uses unsupervised analogical retrieval to find its own near-miss examples. We show that ALIGN out-performs analogical generalization on two perceptual data sets: (1) hand-drawn sketches; and (2) geospatial concepts from strategy-game maps.

The naturalness of qualitative reasoning suggests that qualitative representations might be an important component of the semantics of natural language. Prior work showed that frame-based representations of qualitative process theory constructs could indeed be extracted from natural language texts. That technique relied on the parser recognizing specific syntactic constructions, which had limited coverage. This paper describes a new approach, using narrative function to represent the higher-order relationships between the constituents of a sentence and between sentences in a discourse. We outline how narrative function combined with query-driven abduction enables the same kinds of information to be extracted from natural language texts. Moreover, we also show how the same technique can be used to extract type-level qualitative representations from text, and used to improve performance in playing a strategy game.

Efficient reasoning in large knowledge bases is an important problem for AI systems. Hand-optimization of reasoning becomes impractical as KBs grow, and impossible as knowledge is automatically added via knowledge capture or machine learning. This paper describes a method for automatic extraction of axioms for efficient inference over large knowledge bases, given a set of query types and information about the types of facts in the KB currently as well as what might be learned. We use the highly right skewed distribution of predicate connectivity in large knowledge bases to prune intractable regions of the search space. We show the efficacy of these techniques via experiments using queries from a learning by reading system. Results show that these methods lead to an order of magnitude improvement in time with minimal loss in coverage.

Commonsense reasoning at scale is a core problem for cognitive systems. In this paper, we discuss two ways in which heuristic graph traversal methods can be used to generate plausible inference chains. First, we discuss how Cyc’s predicate-type hierarchy can be used to get reasonable answers to queries. Second, we explain how connection graph-based techniques can be used to identify script-like structures. Finally, we demonstrate through experiments that these methods lead to significant improvement in accuracy for both Q/A and script construction.

One of the major challenges to building intelligent educational software is determining what kinds of feedback to give learners. Useful feedback makes use of models of domain-specific knowledge, especially models that are commonly held by potential students. To empirically determine what these models are, student data can be clustered to reveal common misconceptions or common problem-solving strategies. This paper describes how analogical retrieval and generalization can be used to cluster automatically analyzed hand-drawn sketches incorporating both spatial and conceptual information. We use this approach to cluster a corpus of hand-drawn student sketches to discover common answers. Common answer clusters can be used for the design of targeted feedback and for assessment.

Creating software agents that learn interactively requires the ability to learn from a small number of trials, extracting general, flexible knowledge that can drive behavior from observation and interaction. We claim that qualitative models provide a useful intermediate level of causal representation for dynamic domains, including the formulation of strategies and tactics. We argue that qualitative models are quickly learnable, and enable model-based reasoning techniques to be used to recognize, operationalize, and construct more strategic knowledge. This paper describes an approach to incrementally learning qualitative influences by demonstration in the context of a strategy game. We show how the learned model can help a system play by enabling it to explain which actions could contribute to maximizing a quantitative goal. We also show how reasoning about the model allows it to reformulate a learning problem to address delayed effects and credit assignment, such that it can improve its performance on more strategic tasks such as city placement.

How do reasoning systems that learn evolve over time? What are the properties of different learning strategies? Characterizing the evolution of these systems is important for understanding their limitations and gaining insights into the interplay between learning and reasoning. We describe an inverse ablation model for studying how large knowledge-based systems evolve: Create a small knowledge base by ablating a large KB, and simulate learning by incrementally re-adding facts, using different strategies to simulate types of learners. For each iteration, reasoning properties (including number of questions answered and run time) are collected, to explore how learning strategies and reasoning interact. We describe several experiments with the inverse ablation model, examining how two different learning strategies perform. Our results suggest that different concepts show different rates of growth, and that the density and distribution of facts that can be learned are important parameters for modulating the rate of learning.

Qualitative representations are suitable for sketch understanding systems because they highlight important relationships while leaving out details that are not essential for conceptual understanding. These representations can be used to perform spatial analogies between sketches, which determine qualitative similarities and differences. However, there are cases where including quantitative information is necessary for accurately representing a sketch. We describe a method for using quantitative information to constrain qualitative spatial analogies. The utility of this method is demonstrated in the context of a sketch based educational software system. Importantly, using quantitative information to improve analogical matches is not domain specific. It can be used in any situation where qualitative and quantitative spatial information must be combined to accurately interpret a sketch. This approach has the potential to improve sketch understanding in educational software applications for highly spatial domains.

Analogy is heavily used in instructional texts. We introduce the concept of analogical dialogue acts (ADAs), which represent the roles utterances play in instructional analogies. We describe a catalog of such acts, based on ideas from structure-mapping theory. We focus on the operations that these acts lead to while understanding instructional texts, using the Structure-Mapping Engine (SME) and dynamic case construction in a computational model. We test this model on a small corpus of instructional analogies expressed in simplified English, which were understood via a semi-automatic natural language system using analogical dialogue acts. The model enabled a system to answer questions after understanding the analogies that it was not able to answer without them.

Our goal is to enable military planners to rapidly critique alternative battle plans by simulating multiple outcomes of adversarial plans. We describe a novel simulator, SimPath, that combines qualitative reasoning, a geographic information system (GIS), and targeted probabilistic calculations to envision how adversarial battle plans can play out. We outline the problem and describe the overall operation of the simulator. We then explain how qualitative process theory is extended with actions to model military tasks, how envisioning is factored to reduce combinatorial explosions, and how probabilities are computed for transitions and used to filter possibilities. Empirical results, including an experiment conducted by an independent evaluator, are summarized. The results show that it is possible to identify dozens of possible outcomes on each of 9 combinations of adversarial plans (COAs) in under two minutes. We close with a discussion of future work.

One of the central problems of artificial intelligence is capturing the breadth and flexibility of human common sense reasoning. One way to evaluate common sense is to use versions of human tests that rely on everyday reasoning. The Bennett Mechanical Comprehension Test consists of everyday reasoning problems posed via pictures and is used to evaluate technicians. This test is challenging because it requires conceptual knowledge spanning a broad range of domains, experience with a wide variety of everyday situations, and spatial reasoning. This article describes how we have extended our Companion Cognitive Architecture, which treats analogical processing as central, to perform well over a subset of the Bennett test. We introduce analogical model formulation as a robust method for reasoning about everyday scenarios, by analogy with cases that represent prior experiences. This enables a companion to perform qualitative reasoning (QR) without a complete domain theory, as typically required for QR. We introduce sketch annotations to communicate linkages between visual and conceptual properties in sketches. We introduce analogical reference frames to enable comparative analysis to operate over a broader range of problems than prior techniques. We show that these techniques enable a companion to score reasonably well on a difficult subset of the Bennett test.

Understanding conceptual change is an important problem in modeling human cognition and in making integrated AI systems that can learn autonomously. This paper describes a model of explanation-based conceptual change, integrating sketch understanding, analogical processing, qualitative models, truth-maintenance, and heuristic-based reasoning within the Companions cognitive architecture. Sketch understanding is used to automatically encode stimuli in the form of comic strips. Qualitative models and conceptual quantities are constructed for new phenomena via analogical reasoning and heuristics. Truth-maintenance is used to integrate conceptual and episodic knowledge into explanations, and heuristics are used to modify existing conceptual knowledge in order to produce better explanations. We simulate the learning and revision of the concept of force, testing the concepts learned via a questionnaire of sketches given to students, showing that our model follows a similar learning trajectory.

Intelligent tutoring systems and learning environments can provide important benefits for education, but few have been developed for heavily spatial domains. One bottleneck has been the lack of rich models of visual and conceptual processing in sketch understanding, so that what students draw can be interpreted in a human-like way. This paper describes Sketch Worksheets, a form of sketch-based educational software that mimics aspects of pencil and paper worksheets commonly found in classrooms, but provides on-the-spot feedback and support for richer off-line assessments. The basic architecture of sketch worksheets is described, including an authoring environment that allows non-developers to create them and a coach that uses analogy to compare student and instructor sketches as a means to provide feedback. A pilot experiment where sketch worksheets were used successfully in a college geoscience class in Fall 2009 is summarized to show the potential of the idea.

Using content-specific models to guide information retrieval and extraction can provide richer interfaces to end-users for both understanding the context of news events and navigating related news articles. In this paper we discuss a system, Brussell, that uses semantic models to organize retrieval and extraction results, generating both storylines explaining how news event situations unfold and also biographical sketches of the situation participants. We generalize these models to introduce a new category of knowledge representation, an explanatory structure, that can scale up to include information from hundreds of documents, yet still provide model-based UI support to end-users. An informal survey of business news suggests the broad prevalence of news event situations indicating Brussell's potential utility, while an evaluation quantifies its performance in finding kidnapping situations.

Many information sources use multiple modalities, such as textbooks, which contain both text and diagrams. Each captures information that is hard to express in the other, and evidence suggests that multimodal information leads to better retention and transfer in human learners. This paper describes a system that captures textbook knowledge, using simplified English text and sketched versions of diagrams. We present experimental results showing it can use captured knowledge to answer questions from the textbook's curriculum.

Readers interested in the context of an event covered in the news such as the dismissal of a lawsuit can benefit from easily finding out about the overall news situation, the legal trial, of which the event is a part. Guided by abstract models of news situation types such as legal trials, corporate acquisitions, and kidnappings, Brussell is a system that presents situation instances it creates by reading multiple articles about the specific events that comprise them. We discuss how these situation models are structured and how they drive the creation of particular instances.

A serious barrier to the digitalization of the US military is that commanders find traditional mouse/menu, CAD-style interfaces unnatural. Military commanders develop and communicate battle plans by sketching courses of action (COAs). This paper describes nuSketch Battlespace, the latest version in an evolving line of sketching interfaces that commanders find natural, yet supports significant increased automation. We describe techniques that should be applicable to any specialized sketching domain: glyph bars and compositional symbols to tractably handle the large number of entities that military domains use, specialized glyph types and gestures to keep drawing tractable and natural, qualitative spatial reasoning to provide sketch-based visual reasoning, and comic graphs to describe multiple states and plans. Experiments, both completed and in progress, are described to provide evidence as to the utility of the system.

Sketching provides a natural means of interaction for many spatially-oriented tasks. One task where sketching is used extensively is when military planners are formulating battle plans, called Courses of Action (COAs). This paper describes a system we have built, nuSketch Battlespace (nSB), which provides a sketching interface for creating COAs. The system is described in the paper "Sketching for Military Courses of Action" in these proceedings. The demonstration will highlight:

Many concepts and situations are best explained by sketching. This paper describes our work on sKEA, the sketching Knowledge Entry Associate, a system designed for knowledge capture via sketching. We discuss the key ideas of sKEA: blob semantics for glyphs to sidestep recognition for visual symbols, qualitative spatial reasoning to provide richer visual and conceptual understanding of what is being communicated, arrows to express domain relationships, layers to express within-sketch segmentation (including a meta-layer to express subsketch relationships themselves via sketching), and analogical comparison to explore similarities and differences between sketched concepts. Experiences with sKEA to date and future plans are also discussed.

Many concepts and situations are best explained by sketching. This demonstration will show the key ideas underlying sKEA, the sketching knowledge entry associate, a system we have built for knowledge capture via sketching. In particular, we will demonstrateHow glyph bars and blob semantics are used to sidestep the need for recognition of visual symbols. The use of qualitative spatial reasoning to provide richer visual and conceptual understanding of what is being communicatedHow arrows are used to express domain relationshipsThe use of layers to express within-sketch segmentation, including a meta-layer to express subsketch relationships themselves via sketchingUsing analogical comparison to explore similarities and differences between sketched concepts.

Sketching is a powerful means of interpersonal communication. While many useful multimodal systems have been created, current systems are far from achieving human-like participation in sketching. A computational model of sketching would help characterize these differences and help us better understand how to overcome them. This paper is a first step towards such a model. We start with an example of a sketching system(nuSketch COA Creator)designed to aid military planners, to provide context and a source of examples. We then describe four dimensions of sketching,visual understanding, conceptual understanding, language understanding,anddrawing,that can be used to characterize the competence of existing systems and identify open problems. The issues involved will be illustrated by examples from our experience with nuSketch. Three research challenges are posed, to serve as milestones towards a computational model of sketching that can explain and replicate human abilities in this area.

Knowledge is the fuel for intelligent systems. Building software that comes closer to the breadth and flexibility of human reasoning will require substantially larger knowledge bases than any that have been built to date. This talk describes two ways we are exploring for bootstrapping intelligent systems via knowledge capture. First, analogy is useful for knowledge capture because people find it easier to articulate examples than universally valid principles. By exploiting recent advances in cognitive science, we are creating a technology of analogical processing that can (and has) been used with multiple large knowledge bases. Second, sketching is useful for knowledge capture because people find it easier to express many things spatially. By focusing on deeper visual and conceptual understanding of the contents of sketching, instead of recognition, we are building sketching systems that can be broadly applied in many domains. These efforts have benefited substantially from involvement with DARPA research communities, and some lessons learned from those experiences will be discussed.

Hofstadter and his colleagues have criticized current accounts of analogy, claiming that such accounts do not accurately capture interactions between processes of representation construction and processes of mapping. They suggest instead that analogy should be viewed as a form of high level perception that encompasses both representation building and mapping as indivisible operations within a single model. They argue specifically against SME, our model of analogical matching, on the grounds that it is modular, and offer instead programs such as Mitchell and Hofstadter's Copycat as examples of the high level perception approach. In this paper we argue against this positionon two grounds. First, we demonstrate that most of their specific arguments involving SME and Copycat are incorrect. Second, we argue that the claim that analogy is high-level perception, while in some ways an attractive metaphor, is too vague to be useful as a technical proposal. We focus on five issues: (1) how perceptionrelates to analogy,(2) how flexibilityarises in analogicalprocessing, (3) whether analogy is a domain-general process, (4) how micro-worlds should be used in the study of analogy, and (5) how best to assess the psychological plausibility of a model of analogy. We illustrate our discussion with examples taken from computer models embodying both views.

An unsolved problem in creating diagnostic expert systems is generating a qualitative understanding of how the system is behaving from raw data, especially numerical data taken across time. Yet automating this critical step is necessary for building the next generation of expert systems. The theory described provides a means of interpreting observations made of a physical system across time in terms of qualitative theories. Importantly, the theory is ontology-independent as well as domain-independent in that it only requires a qualitative description of the domain capable of supporting envisioning and domain-specific techniques for providing an initial qualitative description of numerical measurements. The theory is illustrated step by step with two extended examples, one involving qualitative process theory and the other involving a qualitative state vector representation of motion. The performance of an implementation of the theory is also illustrated.