Understanding the implicit values and beliefs of diverse groups and cultures using qualitative texts – such as long-form narratives – and domain-expert interviews is a fundamental goal of social anthropology. This paper builds upon a 2022 study that introduced the NLP task of Recognizing Value Resonance (RVR) for gauging perspective – positive, negative, or neutral – on implicit values and beliefs in textual pairs. This study included a novel hand-annotated dataset, the World Values Corpus (WVC), designed to simulate the task of RVR, and a transformer-based model, Resonance-Tuned RoBERTa, designed to model the task. We extend existing work by refining the task definition and releasing the World Values Corpus (WVC) dataset. We further conduct several validation experiments designed to robustly evaluate the need for task specific modeling, even in the world of LLMs. Finally, we present two additional Resonance-Tuned models trained over extended RVR datasets, designed to improve RVR model versatility and robustness. Our results demonstrate that the Resonance-Tuned models outperform top-performing Recognizing Textual Entailment (RTE) models in recognizing value resonance as well as zero-shot GPT-3.5 under several different prompt structures, emphasizing its practical applicability. Our findings highlight the potential of RVR in capturing cultural values within texts and the importance of task-specific modeling.

Novelty detection in open worlds is a valuable endeavor for improving the robustness of autonomous systems. Open worlds are characterized by a lack of constraints on the types of novelties that might occur. Here we describe recent progress on detecting novelties in open-world numerical properties in our OpenMIND agent, a domain-independent planning-based AI architecture. Our approach has three elements: (1) Feature construction by crossing a tractable number of domain-dependent sensors with some basic domainindependent statistical derivations, (2) online, univariate changepoint detection on each signal, and (3) negative feature selection by screening for false positives in non-novel scenarios. We report on the effectiveness of this approach in a single-blind evaluation in which OpenMIND plays a game and encounters novelties never observed by it or its developers.

Building and evaluating explainable Artificial Intelligence (AI) systems that accommodate human cognition remains a challenge for Human-Computer Interaction (HCI) and the need for practical solutions increases with our reliability on machines to extract, classify, and process information. Recent work has proposed triggers and metrics for explainable AI based on human mental models and psychological explanation quality. We complement this previous work by (1) extending and supporting these triggers and metrics with existing directives for information integrity, transparency, and rigor, (2) outlining a provenance-based framework for recording human-machine collaboration, and (3) demonstrating that a provenance-based approach address many of these explainable AI triggers and metrics. We show that provenance-based analyses help address questions of foundations, alternatives, necessity vs. sufficiency, sensitivity (e.g., what-if analyses), impact, and rationale, and we provide concrete evidence using an implemented human-machine analytic workspace. We outline ways to empirically measure the ability of these additional interpretation strategies to improve human understanding.

Deep learning systems can perform well on some image recognition tasks. However, they have serious limitations, including requiring far more training data than humans do and being fooled by adversarial examples. By contrast, analogical learning over relational representations tends to be far more data-efficient, requiring only human-like amounts of training data. This paper introduces an approach that combines automatically constructed qualitative visual representations with analogical learning to tackle a hard computer vision problem, object recognition from sketches. Results from the MNIST dataset and a novel dataset, the Coloring Book Objects dataset, are provided. Comparison to existing approaches indicates that analogical generalization can be used to identify sketched objects from these datasets with several orders of magnitude fewer examples than deep learning systems require.

Sketching can be a valuable tool for science education, but it is currently underutilized. Sketch worksheets were developed to help change this, by using AI technology to give students immediate feedback and to give instructors assistance in grading. Sketch worksheets use visual representations automatically computed by CogSketch, which are combined with conceptual information from the OpenCyc ontology. Feedback is provided to students by comparing an instructor’s sketch to a student’s sketch, using the Structure-Mapping Engine. This paper describes our experiences in deploying sketch worksheets in two types of classes: Geoscience and AI. Sketch worksheets for introductory geoscience classes were developed by geoscientists at University of Wisconsin-Madison, authored using CogSketch and used in classes at both Wisconsin and Northwestern University. Sketch worksheets were also developed and deployed for a knowledge representation and reasoning course at Northwestern. Our experience indicates that sketch worksheets can provide helpful on-the-spot feedback to students, and significantly improve grading efficiency, to the point where sketching assignments can be more practical to use broadly in STEM education.

Concept learning is a central problem for cognitive systems. Generalization techniques can help organize examples by their commonalities, but comparisons with non-examples, near-misses, can provide discrimination. Early work on near-misses required hand-selected examples by a teacher who understood the learner’s internal representations. This paper introduces Analogical Learning by Integrating Generalization and Near-misses (ALIGN) and describes three key advances. First, domain-general cognitive models of analogical processes are used to handle a wider range of examples. Second, ALIGN’s analogical generalization process constructs multiple probabilistic representations per concept via clustering, and hence can learn disjunctive concepts. Finally, ALIGN uses unsupervised analogical retrieval to find its own near-miss examples. We show that ALIGN out-performs analogical generalization on two perceptual data sets: (1) hand-drawn sketches; and (2) geospatial concepts from strategy-game maps.