We present a rich, multimodal dataset consisting of data from 40 teams of three humans conducting simulated urban search-and-rescue (SAR) missions in a Minecraft-based testbed, collected for the Theory of Mind-based Cognitive Architecture for Teams (ToMCAT) project. Modalities include two kinds of brain scan data---functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG), as well as skin conductance, heart rate, eye tracking, face images, spoken dialog audio data with automatic speech recognition (ASR) transcriptions, game screenshots, gameplay data, game performance data, demographic data, and self-report questionnaires. Each team undergoes up to six consecutive phases: three behavioral tasks, one mission training session, and two collaborative SAR missions. As time-synchronized multimodal data collected under a variety of circumstances, this dataset will support studying a large variety of research questions on topics including teamwork, coordination, plan recognition, affective computing, physiological linkage, entrainment, and dialog understanding. We provide an initial public release of the de-identified data, along with analyses illustrating the utility of this dataset to both computer scientists and social scientists.

Extending machine reading approaches to extract mathematical concepts and their descriptions is useful for a variety of tasks, ranging from mathematical information retrieval to increasing accessibility of scientific documents for the visually impaired. This entails segmenting mathematical formulae into identifiers and linking them to their natural language descriptions. We propose a rule-based approach for this task, which extracts LaTeX representations of formula identifiers and links them to their in-text descriptions, given only the original PDF and the location of the formula of interest. We also present a novel evaluation dataset for this task, as well as the tool used to create it.

We present an analysis of the problem of identifying biological context and associating it with biochemical events described in biomedical texts. This constitutes a non-trivial, inter-sentential relation extraction task. We focus on biological context as descriptions of the species, tissue type, and cell type that are associated with biochemical events. We present a new corpus of open access biomedical texts that have been annotated by biology subject matter experts to highlight context-event relations. Using this corpus, we evaluate several classifiers for context-event association along with a detailed analysis of the impact of a variety of linguistic features on classifier performance. We find that gradient tree boosting performs by far the best, achieving an F1 of 0.865 in a cross-validation study.

The ages of terrains on other planetary bodies are chiefly determined using crater size–frequency distributions. However, primary impacts can generate numerous secondary craters that can affect the crater population. Classifying impact craters as primary or secondary is commonly done via time-consuming manual inspection, which limits the areas that can be analyzed at high resolution. We present a parametric model for characterizing small (100–600 m diameter) impact craters, where the model parameters have implications for describing the physical processes involved in their formation and modification. We infer these parameters from craters in images captured by the high-resolution imaging science experiment (HiRISE) camera onboard the Mars Reconnaissance Orbiter. For each crater within the appropriate size range, our algorithm creates a 3-D surface for a parametrically modeled crater and a 2-D rendering using illumination metadata, including emission, phase, and solar incidence angles at the time when the image was captured. A function describes the likelihood of each set of model parameters in terms of the geometry of craters in a given HiRISE image. These values are then optimized using a Metropolis–Hasting Markov chain Monte Carlo sampler. We evaluated three different prior probability distributions over the parameter space and two different likelihoods: one for digital terrain models and the other for images. We show that after applying t-distributed stochastic neighbor embedding (t-SNE) over the inferred crater parameters, t-SNE is able to project the multidimensional crater parameters into a 2-D space where secondary craters cluster together and are separable from primary craters.

Recent efforts in bioinformatics have achieved tremendous progress in the machine reading of biomedical literature, and the assembly of the extracted biochemical interactions into large-scale models such as protein signaling pathways.However, batch machine reading of literature at today's scale (PubMed alone indexes over 1 million papers per year) is unfeasible due to both cost and processing overhead.In this work, we introduce a focused reading approach to guide the machine reading of biomedical literature towards what literature should be read to answer a biomedical query as efficiently as possible.We introduce a family of algorithms for focused reading, including an intuitive, strong baseline, and a second approach which uses a reinforcement learning (RL) framework that learns when to explore (widen the search) or exploit (narrow it).We demonstrate that the RL approach is capable of answering more queries than the baseline, while being more efficient, i.e., reading fewer documents.

We describe a generalization of the Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) which is able to encode prior information that state transitions are more likely between “nearby” states. This is accomplished by defining a similarity function on the state space and scaling transition probabilities by pairwise similarities, thereby inducing correlations among the transition distributions. We present an augmented data representation of the model as a Markov Jump Process in which: (1) some jump attempts fail, and (2) the probability of success is proportional to the similarity between the source and destination states. This augmentation restores conditional conjugacy and admits a simple Gibbs sampler. We evaluate the model and inference method on a speaker diarization task and a “harmonic parsing” task using four-part chorale data, as well as on several synthetic datasets, achieving favorable comparisons to existing models.

We consider linear dynamical systems, particularly coupled linear oscillators, where the parameters represent meaningful values in a domain theory and thus learning what affects them contributes to explanation. Rather than allow perturbations of latent states, we assume that temporal variation beyond noise is explained by parameter drift, and variation across coupled systems is a function of moderating variables. This change of focus reduces opportunities for efficient inference, and we propose sampling procedures to learn and fit the models. We test our approach on a real dataset of physiological measures of heterosexual couples engaged in a conversation about a potentially emotional topic, with body mass index (BMI) being considered as a moderator. We evaluate several models on their ability to predict future conversation dynamics (the last 20% of the data for each test couple), with shared parameters being learned using held out data. As proof of concept, we validate the hypothesis that BMI affects the conversation dynamic in the experimentally chosen topic.

We present theoretical and empirical results for a framework that combines the benefits of apprenticeship and autonomous reinforcement learning. Our approach modifies an existing apprenticeship learning framework that relies on teacher demonstrations and does not necessarily explore the environment. The first change is replacing previously used Mistake Bound model learners with a recently proposed framework that melds the KWIK and Mistake Bound supervised learning protocols. The second change is introducing a communication of expected utility from the student to the teacher. The resulting system only uses teacher traces when the agent needs to learn concepts it cannot efficiently learn on its own.

Currently, most systems for human-robot teaching allow only one mode of teacher-student interaction (e.g., teaching by demonstration or feedback), and teaching episodes have to be carefully set-up by an expert. To understand how we might integrate multiple, interleaved forms of human instruction into a robot learner, we performed a behavioral study in which 44 untrained humans were allowed to freely mix interaction modes to teach a simulated robot (secretly controlled by a human) a complex task. Analysis of transcripts showed that human teachers often give instructions that are nontrivial to interpret and not easily translated into a form useable by machine learning algorithms. In particular, humans often use implicit instructions, fail to clearly indicate the boundaries of procedures, and tightly interleave testing, feedback, and new instruction. In this paper, we detail these teaching patterns and discuss the challenges they pose to automatic teaching interpretation as well as the machine-learning algorithms that must ultimately process these instructions. We highlight the challenges by demonstrating the difficulties of an initial automatic teacher interpretation system.

We describe the Hats simulator as an information fusion challenge problem. Hats is a virtual world in which many agents engage in individual and collective activities. Most agents are benign, some intend harm. Agent activities are planned by a generative planner. Playing against the simulator, the goal of the analyst is to identify and arrest harmful agents before they carry out their plans. The simulator provides both scalar and categorical information. Information fusion tasks in the Hats domain include assessing information value, choosing information collection strategies, tracking individuals and resources, identifying events, hypothesizing group membership, ascribing suspicion, and identifying plans. After each game, the analyst is assessed a set of scores including the cost of acquiring information, the cost of falsely accusing benign agents, and the cost of failing to detect harmful agents. The simulator is implemented and currently manages hundreds of thousands of agents

COLAB is an analysis environment in which multiple human analysts in different physical locations can collaborate to build hypotheses of unfolding scenarios. COLAB consists of two components: an instrumented analysis working environment built on a blackboard system, and a Web-based user interface that integrates the trellis hypothesis authoring and management tool with a query language. On the blackboard, analysts collect data and perform analyses using relational database queries and a set of analysis tools that include group identification and suspicion scoring algorithms. Evidence extracted from data analysis on the blackboard may then be incorporated in hierarchical argument structures in the trellis tool, combining evidence sources with user-supplied free text. Trellis arguments may then be shared between analysts and collaboratively authored. COLAB has been integrated with the Hats Simulator challenge domain and serves as a prototype mixed-initiative information fusion system

Two novel artificial neural network techniques, evolutionary programming (EP) and probabilistic neural networks (PNN), were applied to the problem of breast cancer diagnosis. The EP is a stochastic optimization technique with the ability to mutate both network connections and weight values. The PNN has the ability to produce optimal Bayesian decision making given sufficient training data. Both techniques offer potential improvements over the well-studied, classic backpropagation networks. Preliminary performances of these new techniques were comparable to but slightly worse than the classic networks. In on-going work, these new techniques will be optimized further and should produce results greater than or equal to the classic networks, but with more information content and confidence.

Situated action is a new approach to artificial intelligence that has thus far functioned without any explicit underlying theoretical foundation. As a result, many researchers in artificial intelligence have misunderstood the goals and claims of situated action. In order to rectify this situation, we provide an explicit formulation of the theoretical foundations of situated action.