We demonstrate an integrated system for building and learning models and structures in both a real and virtual environment. The system combines natural language understanding, planning, and methods for composition of basic concepts into more complicated concepts. The user and the system interact via natural language to jointly plan and execute tasks involving building structures, with clarifications and demonstrations to teach the system along the way. We use the same architecture for building and simulating models of biology, demonstrating the general-purpose nature of the system where domain-specific knowledge is concentrated in sub-modules with the basic interaction remaining domain-independent. These capabilities are supported by our work on semantic parsing, which generates knowledge structures to be grounded in a physical representation, and composed with existing knowledge to create a dynamic plan for completing goals. Prior work on learning from natural language demonstrations enables learning of models from very few demonstrations, and features are extracted from definitions in natural language. We believe this architecture for interaction opens up a wide possibility of human-computer interaction and knowledge transfer through natural language.

This paper describes a novel combination of Java program analysis and automated learning and planning architecture to the domain of Java vulnerability analysis. The key feature of our “HACKAR: Helpful Advice for Code Knowledge and Attack Resilience” system is its ability to analyze Java programs at development-time, identifying vulnerabilities and ways to avoid them. HACKAR uses an improved version of NASA’s Java PathFinder (JPF) to execute Java programs and identify vulnerabilities. The system features new Hierarchical Task Network (HTN) learning algorithms that (1) advance stateof-theart HTN learners with reasoning about numeric constraints, failures, and more general cases of recursion, and (2) contribute to problem-solving by learning a hierarchical dataflow representation of the program from the inputs of the program. Empirical evaluation demonstrates that HACKAR was able to suggest fixes for all of our test program suites. It also shows that HACKAR can analyze programs with string inputs that original JPF implementation cannot.

The POIROT project is a four-year effort to develop an architecture that integrates the products of a number of targeted reasoning and learning components to produce executable representations of demonstrated web service workflow processes. To do this it combines contributions from multiple trace analysis (interpretation) and learning methods guided by a meta-control regime that reviews explicit learning hypotheses and posts new learning goals and internal learning subtasks. POIROT's meta-controller guides the activity of its components through largely distinct phases of processing from trace interpretation, to inductive learning, hypotheses combination and experimental evaluation. In this paper we discuss the impact that various kinds of inference during the trace interpretation phase can have on the quality of the learned models.

Semantic Web services hold the promise of greatly increasing interoperability among software agents and Web services by enabling content-based (as opposed to format-based) automated service discovery and interaction. However, as different services may well use different, only partly compatible ontologies to describe their capabilities, some amount of ontology mapping or translation will be required during the various stages of service discovery and utilization. In this paper, we reexamine some of the processing assumptions that were made in the development of semantic Web service models like DAML-S in order to uncover the very different roles of semantic translation in the subprocesses of service discovery, service process model interpretation, task negotiation, service invocation and response interpretation. We present examples and arguments showing how several different styles of translation will be required based on the informational context and perspectives of agents in each of these stages.

We describe the formal specification and automated synthesis of a strategic airlift scheduler for the Air Mobility Command of the US Air Force. The program synthesis system, the Kestrel Interactive Development System, composes a formal domain theory with a formal description of a class of algorithms (global search with constraint propagation) to produce provably correct and highly efficient code that outperforms more conventional approaches to this scheduling problem.