Our research addresses the issue of developing knowledge-based agents that capture and use the problem solving knowledge of subject matter experts from diverse application domains. This paper emphasizes the use of negative examples in agent learning by presenting several strategies for capturing expert's knowledge when the agent fails to correctly solve a problem. These strategies have been implemented into the disciple learning agent shell and used in complex application domains such as intelligence analysis, center of gravity determination, and emergency response planning.

This paper presents recent results on developing learning agents that can be taught by subject matter experts how to solve problems, through examples and explanations. It introduces the lazy rule refinement method where the expert modifies an example generated by a learned rule. In this case the agent has to decide whether to modify the rule (if the modification applies to all the previous positive examples) or to learn a new rule. However, checking the previous examples would be disruptive or even impossible. The lazy rule refinement method provides an elegant solution to this problem, in which the agent delays the decision whether to modify the rule or to learn a new rule until it accumulated enough examples during the follow-on problem solving process. This method has been incorporated into the disciple learning agent shell and used in the complex application areas of center of gravity analysis and intelligence analysis

Over the years we have developed the Disciple theory, methodology, and family of tools for building knowledge-based agents. This approach consists of developing an agent shell that can be taught directly by a subject matter expert in a way that resembles how the expert would teach a human apprentice when solving problems in cooperation. This paper presents the most recent version of the Disciple approach and its implementation in the Disciple–RKF (rapid knowledge formation) system. Disciple–RKF is based on mixed-initiative problem solving, where the expert solves the more creative parts of the problem and the agent solves the more routine ones, integrated teaching and learning, where the agent helps the expert to teach it, by asking relevant questions, and the expert helps the agent to learn, by providing examples, hints, and explanations, and multistrategy learning, where the agent integrates multiple learning strategies, such as learning from examples, learning from explanations, and learning by analogy, to learn from the expert how to solve problems. Disciple–RKF has been applied to build learning and reasoning agents for military center of gravity analysis, which are used in several courses at the US Army War College.