Black-box classifiers are able to classify unseen instances, once they have been trained on an appropriate (domain) dataset. Such classifiers have the advantage of being generally very efficient but the disadvantage of not being able to explain their processes to a user. For these reasons, over the last decade or so, a number of rule extraction algorithms have been developed which are able to extract a rule-set from classifiers. The focus of this project has been to re-implement a state-of-the-art rule extraction system, OSRE [1], and then to show that when the extracted rules are refined by the Knowledge Refinement system, FIXIT, that the refinement process, in virtually all cases, improves the fidelity of the refined rule-set when compared with the rule-set extracted by OSRE. A statistically significant difference between these two approaches has been demonstrated.

We present a Bayesian analysis of ordinal annotations made by clinicians of patients in intensive care. In particular, we investigate the different ways in which clinicians can disagree and how their disagreement is reduced once they take part in a recently proposed procedure (INSIGHT) that aims at improving consistency. The model combines a nonparametric function (loosely interpretable as the health of the patient) with clinician-specific generative procedures for producing the observed ordinal values. Our analysis provides valuable details of the rating behavior of the individual clinicians and shows that the INSIGHT procedure is particularly effective at removing (some) clinician-specific inconsistencies and biases.

OBJECTIVES: The work reported here focuses on developing novel techniques which enable an expert to detect inconsistencies in 2 (or more) perspectives that the expert might have on the same (classification) task. The high level task which the experts (physicians) had set themselves was to classify, on a 5-point severity scale (A-E), the hourly reports produced by an intensive care unit's patient management system. METHOD: The INSIGHT system has been developed to support domain experts exploring, and removing inconsistencies in their conceptualization of a task. We report here a study of intensive care physicians reconciling 2 perspectives on their patients. The 2 perspectives provided to INSIGHT were an annotated set of patient records where the expert had selected the appropriate category to describe that snapshot of the patient, and a set of rules which are able to classify the various time points on the same 5-point scale. Inconsistencies between these 2 perspectives are displayed as a confusion matrix; moreover INSIGHT then allows the expert to revise both the annotated datasets (correcting data errors, or changing the assigned categories) and the actual rule-set. RESULTS: Each of the 3 experts achieved a very high degree of consensus (~97%) between his refined knowledge sources (i.e., annotated hourly patient records and the rule-set). We then had the experts produce a common rule-set and then refine their several sets of annotations against it; this again resulted in inter-expert agreements of ~97%. The resulting rule-set can then be used in applications with considerable confidence. CONCLUSION: This study has shown that under some circumstances, it is possible for domain experts to achieve a high degree of correlation between 2 perspectives of the same task. The experts agreed that the immediate feedback provided by INSIGHT was a significant contribution to this successful outcome.

Myocardial damage is known to occur relatively frequently, and although it is not often fatal it results in the patient staying in the ICU for significantly longer. Thus it is important for clinicians to detect these events. Confirmation of myocardial damage is by a biomarker (troponin), but these tests are only done at fixed time-points. Consequently it is desirable for doctors, and support systems, to detect myocardial damage from the standard parameters collected for ICU patients. We have undertaken a study with several ICU consultants to determine the conditions which generally precede a myocardial-damaging event. In fact, these knowledge acquisition sessions produced a complex model which we have realized as 2 interacting modules. Subsequently, we compared this model's predictions against the original datasets; the model when run against the test dataset resulted in a relatively high True Positive (TP) rate (75.8%). The implications of these analyses are discussed, as are a number of planned follow-up studies

Physiological monitoring equipment can be found in many hospital settings. This allows a wide range of physiological parameters to be stored, which in turn allows clinicians and analysts to investigate a range of medical hypotheses. This paper introduces ACHE (architecture for clinical hypotheses examination), a framework specifically designed to support the preparation of such analyses. To evaluate the initial version of ACHE, a study to detect acute myocardial infarctions, was conducted with data from Glasgow Royal Infirmary's Intensive Care Unit (ICU). Initial results from the study are very encouraging and ACHE substantially reduced the time required to perform the study. A study of the same phenomena across a much larger patient dataset will be undertaken shortly.

The benefits of reuse have long been recognized in the knowledge engineering community where the dream of creating knowledge based systems (KBSs) on-the-fly from libraries of reusable components is still to be fully realised. In this paper we present a two stage methodology for creating KBSs: first reusing domain knowledge by mapping it, where appropriate, to the requirements of a generic problem solver; and secondly using this mapped knowledge and the requirements of the problem solver to "drive" the acquisition of the additional knowledge it needs.allFor example, suppose we have available a KBS which is composed of a propose-and-revise problem solver linked with an appropriate knowledge base/ontology from the elevator domain. Then to create a diagnostic KBS in the same domain, we require to map relevant information from the elevator knowledge base/ontology, such as component information, to a diagnostic problem solver, and then to extend it with diagnostic information such as malfunctions, symptoms and repairs for each component. We have developed MAKTab, a Protege plug-in which supports both these steps and results in a composite KBS which is executable.

In the Semantic Web, inconsistencies in OWL on- tologies may easily occur. Existing approaches ei- ther identify the minimally unsatisfiable sub-ontologies or calculate the maximally satisfiable sub-ontologies. However practical problems remain; it is not clear which axioms or which parts of axioms should be se- lected for repair, and how to repair those axioms. In this paper, we address this limitation by proposing a fine-grained approach to resolving unsatisfiable ontolo- gies. We revise the axiom tracing technique first pro- posed by Baader and Hollunder, so as to track which parts of the problematic axioms cause the unsatisfiabil- ity. Moreover, we support ontology users in rewriting problematic axioms. In order to minimise the impact of changes and prevent unintended entailment loss, harm- ful and helpful changes are identified and provided as guidelines. Based on the methods described we present a preliminary version of an interactive debugging tool and demonstrate its applicability in practice.

The Designers' Workbench is a system, developed by the Advanced Knowledge Technologies (AKT) consortium to support designers in large organizations, such as Rolls- Royce, by making sure that a design is consistent with the specification for the particular design as well as with the company's design rule book(s). Currently, to capture the constraint information, a domain expert (design engineer) has to work with a knowledge engineer to identify the constraints, and it is then the task of the knowledge engineer to encode these into the Workbench's knowledge base (KB). This is an error prone and time consuming task. It is highly desirable to relieve the knowledge engineer of this task, and so we have developed a tool, ConEditor, that enables domain experts themselves to capture and maintain these constraints. The tool allows the user to combine selected entities from the domain ontology with keywords and operators of a constraint language to form a constraint expression. We hypothesize that to apply constraints appropriately, it is necessary to understand the context in which each constraint is applicable. We refer to this as "application conditions". We plan to make these application conditions machine interpretable and investigate how they, together with a domain ontology, can be used to support the verification and maintenance of constraints.

Effective reuse of Knowledge Bases (KBs) often entails the expensive task of identifying plausible KB-PS (Problem Solver) combinations. We propose a novel technique based on Constraint Satisfaction to enable more rapid identification of incompatible KBs, leaving fewer combinations on which to conduct a thorough investigation. In this paper, we describe our investigation process, its tools, and the latest empirical results applied to non-binary problems that demonstrate our relaxation approach is an effective method for plausibility testing.

To acquire knowledge that is fit for a specific purpose, it is very desirable to have a structured, declarative expression of the knowledge that is needed. This paper introduces a stand-alone knowledge acquisition tool, called COCKATOO (Constraint-Capable Knowledge Acquisition Tool), which uses constraint technology to specify the knowledge it requires. The language in which these specifications are given is based on the meta-language notation of context-free grammars. However, we also took the opportunity to build a tool that is both more flexible and powerful by augmenting context-free grammars with the expressiveness of constraints. COCKATOO was implemented using the SCREAMER+ declarative constraints package.

Machine instructional planners use changing and uncertain data to incrementally configure plans and control the execution and dynamic refinement of these plans. Current instructional planners cannot adequately plan, replan, and monitor the delivery of instruction. This is due in part to the fact that current instructional planners are incapable of planning in a global context, developing competing plans in parallel, monitoring their planning behavior, and dynamically adapting their control behavior. In response to these and other deficiencies of instructional planners a generic system architecture based on the blackboard model was implemented. This self‐improving instructional planner (SUP) dynamically creates instructional plans, requests execution of these plans, replans, and improves its planning behavior based on a student's responses to tutoring. Global planning was facilitated by explicitly representing decisions about past, current, and future plans on a global data structure called the plan blackboard. Planning in multiple worlds is facilitated by labeling plan decisions by the context in which they were generated. Plan monitoring was implemented as a set of monitoring knowledge sources. The flexible control capability for instructional planner was adapted from the blackboard architecture BB1. The explicit control structure of SUP enabled complex and flexible planning behavior while maintaining a simple planning architecture.

The design objective of the Leeds Transformation System is to transform existing programs, written in a variety of languages, into "tidier" programs.The total system was conceived of as having three phases: syntactic transformations, variable usage transformations, and synthesizing features.Because programmers vary greatly in what they consider to be a more acceptable form, we have aimed to make the system as data driven as possible.(That also enables us to deal with a variety of programming languages.)The paper reviews the first two phases, reports the second in some detail, and illustrates the use of the system on an ALGOL 60 program.Redundant assignments, redundant variables, and loop-invariant statements are discovered by means of a novel approach which represents variable usage within a program as a correspondence matrix.Potential enhancements of the system are also discussed.