The Artificial Neural Network is generally considered to be an effective classifier, but also a “Black Box” component whose internal behavior cannot be understood by human users. This lack of transparency forms a barrier to acceptance in high-stakes applications by the general public. This paper investigates the use of a hybrid model comprising multiple artificial neural networks with a final C4.5 decision tree classifier to investigate the potential of explaining the classification decision through production rules. Two large datasets collected from comprehension studies are used to investigate the value of the C4.5 decision tree as the overall comprehension classifier in terms of accuracy and decision transparency. Empirical trials show that higher accuracies are achieved through using a decision tree classifier, but the significant tree size questions the rule transparency to a human.

This paper presents the application of FATHOM, a computerised non-verbal comprehension detection system, to distinguish participant comprehension levels in an interactive tutorial. FATHOM detects high and low levels of human comprehension by concurrently tracking multiple non-verbal behaviours using artificial neural networks. Presently, human comprehension is predominantly monitored from written and spoken language. Therefore, a large niche exists for exploring human comprehension detection from a non-verbal behavioral perspective using artificially intelligent computational models such as neural networks. In this paper, FATHOM was applied to a video-recorded exploratory study containing a learning task designed to elicit high and low comprehension states from the learner. The learning task comprised of watching a video on termites, suitable for the general public and an interview led question and answer session. This paper describes how FATHOM's comprehension classifier artificial neural network was trained and validated in comprehension detection using the standard backpropagation algorithm. The results show that high and low comprehension states can be detected from learner's non-verbal behavioural cues with testing classification accuracies above 76%.

A problem in the field of semantic sentence similarity is the inability of sentence similarity measures to accurately represent perception based (fuzzy) words that are commonly used in natural language. This paper presents a new sentence similarity measure that attempts to solve this problem. The new measure, Fuzzy Algorithm for Similarity Testing (FAST) is an ontology based similarity measure that uses concepts of fuzzy and computing with words to allow for the accurate representation of fuzzy based words. Through human experimentation fuzzy sets were created for six categories of words based on their levels of association with particular concepts. These fuzzy sets were then defuzzified and the results used to create new ontological relations between the words. Using these relationships allows for the creation of a new ontology based semantic text similarity algorithm that is able to show the effect of fuzzy words on computing sentence similarity as well as the effect that fuzzy words have on non-fuzzy words within a sentence. Experiments on FAST were conducted using a new fuzzy dataset, the creation of which is described in this paper. The results of the evaluation showed that there was an improved level of correlation between FAST and human test results over two existing sentence similarity measures.

Semantic similarity is an essential component of numerous applications in fields such as natural language processing, artificial intelligence, linguistics, and psychology. Most of the reported work has been done in English. To the best of our knowledge, there is no word similarity measure developed specifically for Arabic. This paper presents a method to measure the semantic similarity between two Arabic words in the Arabic knowledge base. The semantic similarity is calculated through the combination of the common and different attributes between the Arabic words in the hierarchy semantic net. We use a previously developed Arabic word benchmark dataset to optimize and evaluate the Arabic measure. Experimental evaluation indicates that the Arabic measure is performing well. It has achieved a correlation value of 0.894 compared with the average value of human participants of 0.893 on evaluation dataset.

Natural language dialogue is an important component of interaction between ordinary users and complex computer applications. Short Text Semantic Similarity algorithms have been developed to improve the efficiency of producing sophisticated dialogue systems. Such algorithms are currently unable to discriminate between different dialogue acts (assertions, questions, instructions etc.), requiring the addition of efficient dialogue act classifiers to enhance them. The Slim Function Word Classifier (SFWC) has proved promising, particularly in its computational simplicity. This study optimizes the SFWC by clustering function word features using grammatical principles. Experiments show a significant improvement in classification accuracy for a selection of sentence forms which were challenging for the unoptimized SFWC. Results are expected to be applicable to many intelligent text processing applications ranging from question answering to meeting summarization.

This paper presents the adaptation and application of Silent Talker, a psychological profiling system in the measurement of human comprehension through the monitoring of multiple channels of facial nonverbal behaviour using Artificial Neural Networks (ANN). Everyday human interactions are abundant with almost unconscious nonverbal behaviours, providing a potentially rich source of information once decoded. Existing comprehension assessments techniques are inhibited by inconsistencies, limited to the verbal communication dimension and are often time-consuming with feedback delay. Major weaknesses hinder humans as accurate decoders of nonverbal behaviour with being error prone, inconsistent and poor at simultaneously focusing on multiple channels. Furthermore, human decoders are susceptible to fatigue and require training resulting in a costly, time-consuming process. ANNs are powerful, adaptable, scalable computational models that are able to overcome human decoder and pattern classification weaknesses. Therefore, the neural networks computer-based Silent Talker system has been trained and validated in the measurement of human comprehension using videotaped participant nonverbal behaviour from a Human Immunodeficiency Virus informed consent field study in Tanzania. A series of experiments on training backpropagation ANNs with different topologies were conducted. The results show that comprehension and noncomprehension patterns exist within the monitored multichannels of facial NVB with both experiments consistently yielding classification accuracies above 80%.

Oscar is a conversational intelligent tutoring system (CITS) which dynamically predicts and adapts to a student's learning style throughout the tutoring conversation. Oscar aims to mimic a human tutor to improve the effectiveness of the learning experience by leading a natural language tutorial and adapting material to suit an individual's learning style. Prediction of learning style is undertaken through capturing independent variables during the conversation. The variable with the highest value determines the individuals learning style. This paper proposes a new method which uses a fuzzy classification tree to build a fuzzy predictive model using these variables which are captured through natural language dialogue Experiments have been undertaken on two of the learning style dimensions: perception (sensory-intuitive) and understanding (sequential-global). Early results show the model has substantially increased the predictive accuracy of the Oscar CITS and discovered some interesting relationships amongst these variables.

This paper presents a new approach to combining multiple fuzzy regression trees, which are induced by applying the modified Elgasir fuzzy regression tree algorithm. This method utilises Trapezoidal membership functions for fuzzification and the Takagi-Sugeno fuzzy inference to obtain the final predicted values. A modified version of Artificial Immune Network model (opt-aiNet) is used for the simultaneous optimization of the membership functions across all trees within the forest. Boston housing and Abalone are two real-world datasets from the UCI repository used to evaluate the proposed approach. The empirical results have showed that fuzzy regression tree forests reduce the error rate compared with single fuzzy regression tree.

Academics who wish to commercialise a novel research concept or prototype `product' face many difficult challenges. These typically include limited business knowledge, the technical verses business language barrier, and the time commitment involved in trying to “sell” the idea to industry and develop one-off prototypes. This paper describes the commercialisation of Goal Orientated Conversational Agents (GO-CA) from initial research concepts through to establishing a UK limited company. GO-CAs have the ability to converse with humans through the use of natural language dialogue in order to achieve a specific task. Such systems are ideal for providing clear and consistent advice 24 hours a day and allow the users to be able to ask questions in natural language which are relevant to their personal situation. The paper briefly describes the novel GO-CA architecture and outlines two developed applications. The pathway to commercialisation which was taken is described and lessons which were learnt are identified.

Fuzzy regression trees are a generalization of the standard artificial intelligence technique of regression trees. The Elgasir algorithm has previously been used to create fuzzy regression trees in order to improve the performance of crisp regression trees. A weakness of this approach was that no optimisation of tree node membership functions took place. Artificial Immune Systems are an evolutionary methodology, inspired by the principles and processes of the natural immune system. In this paper a novel method based on an optimization version of Artificial Immune Network model (opt-aiNet) is used to optimize the Elgasir algorithm. In order to illustrate the prediction accuracy of the proposed method, two problem datasets from the UCI repository are used to evaluate the approach. Experimental results have shown the effectiveness of using opt-aiNet for optimization Elgasir algorithm by increasing the prediction accuracy and robustness of fuzzy regression trees.

This paper presents the development and commercialisation of a computerised, non-invasive psychological profiling system (named `Silent Talker'), for the analysis of nonverbal behaviour. Nonverbal signals hold rich information about mental, behavioural and/or physical states the detection of which would be beneficial in a wide variety of commercial domains, particularly deception detection and security. Previous attempts to extract individual signals and classify an overall behaviour have been overly complex, time-consuming, costly, biased and error prone. Silent Talker overcomes these problems by the use of Artificial Neural Networks. The testing and validation of the system was undertaken by detecting processes associated with `deception' and `truth' which the system achieved with significant results. Initial academic recognition led to widespread publicity but the commercialisation of the project proved to be far from straightforward. This paper describes the system itself, its potential uses and the commercialisation process to date, concluding with lessons that have been learnt along the way.

This paper presents a novel Semantic-Based Conversational Agent (CA) and its application to benefit students with debt who might otherwise face serious consequences, not only financially, but psychologically. The novel semantic-based CA interprets scripts consisting of natural language sentences by means of a sentence similarity measure. User input is measured semantically against the content of the scripts in order to respond with an appropriate output string. Scripts are organized into contexts consisting of rules possessing a select number of natural language sentences. Such scripting is effortless and alleviates the burden of the traditional pattern-scripted methodologies. The novel semantic-based CA has promising results following interaction with participants focused on the sensitive domain of student debt management - an issue of growing concern.

Neural networks are a powerful classification technique that are capable of discovering complex relationships between inputs and outputs. This powerful classification ability has meant that neural networks have been widely applied to medical domains. However, neural networks suffer from the so-called blackbox problem: they predict accurately but offer no explanation of how the decision has been derived. This lack of explanation has limited their adoption and has been a concern to the medical community. This paper demonstrates ExTree a rule extraction algorithm being applied to neural networks which have been trained on medical datasets. ExTree extracts from the neural network a decision tree which is a graphical and easily understood representation of a decision process.

This paper presents the application of a novel fuzzy regression trees technique to real-world regression problems. Elgasir algorithm is a fuzzy regression trees technique applied to crisp regression trees in order to overcome the problems of sharp decision boundaries. Fuzzy regression trees are induced by applying Elgasir algorithm to crisp CHAID regression trees based on Trapezoidal membership functions and Takagi-Sugeno fuzzy inference. Elgasir algorithm associated with artificial immune system are used to induce the optimized version of Elgasir algorithm. The Elevators and Compactiv are two real-world datasets from KEEL repository used to perform empirical evaluation for the proposed method. The Elevators dataset has been retrieved from the task of controlling a F16 aircraft. The Compactiv is computer Activity dataset. The empirical results showed show the capability of Elgasir optimized to produce robust fuzzy regression trees.

This paper presents a novel Semantic-Based Conversational Agent (CA) using memory simulation to create self-awareness and character. Semantics has been identified as an important memory system within human cognition. The proposed semantic-based CA interprets scripts consisting of natural language sentences by means of a sentence similarity measure. User input is measured semantically against the content of the scripts in order to respond with an appropriate output string. Scripts are organized into contexts consisting of rules possessing a select number of natural language sentences. Through the application of semantics, a mechanism to human-like memory simulation is proposed. Currently, traditional CAs employ very primitive memory mechanisms. Such CAs use only the structural form of a sentence and take no consideration of semantic content. The semantic-based CA has produced results with mimicry akin to that of an autobiographical memory-base, thus highlighting its potential application to memory simulation.

Fuzzy decision trees have been successfully applied to both classification and regression problems by allowing gradual transitions to exist between attribute values. Methodologies for fuzzification in fuzzy trees currently create such gradual transitions for continuous attributes. This is achieved by automatically creating fuzzy regions around tree nodes using an optimization algorithm or by using the knowledge of a human expert to create a series of fuzzy sets which are representative of the attributes domain. A problem occurs when trying to construct a fuzzy tree from real world data which comprises of only discrete or a mixture of discrete and continuous attributes. Discrete attribute values have no proximity to other values in the decision space, as there is no continuum between values. Consequently, within a fuzzy tree they are interpreted as crisp sets and contribute little towards the final outcome. This paper proposes a new approach for the fuzzification of discrete attributes in fuzzy decision trees. The approach ranks discrete values on the basis of their effect on the outcome rate and assigns a possibility of being a specific outcome. Experiments carried out on two real world financial datasets which contain a significant proportion of discrete attributes show improved classification accuracy compared with a crisp interpretation of such attributes within fuzzy trees.

This paper presents a new fuzzy regression tree algorithm known as Elgasir, which is based on the CHAID regression tree algorithm and Takagi-Sugeno fuzzy inference. The Elgasir algorithm is applied to crisp regression trees to produce fuzzy regression trees in order to soften sharp decision boundaries inherited in crisp trees. Elgasir generates a fuzzy rule base by applying fuzzy techniques to crisp regression trees using trapezoidal membership functions. Then Takagi-Sugeno fuzzy inference is used to aggregate the final output from the fuzzy implications. The approach is evaluated using two problem sets from the UCI repository. Experiments conducted yield an improvement in the performance of fuzzy regression trees compared with crisp CHAID trees. The generated fuzzy regression trees are more robust and presented in a highly visual format which is easy to understand.

The success of fuzzy decision trees when applied to classification problems is usually attributed to the selection and tuning of fuzzy sets to represent the problem domain. The impact of fuzzy inference in combining grades of membership throughout fuzzy trees has not been considered in-depth. A number of parameterized fuzzy operators based on the T-norm model have been proposed but not exploited in practical applications. This paper presents a comparative study which examines a number of T-norm and T-conorms and their application within Fuzzy Decision Trees. The methodology uses a Genetic Algorithm to tune the weights of T-norm operators and optimize fuzzy membership functions simultaneously in fuzzy trees. The paper applies the methodology to two Fuzzy Decision Tree algorithms known as FIA and Fuzzy CHAIRS. Six different T-norm models are investigated across five real world datasets. Experimental results indicate that significant improvements can be made in the performance of fuzzy trees when the most appropriate T-norm is optimised for a specific domain.

Typically regression trees are not capable of generating a continuous output as a function of inputs and are not suitable for modeling process control problems. Fuzzy logic controllers offer an alternative approach to modeling such process efficiently but often rely on human experts to articulate a set of rules. The generation of fuzzy rules by experts is difficult and based on the subjective perception of individuals. This paper presents a novel approach to modeling process control applications using fuzzy regression trees. Two sets of experiments were designed and conducted. The first involved measuring the output gradient of the fuzzy regression tree to investigate the applicability of fuzzy regression trees in modelling process control applications. The second assessed the efficiency of the fuzzy model produced. The results have shown that an effective fuzzy model can be produced and as a consequence a considerable improvement in the performance can be achieved.

Artificial neural networks (ANNs) are a powerful and widely used pattern recognition technique. However, they remain "black boxes" giving no explanation for the decisions they make. This paper presents a new algorithm for extracting a logistic model tree (LMT) from a neural network, which gives a symbolic representation of the knowledge hidden within the ANN. Landwehr's LMTs are based on standard decision trees, but the terminal nodes are replaced with logistic regression functions. This paper reports the results of an empirical evaluation that compares the new decision tree extraction algorithm with Quinlan's C4.5 and ExTree. The evaluation used 12 standard benchmark datasets from the University of California, Irvine machine-learning repository. The results of this evaluation demonstrate that the new algorithm produces decision trees that have higher accuracy and higher fidelity than decision trees created by both C4.5 and ExTree.

This paper investigates a new approach to creating robust fuzzy classifiers that are impartial to the imbalance problem within data sets. The approach uses raw real-world data without the need for sampling or the creation of synthetic examples. The aim is to achieve common currency between the actual classification accuracy and the distribution of this accuracy between the outcome classes. The proposed method first uses a fuzzy inference algorithm (FIA) to construct a fuzzy classifier from a crisp C4.5. A genetic algorithm (GA) is then used to optimize the degree of fuzziness within the classifier. The GA's fitness function consists of two components: classification accuracy and the distribution (or balance) of this accuracy between the outcomes. Both components are optimised concurrently. Four alternative fitness functions are defined, each of which applies different penalties on the classification accuracy depending on a weighting associated with the balance component. The method is then applied to three real world data sets. The results show that it is possible to attain a fuzzy classifier which exhibits both good performance and balance between outcomes regardless of any imbalance within the data set.

Abstract: In generating a suitable fuzzy classifier system, significant effort is often placed on the determination and the fine tuning of the fuzzy sets. However, in such systems little thought is given to the way in which membership functions are combined within the fuzzy rules. Often traditional fuzzy inference strategies are used which consequently provide no control over how strongly or weakly the inference is applied within these rules. Furthermore such strategies will allow no interaction between grades of membership. A number of theoretical fuzzy inference operators have been proposed for both regression and classification problems but they have not been investigated in the context of real-world applications. In this paper we propose a novel genetic algorithm framework for optimizing the strength of fuzzy inference operators concurrently with the tuning of membership functions for a given fuzzy classifier system. Each fuzzy system is generated using two well-established decision tree algorithms: C4.5 and CHAID. This will enable both classification and regression problems to be addressed within the framework. Each solution generated by the genetic algorithm will produce a set of fuzzy membership functions and also determine how strongly the inference will be applied within each fuzzy rule. We investigate several theoretical proven fuzzy inference techniques (T-norms) in the context of both classification and regression problems. The methodology proposed is applied to a number of real-world data sets in order to determine the effects of the simultaneous tuning of membership functions and inference parameters on the accuracy and robustness of fuzzy classifiers.

Sentence similarity measures play an increasingly important role in text-related research and applications in areas such as text mining, Web page retrieval, and dialogue systems. Existing methods for computing sentence similarity have been adopted from approaches used for long text documents. These methods process sentences in a very high-dimensional space and are consequently inefficient, require human input, and are not adaptable to some application domains. This paper focuses directly on computing the similarity between very short texts of sentence length. It presents an algorithm that takes account of semantic information and word order information implied in the sentences. The semantic similarity of two sentences is calculated using information from a structured lexical database and from corpus statistics. The use of a lexical database enables our method to model human common sense knowledge and the incorporation of corpus statistics allows our method to be adaptable to different domains. The proposed method can be used in a variety of applications that involve text knowledge representation and discovery. Experiments on two sets of selected sentence pairs demonstrate that the proposed method provides a similarity measure that shows a significant correlation to human intuition

We present a hybrid radial basis function (RBF) sigmoid neural network with a three-step training algorithm that utilizes both global search and gradient descent training. The algorithm used is intended to identify global features of an input-output relationship before adding local detail to the approximating function. It aims to achieve efficient function approximation through the separate identification of aspects of a relationship that are expressed universally from those that vary only within particular regions of the input space. We test the effectiveness of our method using five regression tasks; four use synthetic datasets while the last problem uses real-world data on the wave overtopping of seawalls. It is shown that the hybrid architecture is often superior to architectures containing neurons of a single type in several ways: lower mean square errors are often achievable using fewer hidden neurons and with less need for regularization. Our global-local artificial neural network (GL-ANN) is also seen to compare favorably with both perceptron radial basis net and regression tree derived RBFs. A number of issues concerning the training of GL-ANNs are discussed: the use of regularization, the inclusion of a gradient descent optimization step, the choice of RBF spreads, model selection, and the development of appropriate stopping criteria.

This paper introduces a novel algorithm which applies the theories of fuzzification in order to fuzzify decision trees for solving classification problems with numeric outcomes. The CHAID algorithm is a highly efficient statistical technique for segmentation, or tree growing. The application of fuzzy logic to pre-generated CHAID decision trees can represent classification knowledge more naturally and in-line with human thinking. Using a genetic algorithm (GA), various sized fuzzy regions are optimised from a training set and are applied to all decision nodes within the tree. A new case passing through the tree results in a membership grade being generated at each branch. Four different fuzzy inference mechanisms, also optimised by the GA, are used to investigate the degree of interaction between membership grades on each specific decision path. A modified approach to Mamdani's inference is also proposed to manage the defuzzification of numeric tree outcomes. Initial comparisons between crisp and fuzzified CHAID trees show that the fuzzy tree is more robust and produces a more balanced classification leading to improved decision making

Semantic similarity between words is becoming a generic problem for many applications of computational linguistics and artificial intelligence. This paper explores the determination of semantic similarity by a number of information sources, which consist of structural semantic information from a lexical taxonomy and information content from a corpus. To investigate how information sources could be used effectively, a variety of strategies for using various possible information sources are implemented. A new measure is then proposed which combines information sources nonlinearly. Experimental evaluation against a benchmark set of human similarity ratings demonstrates that the proposed measure significantly outperforms traditional similarity measures.

This paper presents a novel approach to combining multiple decision trees, which utilizes the power of fuzzy inference techniques and a back-propagation feed forward neural network (BP-FFNN) to improve the overall classification. Crisp multiple decision trees were originally introduced to improve the performance of single decision trees, which offered a restrictive view of the domain. By combining multiple perspectives of the same domain, the information content is increased and the predictive power of the classifier is improved. A predominant weakness in creating multiple crisp trees is the generation of sharp decision boundaries at every node within all trees. The creation of fuzzy decision forests overcomes this weakness by introducing the concepts of fuzzy theory to soften the decision boundaries. To combine information from all fuzzy trees a unique combination of fuzzy inference is used to generate a series of leaf membership grades, which are then used as input to a BP-FFNN. Comparisons are made between using the fuzzy-neural approach and the use of pure fuzzy inference trees and the results indicate that considerable improvement has been made over crisp multiple decision trees.

The creation of multiple decision trees is a relatively new concept, which aims to improve the predictive power of a single decision tree. The approach is based on the induction of more than one C4.5-type decision tree from the same training sample where each decision tree represents a different view of the same domain resulting in a network of decision tree models. The utilization of multiple decision trees has been shown to lead to an improved performance by combining multiple perspectives of the same domain thus increasing the information content whereas, in comparison, a single decision tree can only represent one restricted view of the domain. One predominant weakness in creating a single tree is the generation of sharp decision boundaries at every node within the tree, which results in small changes in attribute values giving radically different classifications. This problem becomes more apparent with the generation of multiple trees. This paper presents a novel approach of overcoming this weakness through the use of fuzzy decision forests. The approach is based upon the induction of multiple fuzzy decision trees from one training sample, where each tree represents a different view of the data domain. A genetic algorithm (GA) is used to select a series of high performance membership functions, which are then applied to branches within all decision trees in the forest. The GA will in addition optimise a pre-selected fuzzy inference technique, which will assign a degree of strength to the conjunction and disjunction of membership grades within the tree. Considerable improvements in classification accuracy over original single C4.5 (crisp) trees were obtained using two real world data sets.

This paper investigates the fuzzification of crisp decision trees using nonlinear membership functions to soften sharp decision boundaries. A novel nonlinear fuzzy algorithm provides the framework for the investigation of four different membership functions. Using a genetic algorithm (GA), various sized fuzzy regions are optimised from a training set and are applied to all decision nodes. A new case passing through the tree will result in a membership grade being generated at each branch. Three different fuzzy inference mechanisms, also optimised by the GA, are used to investigate the degree of interaction between membership grades on each specific decision path. Initial comparisons between crisp trees and the fuzzified trees show that the fuzzy tree is more robust and produces a more balanced classification leading to improved decision-making.

A network's generalisation is extrapolated from the training examples, i.e. it is defined by the network's model of the training data. Simplifying this model will increase the resistance to noise, inherent in the training data, but will also lose information which may not be noise. This paper discusses the use of Voronoi tessellations for highly detailed data models and the effects of noise. An original algorithm is developed for constructing a competitive layer of k-nearest neighbour type neurons. These neurons are trained to model the decision boundaries and therefore approximate the perfect bisector that is required. The new algorithm is empirically compared with three existing hyperplanic algorithms-neural tree network (NTN), backpropagation (BP), and the entropy net, on three real world continuous data sets.