If knowledge such as classification rules are extracted from sample data in a distributed way, it may be necessary to combine or fuse these rules. In a conventional approach this would typically be done either by combining the classifiers' outputs (e.g., in form of a classifier ensemble) or by combining the sets of classification rules (e.g., by weighting them individually). In this paper, we introduce a new way of fusing classifiers at the level of parameters of classification rules. This technique is based on the use of probabilistic generative classifiers using multinomial distributions for categorical input dimensions and multivariate normal distributions for the continuous ones. That means, we have distributions such as Dirichlet or normal-Wishart distributions over parameters of the classifier. We refer to these distributions as hyperdistributions or second-order distributions. We show that fusing two (or more) classifiers can be done by multiplying the hyperdistributions of the parameters and derive simple formulas for that task. Properties of this new approach are demonstrated with a few experiments. The main advantage of this fusion approach is that the hyperdistributions are retained throughout the fusion process. Thus, the fused components may, for example, be used in subsequent training steps (online training).

Intelligent agents often have the same or similar tasks and sometimes they cooperate to solve a given problem. These agents typically know how to observe their local environment and how to react on certain observations, for instance, and this knowledge may be represented in form of rules. However, many environments are dynamic in the sense that from time to time novel rules are required or old rules become obsolete. In this article we propose and investigate new techniques for knowledge acquisition by novelty detection and reaction as well as obsoleteness detection and reaction that an agent may use for self-adaptation to new situations. For that purpose we consider classifiers based on probabilistic rules. Premises of new rules are learned autonomously while conclusions are either obtained from human experts or from other agents which have learned appropriate rules in the past. By means of knowledge exchange, agents will efficiently be enabled to cope with situations they were not confronted with before. This kind of collaborative intelligence follows the human archetype: Humans are able to learn from each other by communicating learned rules. We demonstrate some properties of the knowledge acquisition techniques using artificial data as well as data from the field of intrusion detection.

In this article, we provide a new technique for temporal data mining which is based on classification rules that can easily be understood by human domain experts. Basically, time series are decomposed into short segments, and short-term trends of the time series within the segments (e.g., average, slope, and curvature) are described by means of polynomial models. Then, the classifiers assess short sequences of trends in subsequent segments with their rule premises. The conclusions gradually assign an input to a class. As the classifier is a generative model of the processes from which the time series are assumed to originate, anomalies can be detected, too. Segmentation and piecewise polynomial modeling are done extremely fast in only one pass over the time series. Thus, the approach is applicable to problems with harsh timing constraints. We lay the theoretical foundations for this classifier, including a new distance measure for time series and a new technique to construct a dynamic classifier from a static one, and demonstrate its properties by means of various benchmark time series, for example, Lorenz attractor time series, energy consumption in a building, or ECG data.

For classification tasks, the application of generative classifiers sometimes has advantages over the use of exclusively discriminative classifiers because loss functions can be considered or rejection criteria can be defined more easily, for instance. We show how a radial basis function (RBF) network with multivariate (elliptical) Gaussian basis functions can be trained in two different ways to obtain a classifier with either a more generative or a more discriminative behavior. Our generative classifier allows a probabilistic interpretation of the external outputs (posterior probability of class membership) and the hidden neurons' activations (posterior probability of a component of the model). For that purpose a variational Bayesian inference approach is applied, which also finds an appropriate number of hidden neurons (i.e., components) ldquoon the flyrdquo. A discriminative classifier is obtained using the resilient propagation training technique. We investigate the properties of the two training techniques in detail by introducing a measure for generative properties of the trained classifiers and by comparing these classifiers on various data sets.