Large Language Models (LLMs) have demonstrated remarkable performance across various tasks but remain prone to hallucinations.Detecting hallucinations is essential for safetycritical applications, and recent methods leverage attention map properties to this end, though their effectiveness remains limited.In this work, we investigate the spectral features of attention maps by interpreting them as adjacency matrices of graph structures.We propose the LapEigvals method, which utilizes the topk eigenvalues of the Laplacian matrix derived from the attention maps as an input to hallucination detection probes.Empirical evaluations demonstrate that our approach achieves stateof-the-art hallucination detection performance among attention-based methods.Extensive ablation studies further highlight the robustness and generalization of LapEigvals, paving the way for future advancements in the hallucination detection domain.

Large language models (LLMs) have revolutionized natural language processing, yet their tendency to hallucinate poses serious challenges for reliable deployment.Despite numerous hallucination detection methods, their evaluations often rely on ROUGE, a metric based on lexical overlap that misaligns with human judgments.Through comprehensive human studies, we demonstrate that while ROUGE exhibits high recall, its extremely low precision leads to misleading performance estimates.In fact, several established detection methods show performance drops of up to 45.9% when assessed using human-aligned metrics like LLM-as-Judge.Moreover, our analysis reveals that simple heuristics based on response length can rival complex detection techniques, exposing a fundamental flaw in current evaluation practices.We argue that adopting semantically aware and robust evaluation frameworks is essential to accurately gauge the true performance of hallucination detection methods, ultimately ensuring the trustworthiness of LLM outputs.

Traditional machine-learning approaches face limitations when confronted with insufficient data. Transfer learning addresses this by leveraging knowledge from closely related domains. The key in transfer learning is to find a transferable feature representation to enhance cross-domain classification models. However, in some scenarios, some features correlated with samples in the source domain may not be relevant to those in the target. Causal inference enables us to uncover the underlying patterns and mechanisms within the data, mitigating the impact of confounding factors. Nevertheless, most existing causal inference algorithms have limitations when applied to high-dimensional datasets with nonlinear causal relationships. In this work, a new causal representation method based on a Graph autoencoder embedded AutoEncoder, named GeAE, is introduced to learn invariant representations across domains. The proposed approach employs a causal structure learning module, similar to a graph autoencoder, to account for nonlinear causal relationships present in the data. Moreover, the cross-entropy loss as well as the causal structure learning loss and the reconstruction loss are incorporated in the objective function designed in a united autoencoder. This method allows for the handling of high-dimensional data and can provide effective representations for cross-domain classification tasks. Experimental results on generated and real-world datasets demonstrate the effectiveness of GeAE compared with the state-of-the-art methods.

In this work, we study the unsupervised robust domain adaptation problem where only a single well labeled source domain data is available during the learning process. A new causal representation method based on a Graph autoen-coder embedded AutoEncoder, named GeAE, is introduced to learn invariant representations across domains for robust domain adaption. The proposed method can handle nonlinear causal relations included in the data by a causal structure learning process similar to a graph autoencoder. Moreover, the cross-entropy loss as well as the causal structure loss and the reconstruction loss are incorporated in the objective function designed in a united autoencoder to improve the quality of predictions using causal representations. Experimental results on one generated dataset and three real-world datasets demonstrate the effectiveness of GeAE in comparison with the state-of-the-art methods.

This article proposes a simple yet powerful ensemble classifier, called Random Hyperboxes, constructed from individual hyperbox-based classifiers trained on the random subsets of sample and feature spaces of the training set. We also show a generalization error bound of the proposed classifier based on the strength of the individual hyperbox-based classifiers as well as the correlation among them. The effectiveness of the proposed classifier is analyzed using a carefully selected illustrative example and compared empirically with other popular single and ensemble classifiers via 20 datasets using statistical testing methods. The experimental results confirmed that our proposed method outperformed other fuzzy min-max neural networks (FMNNs), popular learning algorithms, and is competitive with other ensemble methods. Finally, we identify the existing issues related to the generalization error bounds of the real datasets and inform the potential research directions.

In this article, a novel machine learning (ML)-derived control performance assessment (CPA) classification system is proposed. It is dedicated for a wide class of PID-based control industrial loops with processes exhibiting dynamical properties close to second order plus delay time (SOPDT). The proposed concept is very general and easy to configure to distinguish between acceptable and poor closed-loop performance. This approach allows for determining the best (but also robust and practically achievable) closed-loop performance based on very popular and intuitive closed-loop quality factors. Training set can be automatically derived off-line using a number of different, diverse control performance indices (CPIs) used as discriminative features of the assessed control system. The proposed extended set of CPIs is discussed with comprehensive performance assessment of different ML-based classification methods and practical application of the suggested solution. As a result, a general-purpose CPA system is derived that can be immediately applied in practice without any preliminary or additional learning stage during normal closed-loop operation. It is verified by practical application to assess the control system for a laboratory heat exchange and distribution setup.

Neural architecture search (NAS) has attracted a lot of attention and has been illustrated to bring tangible benefits in a large number of applications in the past few years. Architecture topology and architecture size have been regarded as two of the most important aspects for the performance of deep learning models and the community has spawned lots of searching algorithms for both of those aspects of the neural architectures. However, the performance gain from these searching algorithms is achieved under different search spaces and training setups. This makes the overall performance of the algorithms incomparable and the improvement from a sub-module of the searching model unclear. In this paper, we propose NATS-Bench, a unified benchmark on searching for both topology and size, for (almost) any up-to-date NAS algorithm. NATS-Bench includes the search space of 15,625 neural cell candidates for architecture topology and 32,768 for architecture size on three datasets. We analyze the validity of our benchmark in terms of various criteria and performance comparison of all candidates in the search space. We also show the versatility of NATS-Bench by benchmarking 13 recent state-of-the-art NAS algorithms on it. All logs and diagnostic information trained using the same setup for each candidate are provided. This facilitates a much larger community of researchers to focus on developing better NAS algorithms in a more comparable and computationally effective environment. All codes are publicly available at: https://xuanyidong.com/assets/projects/NATS-Bench.

Machine learning (ML) pipeline composition and optimisation have been studied to seek multi-stage ML models' i.e. preprocessor-inclusive, that are both valid and well-performing. These processes typically require the design and traversal of complex configuration spaces consisting of not just individual ML components and their hyperparameters, but also higher-level pipeline structures that link these components together. Optimisation efficiency and resulting ML-model accuracy both suffer if this pipeline search space is unwieldy and excessively large; it becomes an appealing notion to avoid costly evaluations of poorly performing ML components ahead of time. Accordingly, this paper investigates whether, based on previous experience, a pool of available classifiers/regressors can be preemptively culled ahead of initiating a pipeline composition/opti-misation process for a new ML problem, i.e. dataset. The previous experience comes in the form of classifier/regressor accuracy rankings derived, with loose assumptions, from a substantial but non-exhaustive number of pipeline evaluations; this meta-knowledge is considered ‘opportunistic’. Numerous experiments with the AutoWeka4MCPS package, including ones leveraging similarities between datasets via the relative landmarking method, show that, despite its seeming unreliability, opportunistic meta-knowledge can improve ML outcomes. However, results also indicate that the culling of classifiers/regressors should not be too severe either. In effect, it is better to search through a ‘top tier’ of recommended predictors than to pin hopes onto one previously supreme performer.

Motivated by the practical demands for simplification of data toward being consistent with human thinking and problem-solving, as well as tolerance of uncertainty, information granules are becoming important entities in data processing at different levels of data abstraction. This article proposes a method to construct classifiers from multiresolution hierarchical granular representations using hyperbox fuzzy sets. The proposed approach forms a series of granular inferences hierarchically through many levels of abstraction. An attractive characteristic of our classifier is that it can maintain a high accuracy in comparison to other fuzzy min-max models at a low degree of granularity based on reusing the knowledge learned from lower levels of abstraction. In addition, our approach can reduce the data size significantly as well as handle the uncertainty and incompleteness associated with data in real-world applications. The construction process of the classifier consists of two phases. The first phase is to formulate the model at the greatest level of granularity, while the later stage aims to reduce the complexity of the constructed model and deduce it from data at higher abstraction levels. Experimental analyses conducted comprehensively on both synthetic and real datasets indicated the efficiency of our method in terms of training time and predictive performance in comparison to other types of fuzzy min-max neural networks and common machine learning algorithms.

The evaluation of machine learning (ML) pipelines is essential during automatic ML pipeline composition and optimisation. The previous methods such as Bayesian-based and genetic-based optimisation, which are implemented in Auto-Weka, Auto-sklearn and TPOT, evaluate pipelines by executing them. Therefore, the pipeline composition and optimisation of these methods requires a tremendous amount of time that prevents them from exploring complex pipelines to find better predictive models. To further explore this research challenge, we have conducted experiments showing that many of the generated pipelines are invalid, and it is unnecessary to execute them to find out whether they are good pipelines. To address this issue, we propose a novel method to evaluate the validity of ML pipelines using a surrogate model (AVATAR). The AVATAR enables to accelerate automatic ML pipeline composition and optimisation by quickly ignoring invalid pipelines. Our experiments show that the AVATAR is more efficient in evaluating complex pipelines in comparison with the traditional evaluation approaches requiring their execution.

This paper proposes an improved version of the current online learning algorithm for a general fuzzy min-max (GFMM) neural network to tackle existing issues concerning expansion and contraction steps as well as the way of dealing with unseen data located on decision boundaries. These drawbacks lower its classification performance, so an improved algorithm is proposed in this study to address the above limitations. The proposed approach does not use the contraction process for overlapping hyperboxes, which is more likely to increase the error rate as shown in the literature. The empirical results indicated the improvement in the classification accuracy and stability of the proposed method compared to the original version and other fuzzy min-max classifiers. In order to reduce the sensitivity to the training samples presentation order of this new on-line learning algorithm, a simple ensemble method is also proposed.

Cross-platform account matching plays a significant role in social network analytics, and is beneficial for a wide range of applications. However, existing methods either heavily rely on high-quality user generated content (including user profiles) or suffer from data insufficiency problem if only focusing on network topology, which brings researchers into an insoluble dilemma of model selection. In this paper, to address this problem, we propose a novel framework that considers multi-level graph convolutions on both local network structure and hypergraph structure in a unified manner. The proposed method overcomes data insufficiency problem of existing work and does not necessarily rely on user demographic information. Moreover, to adapt the proposed method to be capable of handling large-scale social networks, we propose a two-phase space reconciliation mechanism to align the embedding spaces in both network partitioning based parallel training and account matching across different social networks. Extensive experiments have been conducted on two large-scale real-life social networks. The experimental results demonstrate that the proposed method outperforms the state-of-the-art models with a big margin.

Composition and parameterization of multicomponent predictive systems (MCPSs) consisting of chains of data transformation steps are a challenging task. Auto-WEKA is a tool to automate the combined algorithm selection and hyperparameter (CASH) optimization problem. In this paper, we extend the CASH problem and Auto-WEKA to support the MCPS, including preprocessing steps for both classification and regression tasks. We define the optimization problem in which the search space consists of suitably parameterized Petri nets forming the sought MCPS solutions. In the experimental analysis, we focus on examining the impact of considerably extending the search space (from approximately 22000 to 812 billion possible combinations of methods and categorical hyperparameters). In a range of extensive experiments, three different optimization strategies are used to automatically compose MCPSs for 21 publicly available data sets. The diversity of the composed MCPSs found is an indication that fully and automatically exploiting different combinations of data cleaning and preprocessing techniques is possible and highly beneficial for different predictive models. We also present the results on seven data sets from real chemical production processes. Our findings can have a major impact on the development of high-quality predictive models as well as their maintenance and scalability aspects needed in modern applications and deployment scenarios. Note to Practitioners-The extension of Auto-WEKA to compose and optimize multicomponent predictive systems (MCPSs) developed as part of this paper is freely available on GitHub under GPL license, and we encourage practitioners to use it on a broad variety of classification and regression problems. The software can either be used as a blackbox-where search space is made of all possible WEKA filters, predictors, and metapredictors (e.g., ensembles)-or as an optimization tool on a subset of preselected machine learning methods. The application has a graphical user interface, but it can also run from command line and can be embedded in any project as a Java library. There are three main outputs once an Auto-WEKA run has finished: 1) the trained MCPS ready to make predictions on unseen data; 2) the WEKA configuration (i.e., parameterized components); and 3) the Petri net in a Petri Net Markup Language format that can be analyzed using any tool supporting this standard language. There are, however, some practical considerations affecting the quality of the results that must be taken into consideration, such as the CPU time budget or the search starting point. These are extensively discussed in this paper.

There are many algorithms that can be used for the time-series forecasting problem, ranging from simple (e.g. Moving Average) to sophisticated Machine Learning approaches (e.g. Neural Networks). Most of these algorithms require a number of user-defined parameters to be specified, leading to exponential explosion of the space of potential solutions. Since the trial-and-error approach to finding a good algorithm for solving a given problem is typically intractable, researchers and practitioners need to resort to a more intelligent search strategy, with one option being to constraint the search space using past experience - an approach known as Meta-learning. Although potentially attractive, Meta-learning comes with its own challenges. Gathering a sufficient number of Meta-examples, which in turn requires collecting and processing multiple datasets from each problem domain under consideration is perhaps the most prominent issue. In this paper, we are investigating the situations in which the use of additional data can improve performance of a Meta-learning system, with focus on cross-domain transfer of Meta-knowledge. A similarity-based cluster analysis of Meta-features has also been performed in an attempt to discover homogeneous groups of time-series with respect to Meta-learning performance. Although the experiments revealed limited room for improvement over the overall best base-learner, the Meta-learning approach turned out to be a safe choice, minimizing the risk of selecting the least appropriate base-learner.

In social network analysis , community detection is a basic step to understand the structure and function of networks. Some conventional community detection methods may have limited performance because they merely focus on the networks’ topological structure . Besides topology, content information is another significant aspect of social networks. Although some state-of-the-art methods started to combine these two aspects of information for the sake of the improvement of community partitioning, they often assume that topology and content carry similar information. In fact, for some examples of social networks, the hidden characteristics of content may unexpectedly mismatch with topology. To better cope with such situations, we introduce a novel community detection method under the framework of non-negative matrix factorization (NMF). Our proposed method integrates topology as well as content of networks and has an adaptive parameter (with two variations) to effectively control the contribution of content with respect to the identified mismatch degree. Based on the disjoint community partition result, we also introduce an additional overlapping community discovery algorithm, so that our new method can meet the application requirements of both disjoint and overlapping community detection. The case study using real social networks shows that our new method can simultaneously obtain the community structures and their corresponding semantic description , which is helpful to understand the semantics of communities. Related performance evaluations on both artificial and real networks further indicate that our method outperforms some state-of-the-art methods while exhibiting more robust behavior when the mismatch between topology and content is observed.

Link prediction in social networks is a very challenging research problem. The majority of existing approaches are based on the assumption that a given network evolves following a single phenomenon, e.g. "rich get richer" or "friend of my friend is my friend". However, dynamics of network dynamic changes over time and different parts of the network evolve in different manner. Because of that, we hypothesise that the prediction accuracy can be improved by providing different treatment to different nodes and links. Building on that assumption, we propose a Community Bridge Boosting Prediction Model (CBBPM) that treats certain bridge nodes differently depending on their structural position. For such bridge nodes their similarity score obtained using traditional link-based prediction methods is boosted. By doing so the importance of these nodes is increased and at the same time ensuring that the CBBPM can be used with any existing link prediction method. Our experimental results show that such bridge node similarity boosting mechanism can improve the accuracy of traditional link prediction methods.

In social network analysis, community detection is a basic step to understand the structure, function and semantics of networks. Some conventional community detection methods may have limited performance because they merely focus on topological structure of networks. In addition to topology, content information is another significant aspect of social networks. Some state-of-the-art methods started to combine these two aspects of information, but they often assume that topology and content share the same characteristics. However, for some examples of social networks, content may mismatch with topological structure. In order to better cope with such situations, we introduce a novel community detection method under the framework of non-negative matrix factorization (NMF). Our proposed method integrates topology and content of networks, and introduces a novel adaptive parameter for controlling the contribution of content with respect to the identified mismatch degree between the topological and content information. The case study using real social networks show that our new method can simultaneously obtain community partition and the corresponding semantic descriptions. Experiments on both artificial networks and real social networks further indicate that our method outperforms some state-of-the-art methods while exhibiting more robust behaviour when the mismatch topological and content information is observed.

Abstract Large service companies run complex customer service processes to provide communication services to their customers. The flawless execution of these processes is essential because customer service is an important differentiator. They must also be able to predict if processes will complete successfully or run into exceptions in order to intervene at the right time, preempt problems and maintain customer service. Business process data are sequential in nature and can be very diverse. Thus, there is a need for an efficient sequential forecasting methodology that can cope with this diversity. This paper proposes two approaches, a sequential k nearest neighbour and an extension of Markov models both with an added component based on sequence alignment. The proposed approaches exploit temporal categorical features of the data to predict the process next steps using higher order Markov models and the process outcomes using sequence alignment technique. The diversity aspect of the data is also added by considering subsets of similar process sequences based on k nearest neighbours. We have shown, via a set of experiments, that our sequential k nearest neighbour offers better results when compared with the original ones; our extension Markov model outperforms random guess, Markov models and hidden Markov models.

Internet of Things (IoT) has given rise to the fourth industrial revolution (Industrie 4.0), and it brings great benefits by connecting people, processes and data. However, cybersecurity has become a critical challenge in the IoT enabled cyber physical systems, from connected supply chain, Big Data produced by huge amount of IoT devices, to industry control systems. Evolutionary computation combining with other computational intelligence will play an important role for cybersecurity, such as artificial immune mechanism for IoT security architecture, data mining/fusion in IoT enabled cyber physical systems, and data driven cybersecurity. This paper provides an overview of security challenges in IoT enabled cyber-physical systems and what evolutionary computation and other computational intelligence technology could contribute for the challenges. The overview could provide clues and guidance for research in IoT security with computational intelligence.

The massive increase of spam is posing a very serious threat to email and SMS, which have become an important means of communication. Not only do spams annoy users, but they also become a security threat. Machine learning techniques have been widely used for spam detection. Email spams can be detected through detecting senders' behaviour, the contents of an email, subject and source address, etc, while SMS spam detection usually is based on the tokens or features of messages due to short content. However, a comprehensive analysis of email/SMS content may provide cures for users to aware of email/SMS spams. We cannot completely depend on automatic tools to identify all spams. In this paper, we propose an analysis approach based on information entropy and incremental learning to see how various features affect the performance of an RBF-based SVM spam detector, so that to increase our awareness of a spam by sensing the features of a spam. The experiments were carried out on the spambase and SMSSpemCollection databases in UCI machine learning repository. The results show that some features have significant impacts on spam detection, of which users should be aware, and there exists a feature space that achieves Pareto efficiency in True Positive Rate and True Negative Rate.

Existing adaptive predictive methods often use multiple adaptive mechanisms as part of their coping strategy in non-stationary environments. We address a scenario when selective deployment of these adaptive mechanisms is possible. In this case, deploying each adaptive mechanism results in different candidate models, and only one of these candidates is chosen to make predictions on the subsequent data. After observing the error of each of candidate, it is possible to revert the current model to the one which had the least error. We call this strategy retrospective model correction. In this work we aim to investigate the benefits of such approach. As a vehicle for the investigation we use an adaptive ensemble method for regression in batch learning mode which employs several adaptive mechanisms to react to changes in the data. Using real world data from the process industry we show empirically that the retrospective model correction is indeed beneficial for the predictive accuracy, especially for the weaker adaptive mechanisms.

This study introduces a new multi-layer multi-component ensemble. The components of this ensemble are trained locally on subsets of features for disjoint sets of data. The data instances are assigned to local regions using the similarity of their features pairwise squared correlation. Many ensemble methods encourage diversity among their base predictors by training them on different subsets of data or different subsets of features. In the proposed architecture the local regions contain disjoint sets of data and for this data only the most similar features are selected. The pairwise squared correlations of the features are used to weight the predictions of the ensemble's models. The proposed architecture has been tested on a number of data sets and its performance was compared to five benchmark algorithms. The results showed that the testing accuracy of the developed architecture is comparable to the rotation forest and is better than the other benchmark algorithms.

Predictive modelling is a complex process that requires a number of steps to transform raw data into predictions. Preprocessing of the input data is a key step in such process, and the selection of proper preprocessing methods is often a labour intensive task. Such methods are usually trained offline and their parameters remain fixed during the whole model deployment lifetime. However, preprocessing of non-stationary data streams is more challenging since the lack of adaptation of such preprocessing methods may degrade system performance. In addition, dependencies between different predictive system components make the adaptation process more challenging. In this paper we discuss the effects of change propagation resulting from using adaptive preprocessing in a Multicomponent Predictive System (MCPS). To highlight various issues we present four scenarios with different levels of adaptation. A number of experiments have been performed with a range of datasets to compare the prediction error in all four scenarios. Results show that well managed adaptation considerably improves the prediction performance. However, the model can become inconsistent if adaptation in one component is not correctly propagated throughout the rest of system components. Sometimes, such inconsistency may not cause an obvious deterioration in the system performance, therefore being difficult to detect. In some other cases it may even lead to a system failure as was observed in our experiments.

This research proposes the application of dynamic time warping (DTW) algorithm to analyse multivariate data from virtual reality training simulators, to assess the skill level of trainees. We present results of DTW algorithm applied to trajectory data from a virtual reality haptic training simulator for epidural needle insertion. The proposed application of DTW algorithm serves two purposes, to enable (i) two trajectories to be compared as a similarity measure and also enables (ii) two or more trajectories to be combined together to produce a typical or representative average trajectory using a novel hierarchical DTW process. Our experiments included 100 expert and 100 novice simulator recordings. The data consists of multivariate time series data-streams including multi-dimensional trajectories combined with force and pressure measurements. Our results show that our proposed application of DTW provides a useful time-independent method for (i) comparing two trajectories by providing a similarity measure and (ii) combining two or more trajectories into one, showing higher performance compared to conventional methods such as linear mean. These results demonstrate that DTW can be useful within virtual reality training simulators to provide a component in an automated scoring and assessment feedback system.

Existing adaptive predictive methods often use multiple adaptive mechanisms as part of their coping strategy in non-stationary environments. These mechanisms are usually deployed in a prescribed order which does not change. In this work we investigate and provide a comparative analysis of the effects of using a flexible order of adaptive mechanisms' deployment resulting in varying adaptation sequences. As a vehicle for this comparison, we use an adaptive ensemble method for regression in batch learning mode which employs several adaptive mechanisms to react to the changes in data. Using real world data from the process industry we demonstrate that such flexible deployment of available adaptive methods embedded in a cross-validatory framework can benefit the predictive accuracy over time.

This work shows the results of the practical implementation of the linearizing controller for the example laboratory pneumatic process of the third relative degree. Controller design is based on the Lie algebra framework but in contrast to the previous attempts, the on-line model update method is suggested to ensure offset-free control. The paper details the proposed concept and reports the experiences from the practical implementation of the suggested controller. The superiority of the proposed approach over the conventional PI controller is demonstrated by experimental results. Based on the experiences and the validation results, the possibilities of the potential application of the data-driven soft sensors for further improvement of the control performance are discussed.

In process industry, chemical processes are controlled and monitored by using readings from multiple physical sensors across the plants. Such physical sensors are also supplemented by soft sensors, i.e. adaptive predictive models, which are often used for computing hard-to-measure variables of the process. For soft sensors to work well and adapt to changing operating conditions they need to be provided with relevant data. As production plants are regularly stopped, data instances generated during shutdown periods have to be identified to avoid updating these predictive models with wrong data. We present a case study concerned with a large chemical plant operation over a 2 years period. The task is to robustly and accurately identify the shutdown periods even in case of multiple sensor failures. State-of-the-art methods were evaluated using the first half of the dataset for calibration purposes and the other half for measuring the performance. Results show that shutdowns (i.e. sudden changes) can be quickly detected in any case but the detection delay of startups (i.e. gradual changes) is directly related with the choice of a window size.

Many supervised learning approaches that adapt to changes in data distribution over time (e.g., concept drift) have been developed. The majority of them assume that the data comes already preprocessed or that preprocessing is an integral part of a learning algorithm. In real-application tasks, data that comes from, e.g., sensor readings, is typically noisy, contain missing values, redundant features, and a very large part of model development efforts is devoted to data preprocessing. As data is evolving over time, learning models need to be able to adapt to changes automatically. From a practical perspective, automating a predictor makes little sense if preprocessing requires manual adjustment over time. Nevertheless, adaptation of preprocessing has been largely overlooked in research. In this paper, we introduce and address the problem of adaptive preprocessing. We analyze when and under what circumstances it is beneficial to handle adaptivity of preprocessing and adaptivity of the learning model separately. We present three scenarios where handling adaptive preprocessing separately benefits the final prediction accuracy and illustrate them using computational examples. As a result of our analysis, we construct a prototype approach for combining adaptive preprocessing with adaptive predictor online. Our case study with real sensory data from a production process demonstrates that decoupling the adaptivity of preprocessing and the predictor contributes to improving the prediction accuracy. The developed reference framework and our experimental findings are intended to serve as a starting point in systematic research of adaptive preprocessing mechanisms for adaptive learning with evolving data.

The amount of research done in the area of real--world networked systems is rapidly growing. Everybody knows what six degrees of separation or small--world phenomenon are. Scientists very easily give labels to the networks they analyse. If it has power law node degree distribution then it has to be scale--free network or if there is high clustering coefficient then it must be small--world network. These simplifications, although convenient, are not always very useful from the perspective of understanding phenomena existing within the network. In this paper we decided to go back to the basics and investigate whether analysis of one single measure is enough to describe a network. We analyse both local and global characteristics in order to discover the "true" nature of a network. Not only using local and/or global measures can lead to different classification of a network but we also show how significantly different interpretation can result from analysing the same data by building network models as directed/undirected and/or weighted/binary graphs.

Estimation of the generalization ability of a classification or regression model is an important issue, as it indicates the expected performance on previously unseen data and is also used for model selection. Currently used generalization error estimation procedures, such as cross-validation (CV) or bootstrap, are stochastic and, thus, require multiple repetitions in order to produce reliable results, which can be computationally expensive, if not prohibitive. The correntropy-inspired density-preserving sampling (DPS) procedure proposed in this paper eliminates the need for repeating the error estimation procedure by dividing the available data into subsets that are guaranteed to be representative of the input dataset. This allows the production of low-variance error estimates with an accuracy comparable to 10 times repeated CV at a fraction of the computations required by CV. This method can also be used for model ranking and selection. This paper derives the DPS procedure and investigates its usability and performance using a set of public benchmark datasets and standard classifiers.

Rapidly evolving businesses generate massive amounts of time-stamped data sequences and cause a demand for both univariate and multivariate time series forecasting. For such data, traditional predictive models based on autoregression are often not sufficient to capture complex nonlinear relationships between multidimensional features and the time series outputs. In order to exploit these relationships for improved time series forecasting while also better dealing with a wider variety of prediction scenarios, a forecasting system requires a flexible and generic architecture to accommodate and tune various individual predictors as well as combination methods. In reply to this challenge, an architecture for combined, multilevel time series prediction is proposed, which is suitable for many different universal regressors and combination methods. The key strength of this architecture is its ability to build a diversified ensemble of individual predictors that form an input to a multilevel selection and fusion process before the final optimized output is obtained. Excellent generalization ability is achieved due to the highly boosted complementarity of individual models further enforced through cross-validation-linked training on exclusive data subsets and ensemble output postprocessing. In a sample configuration with basic neural network predictors and a mean combiner, the proposed system has been evaluated in different scenarios and showed a clear prediction performance gain.

There are no algorithms that generally perform better or worse than random when looking at all possible data sets according to the no-free-lunch theorem. A specific forecasting method will hence naturally have different performances in different empirical studies. This makes it impossible to draw general conclusions, however, there will of course be specific problems for which one algorithm performs better than another in practice. Meta-learning exploits this fact by linking characteristics of the data set to the performances of methods, adapting the selection or combination of base methods to a specific problem. This contribution describes an approach using meta-learning for time series forecasting in the NN GC1 competition. In order to generate bigger and more reliable meta-data set, data of the past NN3 and NN5 competitions have been included. A pool of individual forecasting and combination models are combined using a ranking algorithm with weights being determined by past performance on similar series.

Estimation of the generalization ability of a predictive model is an important issue, as it indicates expected performance on previously unseen data and is also used for model selection. Currently used generalization error estimation procedures like cross-validation (CV) or bootstrap are stochastic and thus require multiple repetitions in order to produce reliable results, which can be computationally expensive if not prohibitive. The correntropy-based Density Preserving Sampling procedure (DPS) proposed in this paper eliminates the need for repeating the error estimation procedure by dividing the available data into subsets, which are guaranteed to be representative of the input dataset. This allows to produce low variance error estimates with accuracy comparable to 10 times repeated cross-validation at a fraction of computations required by CV, which has been investigated using a set of publicly available benchmark datasets and standard classifiers.

Current soft sensing algorithms assume the availability of a large amount of training data. The collection of the historical data often takes a lot of time and can be expensive. At the same time not being able to provide sufficient amount of training data can result in sacrificing the performance of the soft sensor. This can be problematic in situations, where a soft sensor is urgently required and, at the same time, there is not enough training data available. This situation can occur, for example, when a new plant is taken into operation or, more critically, when there is a significant change in some parameters (e.g. operating point or the input materials) in a running plant. To deal with such a situation, we propose an algorithm, called Recursive Soft Sensing Algorithm (ReSSA), which delivers predictions without any explicit training phase. The proposed algorithm is based on the recursive functionality of the RPLS technique, which is embedded into local learning framework. More than that, during the run-time of the algorithm, it is not necessary to store any past data as the algorithm requires only the latest data point for its operation and recursive adaptation. In order to demonstrate the performance of the proposed method, it is applied to the prediction of a catalyst activity in a multi-tube reactor.

The combination of forecasts is a well established procedure for improving forecast performance and decreasing the risk of selecting an inferior model out of an existing pool of models. Work in this area mainly focuses on combining several functionally different models, but some publications also deal with combining forecasts with the same functional approach. In the latter case, individual forecasts are generated by diversifying one or more model parameters or, if dealing with hierarchical data, by using forecasts from different levels. This work looks at multi-dimensional data from airline industry, with the aim of improving the forecast of cancellation rates for bookings. Three different methods are employed for the generation of individual forecasts. Forecast combinations are usually implemented in a more or less static structure, either including all available forecasts or trimming a fixed percentage of the worst performing models. For a big number of individual forecasts, this procedure can become inefficient. In this paper, a dynamic approach of pooling and trimming is applied to the generated forecasts for airline cancellation data.

In this paper, we provide a theoretical analysis of effects of applying different forecast diversification methods on the structure of the forecast error covariance matrices and decomposed forecast error components based on the bias-variance-Bayes error decomposition of James and Hastie. We express the "diversityrdquo of different forecasts in relation to different error components and propose a measure in order to quantify it. We illustrate and discuss typical inhomogeneities frequently occurring in the forecast error covariance matrices and show that previously proposed pooling based only on error variances cannot fully exploit the complementary information present in a set of diverse forecasts to be combined. If covariance values could be reliably calculated, they could be taken into account during the pooling process. We study the difficult case in which covariance information cannot be measured properly and propose a novel simplified representation of the covariance matrix, which is only based on knowledge about the forecast generation process. Finally, we propose a new pooling approach that avoids inhomogeneities in the forecast error covariance matrix by considering the information contained in the simplified covariance representation and compare it with the error-variance-based pooling approach introduced by Aiolfi and Timmermann. Applying our approach more than once leads to the generation of multistep and multilevel forecast combination structures, which have generated significantly improved forecasts in our previous extensive experimental work; the summary of which is also provided.

This work combines several established regression and meta-learning techniques to give a holistic regression model and presents the proposed learnt topology gating artificial neural networks (LTGANN) model in the context of a general architecture previously published by the authors. The applied regression techniques are artificial neural networks, which are on one hand used as local experts for the regression modelling and on the other hand as gating networks. The role of the gating networks is to estimate the prediction error of the local experts dependent on the input data samples. This is achieved by relating the input data space to the performance of the local experts, and thus building a performance map, for each of the local experts. The estimation of the prediction error is then used for the weighting of the local experts predictions. Another advantage of our approach is that the particular neural networks are unconstrained in terms of the number of hidden units. It is only necessary to define the range within which the number of hidden units has to be generated. The model links the topology to the performance, which has been achieved by the network with the given complexity, using a probabilistic approach. As the model was developed in the context of process industry data, it is evaluated using two industrial data sets. The evaluation has shown a clear advantage when using a model combination and meta-learning approach as well as demonstrating the higher performance of LTGANN when compared to a standard combination method.

Incremental learning (IL) plays a key role in many real-world applications where data arrives over time. It is mainly concerned with learning models in an ever-changing environment. In this paper, we review some of the incremental learning algorithms and evaluate them within the same experimental settings in order to provide as objective comparative study as possible. These algorithms include fuzzy ARTMAP, nearest generalized exemplar, growing neural gas, generalized fuzzy min-max neural network, and IL based on function decomposition (ILFD).

In this paper we provide experimental results and extensions to our previous theoretical findings concerning the combination of forecasts that have been diversified by three different methods: with parameters learned at different data aggregation levels, by thick modeling and by the use of different forecasting methods. An approach of error variance based pooling as proposed by Aiolfi and Timmermann has been compared with flat combinations as well as an alternative pooling approach in which we consider information about the used diversification. An advantage of our approach is that it leads to the generation of novel multi step multi level forecast generation structures that carry out the combination in different steps of pooling corresponding to the different types of diversification. We describe different evolutionary approaches in order to evolve the order of pooling of the diversification dimensions. Extensions of such evolutions allow the generation of more flexible multi level multi step combination structures containing better adaptive capabilities. We could prove a significant error reduction comparing results of our generated combination structures with results generated with the algorithm of Aiolfi and Timmermann as well as with flat combination for the application of Revenue Management seasonal forecasting.

Rapidly evolving businesses generate massive amounts of time-stamped data sequences and defy a demand for massively multivariate time series analysis. For such data the predictive engine shifts from the historical auto-regression to modelling complex non-linear relationships between multidimensional features and the time series outputs. In order to exploit these time-disparate relationships for the improved time series forecasting, the system requires a flexible methodology of combining multiple prediction models applied to multiple versions of the temporal data under significant noise component and variable temporal depth of predictions. In reply to this challenge a composite time series prediction model is proposed which combines the strength of multiple neural network (NN) regressors applied to the temporally varied feature subsets and the postprocessing smoothing of outputs developed to further reduce noise. The key strength of the model is its excellent adaptability and generalisation ability achieved through a highly diversified set of complementary NN models. The model has been evaluated within NISIS Competition 2006 and NN3 Competition 2007 concerning prediction of univariate and multivariate time-series. It showed the best predictive performance among 12 competitive models in the NISIS 2006 and is under evaluation within NN3 2007 Competition.

The problem of pattern classification for inputs with missing values is considered. A general fuzzy min-max (GFMM) neural network utilising hyperbox fuzzy sets as a representation of data cluster prototypes is used. It is shown how a classification decisions can be carried out on a subspace of high dimensional input data. No substitution scheme for missing values is utilised. The result is a classification procedure that reduces a number of viable class alternatives on the basis of available information rather than attempting to produce one winning class without supporting evidence. A number of simulation results for well known data sets are provided to illustrate the properties and performance of the proposed approach.

This paper describes a general fuzzy min-max (GFMM) neural network which is a generalization and extension of the fuzzy min-max clustering and classification algorithms developed by Simpson. The GFMM method combines the supervised and unsupervised learning within a single training algorithm. The fusion of clustering and classification resulted in an algorithm that can be used as pure clustering, pure classification, or hybrid clustering classification. This hybrid system exhibits an interesting property of finding decision boundaries between classes while clustering patterns that cannot be said to belong to any of existing classes. Similarly to the original algorithms, the hyperbox fuzzy sets are used as a representation of clusters and classes. Learning is usually completed in a few passes through the data and consists of placing and adjusting the hyperboxes in the pattern space which is referred to as an expansion-contraction process. The classification results can be crisp or fuzzy. New data can be included without the need for retraining. While retaining all the interesting features of the original algorithms, a number of modifications to their definition have been made in order to accommodate fuzzy input patterns in the form of lower and upper bounds, combine the supervised and unsupervised learning, and improve the effectiveness of operations. A detailed account of the GFMM neural network, its comparison with the Simpson's fuzzy min-max neural networks, a set of examples, and an application to the leakage detection and identification in water distribution systems are given.