Wearable sensors are becoming a popular method of objectively evaluating motor performance in various exercise tasks. A challenge in working with this motion capture data is the personalised nature of the data where one individual’s data may be different to that of others due to factors such as differences in exercise form. Hence global models generally perform poorly in this domain. The aim of this study is to investigate a technique to push personalised classification models into global models. In this research, a dataset of Inertial Measurement Unit data consisting of fatigued and non-fatigued running is used to employ a clustering strategy where participants are grouped together using a hierarchical clustering methodology. These clusters are used to form semi-globalised models. The investigation shows that this strategy can improve the global model performance by up to 20%.

Linyi Yang, Jiazheng Li, Padraig Cunningham, Yue Zhang, Barry Smyth, Ruihai Dong. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

We describe and evaluate a novel application of case-based reasoning to help marathon runners to achieve a personal best by: (a) predicting a challenging, but realistic race-time; and (b) recommending a race-plan to achieve this time.

We describe a novel application for recommender systems -- helping marathon runners to run a new personal-best race-time -- by predicting a challenging, but achievable target-time, and by recommending a tailored race-plan to achieve this time. A comprehensive evaluation of prediction accuracy and race-plan quality is provided using a large-scale dataset with almost 400,000 runners from the last 12 years of the Chicago marathon.

The proliferation of online communities has attracted much attention to modelling user behaviour in terms of social interaction, language adoption and contribution activity. Nevertheless, when applied to large-scale and cross-platform behavioural data, existing approaches generally suffer from expressiveness, scalability and generality issues. This paper proposes trans-dimensional von Mises-Fisher (TvMF) mixture models for $\mathcal{L}_{2}$ normalised behavioural data, which encapsulate: (1) a Bayesian framework for vMF mixtures that enables prior knowledge and information sharing among clusters, (2) an extended version of reversible jump MCMC algorithm that allows adaptive changes in the number of clusters for vMF mixtures when the model parameters are updated, and (3) an online TvMF mixture model that accommodates the dynamics of clusters for time-varying user behavioural data. We develop efficient collapsed Gibbs sampling techniques for posterior inference, which facilitates parallelism for parameter updates. Empirical results on simulated and real-world data show that the proposed TvMF mixture models can discover more interpretable and intuitive clusters than other widely-used models, such as k-means, non-negative matrix factorization (NMF), Dirichlet process Gaussian mixture models (DP-GMM), and dynamic topic models (DTM). We further evaluate the performance of proposed models in real-world applications, such as the churn prediction task, that shows the usefulness of the features generated.

The Syria conflict has been described as the most socially mediated in history, with online social media playing a particularly important role. At the same time, the ever-changing landscape of the conflict leads to difficulties in applying analytical approaches taken by other studies of online political activism. Therefore, in this paper, we use an approach that does not require strong prior assumptions or the proposal of an advance hypothesis to analyze Twitter and YouTube activity of a range of protagonists to the conflict, in an attempt to reveal additional insights into the relationships between them. By means of a network representation that combines multiple data views, we uncover communities of accounts falling into four categories that broadly reflect the situation on the ground in Syria. A detailed analysis of selected communities within the anti-regime categories is provided, focusing on their central actors, preferred online platforms, and activity surrounding “real world” events. Our findings indicate that social media activity in Syria is considerably more convoluted than reported in many other studies of online political activism, suggesting that alternative analytical approaches can play an important role in this type of scenario.

In this paper we present work on the development of a system for automated classification of digitized H&E histopathology images of prostate carcinoma (PCa). In our system, images are transformed into a tiled grid from which various texture and morphological features are extracted. We evaluate the contribution of high-level morphological features such as those derived from tissue segmentation algorithms as they relate to the accuracy of our classifier models. We also present work on an algorithm for tissue segmentation in image tiles, and introduce a novel feature vector representation of tissue classes in same. Finally, we present the classification accuracy, sensitivity and specificity results of our system when performing three tasks: distinguishing between cancer and non-cancer tiles, between low and high-grade cancer and between Gleason grades 3, 4 and 5. Our results show that the novel tissue representation outperforms the morphological features derived from tissue segmentation by a significant margin, but that neither feature sets improve on the accuracy gained by features from low-level texture methods.

Empirical social networks are often aggregate proxies for several heterogeneous relations. In online social networks, for instance, interactions related to friendship, kinship, business, interests, and other relationships may all be represented as catchall "friendships." Because several relations are mingled into one, the resulting networks exhibit relatively high and uniform density. As a consequence, the variation in positional differences and local cohesion may be too small for reliable analysis.

Temporal information is increasingly available with network data sets. This information can expose underlying processes in the data via sequences of link activations. Examples range from the propagation of ideas through a scientific collaboration network, to the spread of disease via contacts between infected and susceptible individuals. We focus on the flow of funds through an online financial transaction network, in which given patterns might signify suspicious behaviour. The search for these patterns may be formulated as a temporally constrained subgraph isomorphism problem. We compare two algorithms which use temporal data at different stages during the search, and empirically demonstrate one to be significantly more efficient.

State-of-the-art link prediction utilizes combinations of complex features derived from network panel data. We here show that computationally less expensive features can achieve the same performance in the common scenario in which the data is available as a sequence of interactions. Our features are based on social vector clocks, an adaptation of the vector-clock concept introduced in distributed computing to social interaction networks. In fact, our experiments suggest that by taking into account the order and spacing of interactions, social vector clocks exploit different aspects of link formation so that their combination with previous approaches yields the most accurate predictor to date.

The social media website last.fm provides a detailed snapshot of what its users in hundreds of cities listen to each week. After suitably normalizing this data, we use it to test one hypothesis related to the geographic flow of music: that some cities are consistently early adopters of new music (and early to snub stale music). To test this hypothesis, we adapt a method previously used to detect the leadership networks present in flocks of birds. We find empirical support for the claim that a similar leadership network exists among cities.

EgoNav is a visual analytics system that characterizes egos based on the relationship structure of their egocentric networks and presents the results as a spatialization. An ego, or individual node in a network, is most closely related to its neighbors, and to a lesser degree, to its neighbor's neighbors. For example, in social networks, people are closely related to their friends and family. In financial networks, the affairs of borrowers and lenders are more closely tied to each other. In fact, the relationship structure surrounding an ego, or an egocentric network, can provide characteristic information about the ego itself. Using network motif analysis and dimensionality reduction techniques, the system places egos in similar areas of a spatialization if their egocentric networks are structurally similar. This view of a network discriminates between the various classes of typical and exceptional egos. We demonstrate its effectiveness using appropriate synthetic datasets, real-world mobile phone call and peer-to-peer lending datasets. We subsequently elicit user feedback from experts involved in the investigation of financial fraud to assess the tool's applicability in this domain.

With increased adoption of social networking tools, it is becoming more difficult to extract useful information from the mass of data generated daily by users. Curation of content and sources is an important filter in separating the signal from noise. A good set of credible sources often requires painstaking manual curation, which often yields incomplete coverage of a topic. In this demo, we present a recommender system to aid this process, improving the quality and quantity of sources. The system is highly-adaptable to the goals of the curator, enabling some novel uses for curating and monitoring lists of users.

Real-world social networks from a variety of domains can naturally be modelled as dynamic graphs. However, approaches to detecting communities have largely focused on identifying communities in static graphs. Recently, researchers have begun to consider the problem of tracking the evolution of groups of users in dynamic scenarios. Here we describe a model for tracking the progress of communities over time in a dynamic network, where each community is characterised by a series of significant evolutionary events. This model is used to motivate a community-matching strategy for efficiently identifying and tracking dynamic communities. Evaluations on synthetic graphs containing embedded events demonstrate that this strategy can successfully track communities over time in volatile networks. In addition, we describe experiments exploring the dynamic communities detected in a real mobile operator network containing millions of users.

In speaker verification several techniques have emerged to map variable length utterances into a fixed dimensional space for classification. One popular approach uses Maximum A-Posteriori (MAP) adaptation of a Gaussian Mixture Model (GMM) to create a super-vector. This paper investigates using Vector Quantisation (VQ) as the global model to provide a similar mapping. This less computationally complex mapping gives comparable results to its GMM counterpart while also providing the ability for an efficient iterative update enabling media files to be scanned with a fixed length window.

Assessing the trustworthiness of reviews is a key issue for the maintainers of opinion sites such as TripAdvisor, given the rewards that can be derived from posting false or biased reviews. In this paper we present a number of criteria that might be indicative of suspicious reviews and evaluate alternative methods for integrating these criteria to produce a unified "suspiciousness" ranking. The criteria derive from characteristics of the network of reviewers and also from analysis of the content and impact of reviews and ratings. The integration methods that are evaluated are singular value decomposition and the unsupervised hedge algorithm. These alternatives are evaluated in a user study on TripAdvisor reviews, where volunteers were asked to rate the suspiciousness of reviews that have been highlighted by the criteria.

BACKGROUND: Epistatic miniarray profiling (E-MAPs) is a high-throughput approach capable of quantifying aggravating or alleviating genetic interactions between gene pairs. The datasets resulting from E-MAP experiments typically take the form of a symmetric pairwise matrix of interaction scores. These datasets have a significant number of missing values - up to 35% - that can reduce the effectiveness of some data analysis techniques and prevent the use of others. An effective method for imputing interactions would therefore increase the types of possible analysis, as well as increase the potential to identify novel functional interactions between gene pairs. Several methods have been developed to handle missing values in microarray data, but it is unclear how applicable these methods are to E-MAP data because of their pairwise nature and the significantly larger number of missing values. Here we evaluate four alternative imputation strategies, three local (Nearest neighbor-based) and one global (PCA-based), that have been modified to work with symmetric pairwise data. RESULTS: We identify different categories for the missing data based on their underlying cause, and show that values from the largest category can be imputed effectively. We compare local and global imputation approaches across a variety of distinct E-MAP datasets, showing that both are competitive and preferable to filling in with zeros. In addition we show that these methods are effective in an E-MAP from a different species, suggesting that pairwise imputation techniques will be increasingly useful as analogous epistasis mapping techniques are developed in different species. We show that strongly alleviating interactions are significantly more difficult to predict than strongly aggravating interactions. Finally we show that imputed interactions, generated using nearest neighbor methods, are enriched for annotations in the same manner as measured interactions. Therefore our method potentially expands the number of mapped epistatic interactions. In addition we make implementations of our algorithms available for use by other researchers. CONCLUSIONS: We address the problem of missing value imputation for E-MAPs, and suggest the use of symmetric nearest neighbor based approaches as they offer consistently accurate imputations across multiple datasets in a tractable manner.

Assessing the similarity between cases is a key aspect of the retrieval phase in case-based reasoning (CBR). In most CBR work, similarity is assessed based on feature value descriptions of cases using similarity metrics, which use these feature values. In fact, it might be said that this notion of a feature value representation is a defining part of the CBR worldview-it underpins the idea of a problem space with cases located relative to each other in this space. Recently, a variety of similarity mechanisms have emerged that are not founded on this feature space idea. Some of these new similarity mechanisms have emerged in CBR research and some have arisen in other areas of data analysis. In fact, research on kernel-based learning is a rich source of novel similarity representations because of the emphasis on encoding domain knowledge in the kernel function. In this paper, we present a taxonomy that organizes these new similarity mechanisms and more established similarity mechanisms in a coherent framework.

Heterogeneous genome-wide data sources capture information on various aspects of complex biological systems. For instance, transcriptome, interactome and phenome-level information may be derived from mRNA expression data, protein-protein interaction networks, and biomedical literature corpora. Each source provides a distinct ldquoviewrdquo of the same domain, but potentially encodes different biologically-relevant patterns. Effective integration of such views can provide a richer, more informative model of an organismpsilas functional modules than that produced on a single view alone. Existing machine learning strategies for information fusion largely focus on the production of a consensus model that reflects patterns shared between views. However, the information provided by different views may not always be easily reconciled, due to the incomplete nature of the data, or the fact that some patterns will be present in one view but not in another. To address this problem, we present the Parallel Integration Clustering Algorithm (PICA), a novel cluster analysis approach which supports the simultaneous integration of information from two or more sources. The resulting model preserves patterns that are unique to individual views, as well as those common to all views. We demonstrate the effectiveness of PICA in identifying significant patterns corresponding to functional groupings, when applied to three genome-wide datasets.

Trust, reputation and recommendation are key components of successful e-commerce systems. However, e-commerce systems are also vulnerable in this respect because there are opportunities for sellers to gain advantage through manipulation of reputation and recommendation. One such vulnerability is the use of fraudulent user profiles to boost (or damage) the ratings of items in an online recommender system. In this paper we cast this problem as a problem of detecting anomalous structure in network analysis and propose a novel mechanism for detecting this anomalous structure. We present an evaluation that shows that this approach is effective at uncovering the types of recommender systems attack described in the literature.

MOTIVATION: When working with large-scale protein interaction data, an important analysis task is the assignment of pairs of proteins to groups that correspond to higher order assemblies. Previously a common approach to this problem has been to apply standard hierarchical clustering methods to identify such a groups. Here we propose a new algorithm for aggregating a diverse collection of matrix factorizations to produce a more informative clustering, which takes the form of a 'soft' hierarchy of clusters. RESULTS: We apply the proposed Ensemble non-negative matrix factorization (NMF) algorithm to a high-quality assembly of binary protein interactions derived from two proteome-wide studies in yeast. Our experimental evaluation demonstrates that the algorithm lends itself to discovering small localized structures in this data, which correspond to known functional groupings of complexes. In addition, we show that the algorithm also supports the assignment of putative functions for previously uncharacterized proteins, for instance the protein YNR024W, which may be an uncharacterized component of the exosome.

BACKGROUND: Metabolomics, or metabonomics, refers to the quantitative analysis of all metabolites present within a biological sample and is generally carried out using NMR spectroscopy or Mass Spectrometry. Such analysis produces a set of peaks, or features, indicative of the metabolic composition of the sample and may be used as a basis for sample classification. Feature selection may be employed to improve classification accuracy or aid model explanation by establishing a subset of class discriminating features. Factors such as experimental noise, choice of technique and threshold selection may adversely affect the set of selected features retrieved. Furthermore, the high dimensionality and multi-collinearity inherent within metabolomics data may exacerbate discrepancies between the set of features retrieved and those required to provide a complete explanation of metabolite signatures. Given these issues, the latter in particular, we present the MetaFIND application for 'post-feature selection' correlation analysis of metabolomics data. RESULTS: In our evaluation we show how MetaFIND may be used to elucidate metabolite signatures from the set of features selected by diverse techniques over two metabolomics datasets. Importantly, we also show how MetaFIND may augment standard feature selection and aid the discovery of additional significant features, including those which represent novel class discriminating metabolites. MetaFIND also supports the discovery of higher level metabolite correlations. CONCLUSION: Standard feature selection techniques may fail to capture the full set of relevant features in the case of high dimensional, multi-collinear metabolomics data. We show that the MetaFIND 'post-feature selection' analysis tool may aid metabolite signature elucidation, feature discovery and inference of metabolic correlations.

Microarrays have the capacity to measure the expressions of thousands of genes in parallel over many experimental samples. The unsupervised technique of bicluster analysis has been employed previously to uncover gene expression correlations over subsets of samples with the aim of modelling the natural gene functional classes. However the bicluster model also has the potential to shed light on the functions of unannotated open reading frames (ORFs). This aspect of biclustering has been under-explored. In this work we illustrate how the bicluster representation of expression data may be extended to enable putative functional classification of unannotated ORFs. We develop an ORF annotation approach, referred to as BALBOA, in which classifiers are constructed from the class specific expression patterns discovered by bicluster analysis. We demonstrate the efficacy of this approach via cross validation and carry out a comparative evaluation with kNN classification across three yeast expression datasets. Finally, we assign putative functions to unannotated ORFs and attempt to corroborate the best supported annotations with external experimental and protein sequence information.

The paper presents a comparison of the data-based and gene ontology (GO)-based approaches to cluster validation methods for gene microarray analysis. We apply a homogeneous approach to obtaining metrics from different GO-based similarity measures and a normalization of validation index values, that allows us to compare them to each other as well as to data-based validation indices. The results show strong correlation between both GO-based and data-based validation indices. The results suggest that this may represent an effective tool to support biomedical knowledge discovery tasks based on gene expression data

In the real world concepts and data distributions are often not stable but change with time. This problem, known as concept drift, complicates the task of learning a model from data and requires special approaches, different from commonly used techniques, which treat arriving instances as equally important contributors to the target concept. Among the most popular and effective approaches to handle concept drift is ensemble learning, where a set of models built over different time periods is maintained and the best model is selected or the predictions of models are combined. In this paper we consider the use of an ensemble integration technique that helps to better handle concept drift at the instance level. Our experiments with real-world antibiotic resistance data demonstrate that dynamic integration of classifiers built over small time intervals can be more effective than globally weighted voting which is currently the most commonly used integration approach for handling concept drift with ensembles

In a gene expression data matrix a bicluster is a sub-matrix of genes and conditions that exhibits a high correlation of expression activity across both rows and columns. The premise behind biclustering is that even related genes may only be expressed in a synchronized fashion over certain conditions. Conventional clustering groups over all features and may not capture these local relationships. Biclustering has the potential to retrieve these local signals and also to model overlapping groups of genes. These factors allow better representation of the natural state of functional modules in the cell. The mean squared residue is a popular measure of bicluster quality. One drawback however is that it is biased toward flat biclusters with low row variance. In this paper we introduce an improved bicluster score that removes this bias and promotes the discovery the most significant biclusters in the dataset. We employ this score within a new biclustering approach based on the bottom-up search strategy. We believe that the bottom-up search approach better models the underlying functional modules of the gene expression dataset. We evaluate our new score against the mean squared residue score using a yeast cell cycle expression dataset. We then carry out a comparative analysis of our biclustering technique against previously published clustering and biclustering approaches. Lastly, we use the biclusters discovered by our method to attempt to putatively annotate unclassified genes

In supervised kernel methods, it has been observed that the performance of the SVM classifier is poor in cases where the diagonal entries of the Gram matrix are large relative to the off-diagonal entries. This problem, referred to as diagonal dominance, often occurs when certain kernel functions are applied to sparse high-dimensional data, such as text corpora. In this paper we investigate the implications of diagonal dominance for unsupervised kernel methods, specifically in the task of document clustering. We propose a selection of strategies for addressing this issue, and evaluate their effectiveness in producing more accurate and stable clusterings.

The research presented here explores the hypothesis that the deployment and acceptance of decision support systems in medicine will be enhanced if the basis for the recommendation produced by the system is apparent. We describe a decision support system for advising on patients suffering from bronchiolitis. This system supports its recommendations with precedent cases selected to support the recommendation along with justification text that highlights aspects of these cases relevant to the query case. It also presents an estimate of its confidence in the recommendation. The main contribution of this paper is an evaluation of this system in a clinical context. The evaluation shows that this type of explanation does enhance the usefulness of the system for practitioners.

In a gene expression data matrix, a bicluster is a submatrix of genes and conditions that exhibits a high correlation of expression activity across both rows and columns. The problem of locating the most significant bicluster has been shown to be NP-complete. Heuristic approaches such as Cheng and Church's greedy node deletion algorithm have been previously employed. It is to be expected that stochastic search techniques such as evolutionary algorithms or simulated annealing might improve upon such greedy techniques. In this paper we show that an approach based on simulated annealing is well suited to this problem, and we present a comparative evaluation of simulated annealing and node deletion on a variety of datasets. We show that simulated annealing discovers more significant biclusters in many cases. Furthermore, we also test the ability of our technique to locate biologically verifiable biclusters within an annotated set of genes.

In a gene expression data matrix a bicluster is a grouping of a subset of genes and a subset of conditions which show correlating levels of expression activity. The difficulty of finding significant biclusters in gene expression data grows exponentially with the size of the dataset and heuristic approaches such as Cheng and Church's greedy node deletion algorithm are required. It is to be expected that stochastic search techniques such as genetic algorithms or simulated annealing might produce better solutions than greedy search. In this paper we show that a simulated annealing approach is well suited to this problem and we present a comparative evaluation of simulated annealing and node deletion on a variety of datasets. We show that simulated annealing discovers more significant biclusters in many cases.

Abstract. There is a range of potential applications of Machine Learning where it would be more useful to predict the probability distribution for a variable rather than simply the most likely value for that variable. In meteorology and in finance it is often important to know the probability of a variable falling within (or outside) different ranges. In this paper we consider the prediction of surf height with the objective of predicting if it will fall within a given ‘surfable ’ range. Prediction problems such as this are considerably more difficult if the distribution of the phenomenon is significantly different from a normal distribution. This is the case with the surf data we have studied. To address this we use an ensemble of mixture density networks to predict the probability density function. Our evaluation shows that this is an effective solution. We also describe a web-based application that presents these predictions in a usable manner. 1

UNLABELLED: In this paper we present a data mining system, which allows the application of different clustering and cluster validity algorithms for DNA microarray data. This tool may improve the quality of the data analysis results, and may support the prediction of the number of relevant clusters in the microarray datasets. This systematic evaluation approach may significantly aid genome expression analyses for knowledge discovery applications. The developed software system may be effectively used for clustering and validating not only DNA microarray expression analysis applications but also other biomedical and physical data with no limitations. AVAILABILITY: The program is freely available for non-profit use on request at http://www.cs.tcd.ie/Nadia.Bolshakova/Machaon.html CONTACT: [email protected].

Summary: This paper presents an approach to assessing cluster validity based on similarity knowledge extracted from the Gene Ontology. Availability: The program is freely available for non-profit use on request from the authors. Contact: [email protected] Supplementary information: http://www.cs.tcd.ie/Nadia.Bolshakova/GOtool.html

Ensemble techniques have been successfully applied in the context of supervised learning to increase the accuracy and stability of classification. Recently, analogous techniques for cluster analysis have been suggested. Research has demonstrated that, by combining a collection of dissimilar clusterings, an improved solution can be obtained. In this paper, we examine the potential of applying ensemble clustering techniques with a focus on the area of medical diagnostics. We present several ensemble generation and integration strategies, and evaluate each approach on a number of synthetic and real-world datasets. In addition, we show that diversity among ensemble members is necessary, but not sufficient to yield an improved solution without the selection of an appropriate integration method.

The goal of this paper is to propose, evaluate, and compare four search strategies for ensemble feature selection, and to consider their application to medical diagnostics, with a focus on the problem of the classification of acute abdominal pain. Ensembles of learnt models constitute one of the main current directions in machine learning and data mining. Ensembles allow us to get higher accuracy, sensitivity, and specificity, which are often not achievable with single models. One technique, which proved to be effective for ensemble construction, is feature selection. Lately, several strategies for ensemble feature selection were proposed, including random subspacing, hill-climbing-based search, and genetic search. In this paper, we propose two new sequential-search-based strategies for ensemble feature selection, and evaluate them, constructing ensembles of simple Bayesian classifiers for the problem of acute abdominal pain classification. We compare the search strategies with regard to achieved accuracy, sensitivity, specificity, and the average number of features they select.

In this paper we evaluate the usefulness of seeding genetic algorithms (GAs) from a case-base. This is motivated by the expectation that the seeding will speed up the GA by starting the search in promising regions of the search space. We evaluate this case-based seeding on popular GA solutions to the Travelling Salesman Problem (TSP) and the Job-Shop Scheduling Problem (JSSP). We find that seeding the initial population gives the GA a head start and that this advantage is maintained to the end of the GA in both problem domains. In addition, we have also found that the scale of the initial advantage can be greatly improved through the introduction of problem specific information in the seeding process. Finally, the improvements of the seeded GA over the random GA can be used to reduce the number of generations required to find suitably fit solutions.

Case based reasoning (CBR) is an artificial intelligence technique that emphasises the role of past experience during future problem solving. New problems are solved by retrieving and adapting the solutions to similar problems, solutions that have been stored and indexed for future reuse as cases in a case-base. The power of CBR is severely curtailed if problem solving is limited to the retrieval and adaptation of a single case, so most CBR systems dealing with complex problem solving tasks have to use multiple cases. The paper describes and evaluates the technique of hierarchical case based reasoning, which allows complex problems to be solved by reusing multiple cases at various levels of abstraction. The technique is described in the context of Deja Vu, a CBR system aimed at automating plant-control software design.

In this paper, we investigate how the level of diversity amongst individual neural networks in a bagged ensemble can significantly influence overall ensemble generalization performance. We propose a new technique that tunes this diversity so that ensemble generalization performance is optimized and evaluate its performance on benchmark regression data-sets.

We propose a technique that uses the bootstrap method to estimate confidence and prediction intervals for neural network (regression) ensembles. Our proposed technique can be applied to any ensemble technique that uses the bootstrap to generate the training sets for the ensemble, such as bagging and balancing. Confidence and prediction intervals are estimated that include a significantly improved estimate of underlying model uncertainty (i.e.) the uncertainty of our estimate of the "true" regression. Unlike existing techniques, this estimate of uncertainty will vary according to which ensemble technique is used if the effect of using a specific ensemble technique is to produce less model uncertainty than using another ensemble technique, then this will be reflected in the confidence and prediction intervals. Preliminary results illustrate how our technique can provide more accurate confidence and prediction intervals (intervals that better reflect the desired level of confidence (e.g.) 90%, 95%, etc.) for neural network ensembles than previous attempts.