Unsupervised classification has been one of the most popular machine learning approaches for biological data analysis such as microarray gene expression data. The simplest statistical/machine learning approach for analyzing microarray gene expression data is to predict differentially expressed genes (DEGs), which is a two-class unsupervised classification process. When there are only two experimental conditions, t test or modified t test are the matured approaches to solve the problem. However when there are more than two experimental conditions, t test or modified t test can only be used to predict homogeneously DEGs. A homogeneously DEG shows the same regulation direction across multiple stages. A heterogeneously DEG shows different regulation directions. Multivariate unsupervised classification is then an option. Bayesian clustering algorithm (mclust) has been widely used for unsupervised classification of biological data. However it is extremely slow and is less reliable when the number of stages is small. Here we introduce a similar machine learning algorithm referred to as multivariate multi-scale Gaussian (MSG) as an alternative to mclust. We show that MSG is more accurate and faster. We have validated MSG using both simulated and real data.

MOTIVATION: Finding diagnostic patterns for fighting diseases like Burkholderia pseudomallei using biomarkers involves two key issues. First, exhausting all subsets of testable biomarkers (antigens in this context) to find a best one is computationally infeasible. Therefore, a proper optimization approach like evolutionary computation should be investigated. Second, a properly selected function of the antigens as the diagnostic pattern which is commonly unknown is a key to the diagnostic accuracy and the diagnostic effectiveness in clinical use. RESULTS: A conversion function is proposed to convert serum tests of antigens on patients to binary values based on which Boolean functions as the diagnostic patterns are developed. A genetic programming approach is designed for optimizing the diagnostic patterns in terms of their accuracy and effectiveness. During optimization, it is aimed to maximize the coverage (the rate of positive response to antigens) in the infected patients and minimize the coverage in the non-infected patients while maintaining the fewest number of testable antigens used in the Boolean functions as possible. The final coverage in the infected patients is 96.55% using 17 of 215 (7.4%) antigens with zero coverage in the non-infected patients. Among these 17 antigens, BPSL2697 is the most frequently selected one for the diagnosis of Burkholderia Pseudomallei. The approach has been evaluated using both the cross-validation and the Jack-knife simulation methods with the prediction accuracy as 93% and 92%, respectively. A novel approach is also proposed in this study to evaluate a model with binary data using ROC analysis.

BACKGROUND: Tyrosine sulfation is one of the most important posttranslational modifications. Due to its relevance to various disease developments, tyrosine sulfation has become the target for drug design. In order to facilitate efficient drug design, accurate prediction of sulfotyrosine sites is desirable. A predictor published seven years ago has been very successful with claimed prediction accuracy of 98%. However, it has a particularly low sensitivity when predicting sulfotyrosine sites in some newly sequenced proteins. RESULTS: A new approach has been developed for predicting sulfotyrosine sites using the random forest algorithm after a careful evaluation of seven machine learning algorithms. Peptides are formed by consecutive residues symmetrically flanking tyrosine sites. They are then encoded using an amino acid hydrophobicity scale. This new approach has increased the sensitivity by 22%, the specificity by 3%, and the total prediction accuracy by 10% compared with the previous predictor using the same blind data. Meanwhile, both negative and positive predictive powers have been increased by 9%. In addition, the random forest model has an excellent feature for ranking the residues flanking tyrosine sites, hence providing more information for further investigating the tyrosine sulfation mechanism. A web tool has been implemented at http://ecsb.ex.ac.uk/sulfotyrosine for public use. CONCLUSION: The random forest algorithm is able to deliver a better model compared with the Hidden Markov Model, the support vector machine, artificial neural networks, and others for predicting sulfotyrosine sites. The success shows that the random forest algorithm together with an amino acid hydrophobicity scale encoding can be a good candidate for peptide classification.

Although various machine learning approaches have been used for predicting protease cleavage sites, constructing a probabilistic model for these tasks is still challenging. This paper proposes a novel algorithm termed as a probabilistic peptide machine where estimating probability density functions and constructing a classifier for predicting protease cleavage sites are combined into one process. The simulation based on experimentally determined Hepatitis C virus (HCV) protease cleavage data has demonstrated the success of this new algorithm.

Mutations in HIV-1 drug targets can cause reduced affinity to antiretroviral inhibitors, leading to the emergence of resistant variants resulting in failure of treatment in infected individuals. Resistance testing is an important factor in the continued success of viral therapy. We found that through combining a structural based computational docking method and a classic machine learning technique we could create a consensus system capable of improving the prediction accuracy by 5.56% over either method used individually. The result was the creation of a genotypic resistance testing approach capable of classifying a wider cross-section of strains, hence making it a more accurate resistance testing method.

T Cell lymphocytes play a major role in the immune response against HIV-1. Development of a safe and effective T Cell immunogen based vaccine for HIV-1 depends on robust prediction of T Cell epitopes. 13 HIV-1 HXB2 protein sequences with marked CTL epitope sites were obtained from the Los Alamos HIV molecular immunology database. Bio-basis function neural networks with regression and Fisher ratio methods are used for epitope prediction. A residue-by-residue approach has been taken to avoid over-estimating the accuracy of prediction. Epitopes for all HLA alleles are considered, as opposed to creating allele specific models. Mean accuracy varies from 54.55% to 81.08% amongst the HIV-1 proteins.

A novel radial basis function neural network for discriminant analysis is presented in this paper. In contrast to many other researches, this work focuses on the exploitation of the weight structure of radial basis function neural networks using the Bayesian method. It is expected that the performance of a radial basis function neural network with a well-explored weight structure can be improved. As the weight structure of a radial basis function neural network is commonly unknown, the Bayesian method is, therefore, used in this paper to study this a priori structure. Two weight structures are investigated in this study, i.e., a single-Gaussian structure and a two-Gaussian structure. An expectation-maximization learning algorithm is used to estimate the weights. The simulation results showed that the proposed radial basis function neural network with a weight structure of two Gaussians outperformed the other algorithms.

MOTIVATION: Apoptosis has drawn the attention of researchers because of its importance in treating some diseases through finding a proper way to block or slow down the apoptosis process. Having understood that caspase cleavage is the key to apoptosis, we find novel methods or algorithms are essential for studying the specificity of caspase cleavage activity and this helps the effective drug design. As bio-basis function neural networks have proven to outperform some conventional neural learning algorithms, there is a motivation, in this study, to investigate the application of bio-basis function neural networks for the prediction of caspase cleavage sites. RESULTS: Thirteen protein sequences with experimentally determined caspase cleavage sites were downloaded from NCBI. Bayesian bio-basis function neural networks are investigated and the comparisons with single-layer perceptrons, multilayer perceptrons, the original bio-basis function neural networks and support vector machines are given. The impact of the sliding window size used to generate sub-sequences for modelling on prediction accuracy is studied. The results show that the Bayesian bio-basis function neural network with two Gaussian distributions for model parameters (weights) performed the best and the highest prediction accuracy is 97.15 +/- 1.13%. AVAILABILITY: The package of Bayesian bio-basis function neural network can be obtained by request to the author.

MOTIVATION: Although the outbreak of the severe acute respiratory syndrome (SARS) is currently over, it is expected that it will return to attack human beings. A critical challenge to scientists from various disciplines worldwide is to study the specificity of cleavage activity of SARS-related coronavirus (SARS-CoV) and use the knowledge obtained from the study for effective inhibitor design to fight the disease. The most commonly used inductive programming methods for knowledge discovery from data assume that the elements of input patterns are orthogonal to each other. Suppose a sub-sequence is denoted as P2-P1-P1'-P2', the conventional inductive programming method may result in a rule like 'if P1 = Q, then the sub-sequence is cleaved, otherwise non-cleaved'. If the site P1 is not orthogonal to the others (for instance, P2, P1' and P2'), the prediction power of these kind of rules may be limited. Therefore this study is aimed at developing a novel method for constructing non-orthogonal decision trees for mining protease data. RESULT: Eighteen sequences of coronavirus polyprotein were downloaded from NCBI (http://www.ncbi.nlm.nih.gov). Among these sequences, 252 cleavage sites were experimentally determined. These sequences were scanned using a sliding window with size k to generate about 50,000 k-mer sub-sequences (for short, k-mers). The value of k varies from 4 to 12 with a gap of two. The bio-basis function proposed by Thomson et al. is used to transform the k-mers to a high-dimensional numerical space on which an inductive programming method is applied for the purpose of deriving a decision tree for decision-making. The process of this transform is referred to as a bio-mapping. The constructed decision trees select about 10 out of 50,000 k-mers. This small set of selected k-mers is regarded as a set of decisive templates. By doing so, non-orthogonal decision trees are constructed using the selected templates and the prediction accuracy is significantly improved.

Motivation: Recent studies have found many proteins containing regions that do not form well-defined three-dimensional structures in their native states. The study and detection of such disordered regions is important both for understanding protein function and for facilitating structural analysis since disordered regions may affect solubility and/or crystallizability. Results: We have developed the regional order neural network (RONN) software as an application of our recently developed ‘bio-basis function neural network’ pattern recognition algorithm to the detection of natively disordered regions in proteins. The results of blind-testing a panel of nine disorder prediction tools (including RONN) against 80 protein sequences derived from the Protein Data Bank shows that, based on the probability excess measure, RONN performed the best. Availability: RONN is available at http://www.strubi.ox.ac.uk/RONN. Requests for the RONN software and the database of disorder (XML format) can be directed to the corresponding author. Contact: [email protected] Supplementary information: Details of all predictions made during blind testing, also available at http://www.strubi.ox.ac.uk/RONN3_Supplementary.pdf

The prediction of protease cleavage sites in proteins is critical to effective drug design. One of the important issues in constructing an accurate and efficient predictor is how to present nonnumerical amino acids to a model effectively. As this issue has not yet been paid full attention and is closely related to model efficiency and accuracy, we present a novel neural learning algorithm aimed at improving the prediction accuracy and reducing the time involved in training. The algorithm is developed based on the conventional radial basis function neural networks (RBFNNs) and is referred to as a bio-basis function neural network (BBFNN). The basic principle is to replace the radial basis function used in RBFNNs by a novel bio-basis function. Each bio-basis is a feature dimension in a numerical feature space, to which a nonnumerical sequence space is mapped for analysis. The bio-basis function is designed using an amino acid mutation matrix verified in biology. Thus, the biological content in protein sequences can be maximally utilized for accurate modeling. Mutual information (MI) is used to select the most informative bio-bases and an ensemble method is used to enhance a decision-making process, hence, improving the prediction accuracy further. The algorithm has been successfully verified in two case studies, namely the prediction of Human Immunodeficiency Virus (HIV) protease cleavage sites and trypsin cleavage sites in proteins.

A novel pattern recognition algorithm called an orthogonal kernel machine (OKM) is presented for the prediction of functional sites in proteins. Two novelties in OKM are that the kernel function is specially designed for measuring the similarity between a pair of protein sequences and the kernels are selected using the orthogonal method. Based on a set of well-recognized orthogonal kernels, this algorithm demonstrates its superior performance compared with other methods. An application of this algorithm to a real problem is presented.

One of the major tasks in bioinformatics is the classification and prediction of biological data. With the rapid increase in size of the biological databanks, it is essential to use computer programs to automate the classification process. At present, the computer programs that give the best prediction performance are support vector machines (SVMs). This is because SVMs are designed to maximise the margin to separate two classes so that the trained model generalises well on unseen data. Most other computer programs implement a classifier through the minimisation of error occurred in training, which leads to poorer generalisation. Because of this, SVMs have been widely applied to many areas of bioinformatics including protein function prediction, protease functional site recognition, transcription initiation site prediction and gene expression data classification. This paper will discuss the principles of SVMs and the applications of SVMs to the analysis of biological data, mainly protein and DNA sequences.

MOTIVATION: It is understood that clustering genes are useful for exploring scientific knowledge from DNA microarray gene expression data. The explored knowledge can be finally used for annotating biological function for novel genes. Representing the explored knowledge in an efficient manner is then closely related to the classification accuracy. However, this issue has not yet been paid the attention it deserves. RESULT: A novel method based on template theory in cognitive psychology and pattern recognition is developed in this study for representing knowledge extracted from cluster analysis effectively. The basic principle is to represent knowledge according to the relationship between genes and a found cluster structure. Based on this novel knowledge representation method, a pattern recognition algorithm (the decision tree algorithm C4.5) is then used to construct a classifier for annotating biological functions of novel genes. The experiments on five published datasets show that this method has improved the classification performance compared with the conventional method. The statistical tests indicate that this improvement is significant. AVAILABILITY: The software package can be obtained upon request from the author.

MOTIVATION: One of the most important issues in computational proteomics is to produce a prediction model for the classification or annotation of biological function of novel protein sequences. In order to improve the prediction accuracy, much attention has been paid to the improvement of the performance of the algorithms used, few is for solving the fundamental issue, namely, amino acid encoding as most existing pattern recognition algorithms are unable to recognize amino acids in protein sequences. Importantly, the most commonly used amino acid encoding method has the flaw that leads to large computational cost and recognition bias. RESULTS: By replacing kernel functions of support vector machines (SVMs) with amino acid similarity measurement matrices, we have modified SVMs, a new type of pattern recognition algorithm for analysing protein sequences, particularly for proteolytic cleavage site prediction. We refer to the modified SVMs as bio-support vector machine. When applied to the prediction of HIV protease cleavage sites, the new method has shown a remarkable advantage in reducing the model complexity and enhancing the model robustness.

MOTIVATION: Although, it is known that O-glycosidically linked oligosaccharides are commonly conjugated to a serine, threonine or hydroxylysine residue of the polypeptide, the chemical nature of the anchoring monosaccharide and the size of the oligosaccharide unit varies. Among different types, O-linked or mucin-type oligosaccharides are intimately involved in the secretion of proteins, be they enzymes, hormones or structural glycoproteins. Knowledge of the linkage sites in glycoproteins is critical to the design of specific and efficient inhibitors against the enzyme to catalyse the formation of the carbohydrate-peptide linkage. RESULTS: We present a method for predicting the linkage sites in O-linked glycoproteins using bio-basis function neural networks. The mean prediction accuracy of this method is 91.15 +/- 2.75% while it is 82.28 +/- 6.45% using back-propagation neural networks. Importantly, this method has significantly reduced the CPU time for modelling.

MOTIVATION: In order to design effective HIV inhibitors, studying and understanding the mechanism of HIV protease cleavage specification is critical. Various methods have been developed to explore the specificity of HIV protease cleavage activity. However, success in both extracting discriminant rules and maintaining high prediction accuracy is still challenging. The earlier study had employed genetic programming with a min-max scoring function to extract discriminant rules with success. However, the decision will finally be degenerated to one residue making further improvement of the prediction accuracy difficult. The challenge of revising the min-max scoring function so as to improve the prediction accuracy motivated this study. RESULTS: This paper has designed a new scoring function called a sum-product function for extracting HIV protease cleavage discriminant rules using genetic programming methods. The experiments show that the new scoring function is superior to the min-max scoring function. AVAILABILITY: The software package can be obtained by request to Dr Zheng Rong Yang.

This paper applies support vector machines (SM), a new powerful learning algorithm, to company failure prediction based on 2048 UK construction companies. The study shows that the SVM model outperforms linear statistical models and other neural network models.

This paper presents a biology inspired neural learning algorithm called bio-basis function neural network (BBFNN) for analysing protein sequences. The basic principle is to replace radial basis functions of conventional radial basis function neural networks with amino acid similarity measurement matrices. From this, model complexity can be significantly reduced and hence model robustness can be enhanced dramatically. We have applied the algorithm to the prediction of the phosphorylation sites in proteins and the cleavage sites in hepatitis C virus (HCV) polyproteins with success.

MOTIVATION: In protein chemistry, proteomics and biopharmaceutical development, there is a desire to know not only where a protein is cleaved by a protease, but also the susceptibility of its cleavage sites. The current tools for proteolytic cleavage prediction have often relied purely on regular expressions, or involve models that do not represent biological data well. RESULTS: A novel methodology for characterizing proteolytic cleavage site activities has been developed, which incorporates two fundamental features: activity class prediction and the use of an amino acid similarity matrix for (non-parametric) neural learning. The first solved the problem of predicting proteolytic efficiency. The second significantly improved the robustness in prediction and reduced the time complexity for learning. This study shows that activity class prediction is successful when applying this methodology to the prediction and characterization of Trypsin cleavage sites and the prediction of HIV protease cleavage sites. AVAILABILITY: Requests for software and data should be made respectively to Dr Zheng Rong Yang and Miss Rebecca Thomson.

An unsupervised probabilistic net (UPN) is introduced to identify health inequalities among countries according to their health status measured by the collected health indicators. By estimating the underlying probability density function of the health indicators using UPN, countries, which have similar health status, will be categorized into the same cluster. From this, the intercluster health inequalities are identified by the Mahalanobis distance, and the intracluster health inequalities are identified by the diversity within the clusters. To extract the typical health status, the concept of virtual objects is used in this study. Each virtual object in this study, therefore, represents a hypothetical country, which does not exist in a data set but can be found through learning. The identified virtual objects represent the hidden knowledge in a data set and can be valuable to social scientists in health promotion planning. Moreover, the investigation of the behavior of the virtual objects can help us to find the realistic and reasonable health promotion target for a country with a poor health status.

Abstract Motivation: The motivation is to identify, through machine learning techniques, specific patterns in HIV and HCV viral polyprotein amino acid residues where viral protease cleaves the polyprotein as it leaves the ribosome. An understanding of viral protease specificity may help the development of future anti-viral drugs involving protease inhibitors by identifying specific features of protease activity for further experimental investigation. While viral sequence information is growing at a fast rate, there is still comparatively little understanding of how viral polyproteins are cut into their functional unit lengths. The aim of the work reported here is to investigate whether it is possible to generalise from known cleavage sites to unknown cleavage sites for two specific viruses—HIV and HCV. An understanding of proteolytic activity for specific viruses will contribute to our understanding of viral protease function in general, thereby leading to a greater understanding of protease families and their substrate characteristics. Results: Our results show that artificial neural networks and symbolic learning techniques (See5) capture some fundamental and new substrate attributes, but neural networks outperform their symbolic counterpart. Availability: Publicly available software was used (Stuttgart Neural Network Simulator—http://www-ra.informatik.uni-tuebingen.de/SNNS/ and See5—http://www.rulequest.com. The datasets used (HIV, HCV) for See5 are available at: http://www.dcs.ex.ac.uk/~anarayan/bioinf/ismbdatasets/ Keywords: protase cleavage; protease inhibitors; machine learning; neural networks; decision trees. Contact: [email protected]; [email protected]; [email protected]

Many pattern recognition systems need to estimate an underlying probability density function (pdf). Mixture models are commonly used for this purpose in which an underlying pdf is estimated by a finite mixing of distributions. The basic computational element of a density mixture model is a component with a nonlinear mapping function, which takes part in mixing. Selecting an optimal set of components for mixture models is important to ensure an efficient and accurate estimate of an underlying pdf. Previous work has commonly estimated an underlying pdf based on the information contained in patterns. In this paper, mutual information theory is employed to measure whether two components are statistically dependent. If a component has small mutual information, it is statistically independent of the other components. Hence, that component makes a significant contribution to the system pdf and should not be removed. However, if a particular component has large mutual information, it is unlikely to be statistically independent of the other components and may be removed without significant damage to the estimated pdf. Continuing to remove components with large and positive mutual information will give a density mixture model with an optimal structure, which is very close to the true pdf.

We apply artificial neural networks to identify global health inequalities of 191 WHO countries by clustering them into eight groups. From this, we are able to provide key indicators for the health promotion direction for these countries.

Reports a method for breast cancer diagnosis using a robust heteroscedastic probabilistic neural network. The network has the inherent property of clustering patients into several groups, each of which has a distinct significance level: e.g. the larger the significance level of a benign (malignant) group, the more typical the benign (malignant) symptoms. From this, false benign patients can be identified through investigating the probabilistic relationships between each benign group with a small significance level and malignant groups. A false benign analysis table has thus been designed based on this approach. By detecting false benign, the misclassification rate of malignant patients can be reduced to a minimum without significantly increasing the misclassification rate of benign patients. In applying this method to Wisconsin diagnostic breast cancer data, the correct classification rates are 100% for malignant and 98% for benign.

The problem of distinguishing and classifying the responses of analog integrated circuits containing catastrophic faults has aroused recent interest. The problem is made more difficult when parametric variations are taken into account. Hence, statistical methods and techniques such as neural networks have been employed to automate classification. The major drawback to such techniques has been the implicit assumption that the variances of the responses of faulty circuits have been the same as each other and the same as that of the fault-free circuit. This assumption can be shown to be false. Neural networks, moreover, have proved to be slow. This paper describes a new neural network structure that clusters responses assuming different means and variances. Sophisticated statistical techniques are employed to handle situations where the variance tends to zero, such as happens with a fault that causes a response to be stuck at a supply rail. Two example circuits are used to show that this technique is significantly more accurate than other classification methods. A set of responses can be classified in the order of 1 s.

The evaluation of analogue and mixed-signal test strategies and design for test techniques requires the fault simulation of analogue circuits. The need to reduce fault simulation time for has resulted in the research into concurrent analogue fault simulation, analogous to digital fault simulation. Concurrent simulation can reduce the simulation time by avoiding repeated construction of the circuit matrix. Fault collapsing and dropping is also desirable. A robust, fast algorithm for concurrent analogue fault simulation is presented in this paper. Three techniques for the automatic dropping of faults have been addressed: a robust closeness measurement technique; a late start rule and an early stop rule. The algorithm has been successfully applied to both DC and transient analyses. A significant increase in the speed of analogue fault simulation has been obtained.