Implementing local contextual guidance principles in a single-layer CNN architecture, we propose an efficient algorithm for developing broad-purpose representations (i.e., representations transferable to new tasks without additional training) in shallow CNNs trained on limited-size datasets. A contextually guided CNN (CG-CNN) is trained on groups of neighboring image patches picked at random image locations in the dataset. Such neighboring patches are likely to have a common context and therefore are treated for the purposes of training as belonging to the same class. Across multiple iterations of such training on different context-sharing groups of image patches, CNN features that are optimized in one iteration are then transferred to the next iteration for further optimization, etc. In this process, CNN features acquire higher pluripotency, or inferential utility for any arbitrary classification task. In our applications to natural images and hyperspectral images, we find that CG-CNN can learn transferable features similar to those learned by the first layers of the well-known deep networks and produce favorable classification accuracies.

The canonical correlation analysis (CCA) aims at measuring linear relationships between two sets of variables (views) that can be used for feature extraction in classification problems with multiview data. However, the correlated features extracted by the CCA may not be class discriminative, since CCA does not utilize the class labels in its traditional formulation. Although there is a method called discriminative CCA (DCCA) that aims to increase the discriminative ability of CCA inspired from the linear discriminant analysis (LDA), it has been shown that the extracted features with this method are identical to those by the LDA with respect to an orthogonal transformation. Therefore, DCCA is simply equivalent to applying single-view (regular) LDA to each one of the views separately. Besides, DCCA and the other similar DCCA approaches have generalization problems due to the sample covariance matrices used in their computation, which are sensitive to outliers and noisy samples. In this paper, we propose a method, called discriminative alternating regression (D-AR), to explore correlated and also discriminative features. D-AR utilizes two (alternating) multilayer perceptrons, each with a linear hidden layer, learning to predict both the class labels and the outputs of each other. We show that the features found by D-AR on training sets significantly accomplish higher classification accuracies on test sets of facial expression recognition, object recognition, and image retrieval experimental data sets.

Recently, there is an increasing motivation to develop telemonitoring systems that enable cost-effective screening of Parkinson's Disease (PD) patients. These systems are generally based on measuring the motor system disorders seen in PD patients by the help of non-invasive data collection tools. Vocal impairments one of the most commonly seen PD symptoms in the early stages of the disease, and building such telemonitoring systems based on detecting the level of vocal impairments results in reliable motor UPDRS tracking systems. In this paper, we aim to determine the optimal UPDRS threshold value that can be discriminated by the vocal features extracted from the sustained vowel phonations of PD patients. For this purpose, we used an online available PD telemonitoring dataset consisting of speech recordings of 42 PD patients. We converted the UPDRS prediction problem into a binary classification problem for various motor UPDRS threshold values, and fed the features to k-Nearest Neighbor and Support Vector Machines classifiers to discriminate the PD patients whose UPDRS is less than or greater than the specified threshold value. The results indicate that speech disorders are more significantly seen in the patients whose UPDRS exceeds the experimentally determined threshold value (15). Besides, considering that the motor UPDRS ranges from 0 to 108, relatively low UPDRS threshold of 15 validates that vocal impairments can be used as early indicators of the disease.

Intrusion detection system (IDS) is an important component to ensure network security. In this paper we build an online Naïve Bayes classifier to discriminate normal and bad (intrusion) connections on KDD 99 dataset for network intrusion detection. The classifier starts with a small number of training examples of normal and bad classes; then, as it classifies the rest of the samples one at a time, it continuously updates the mean and the standard deviations of the features (IDS variables). We present experimental results of parameter updating methods and their parameters for the online Naïve Bayes classifier. The obtained results show that our proposed method performs comparably to the simple incremental update.

Leave-one-out (LOO) and its generalization, K-Fold, are among most well-known cross-validation methods, which divide the sample into many folds, each one of which is, in turn, left out for testing, while the other parts are used for training. In this study, as an extension of this idea, we propose a new cross-validation approach that we called miss-one-out (MOO) that mislabels the example(s) in each fold and keeps this fold in the training set as well, rather than leaving it out as LOO does. Then, MOO tests whether the trained classifier can correct the erroneous label of the training sample. In principle, having only one fold deliberately labeled incorrectly should have only a small effect on the classifier that uses this bad-fold along with K - 1 good folds and can be utilized as a generalization measure of the classifier. Experimental results on a number of benchmark datasets and three real bioinformatics dataset show that MOO can better estimate the test set accuracy of the classifier.

There has been an increased interest in speech pattern analysis applications of Parkinsonism for building predictive telediagnosis and telemonitoring models. For this purpose, we have collected a wide variety of voice samples, including sustained vowels, words, and sentences compiled from a set of speaking exercises for people with Parkinson's disease. There are two main issues in learning from such a dataset that consists of multiple speech recordings per subject: 1) How predictive these various types, e.g., sustained vowels versus words, of voice samples are in Parkinson's disease (PD) diagnosis? 2) How well the central tendency and dispersion metrics serve as representatives of all sample recordings of a subject? In this paper, investigating our Parkinson dataset using well-known machine learning tools, as reported in the literature, sustained vowels are found to carry more PD-discriminative information. We have also found that rather than using each voice recording of each subject as an independent data sample, representing the samples of a subject with central tendency and dispersion metrics improves generalization of the predictive model.

Location and time are two critical aspects of most security-related events, and thus, spatiotemporal data analysis plays a central role in many security-related applications. The human brain has great capabilities of developing invariant representations of objects by taking advantage of both spatial similarity of features of objects/events and their relative timings (temporal information). Trace learning rule is one well-known solution for this problem of combining temporal relations with spatial proximity in clustering tasks such as the one performed by self organizing maps. In this work, we investigate a two stage mechanism: i) finding local clusters using spatial proximity, ii) grouping these clusters as suggested by temporal continuity patterns. We show our experimental results on a movie created from face images.

Finding large deletions in genome sequences have become increasingly more useful in bioinformatics, such as in clinical research and diagnosis. Several partial alignment approaches based on the Smith-Waterman (SW) algorithm has been proposed for alignment with large gaps. However, in the literature, no detailed comparisons of these three SW-based methods were given in terms of the runtimes and errors in estimated position of the start of the deletion in the query sequences. Our comparative simulations show that BinaryPartialAlign has the lowest error and very high speed.

MOTIVATION: Gene therapy aims at using viral vectors for attaching helpful genetic code to target genes. Therefore, it is of great importance to develop methods that can discover significant patterns around viral integration sites. Canonical correlation analysis is an unsupervised statistical tool that is used to describe the relations between two related views of the same semantic object, which fits well for identifying such salient patterns. RESULTS: Proposed method is demonstrated on a sequence dataset obtained from a study on HIV-1 preferred integration regions. The subsequences on the left and right sides of the integration points are given to the method as the two views, and statistically significant relations are found between sequence-driven features derived from these two views, which suggest that the viral preference must be the factor responsible for this correlation. We found that there are significant correlations at x=5 indicating a palindromic behavior surrounding the viral integration site, which complies with the previously reported results. AVAILABILITY: Developed software tool is available at http://ce.istanbul.edu.tr/bioinformatics/hiv1/.

Mutual Information (MI) is a classical and widely used dependence measure that generally can serve as a good feature selection algorithm. However, under-sampled classes or rare but certain relations are overlooked by this measure, which can result in missing relevant features that could be very predictive of variables of interest, such as certain phenotypes or disorders in biomedical research, rare but dangerous factors in ecology, intrusions in network systems, etc. On the other hand, Kernel Canonical Correlation Analysis (KCCA) is a nonlinear correlation measure effectively used to detect independence but its use for feature selection or ranking is limited due to the fact that its formulation is not intended to measure the amount of information (entropy) of the dependence. In this paper, we propose Predictive Mutual Information (PMI), a hybrid measure of relevance not only is based on MI but also accounts for predictability of signals from one another as in KCCA. We show that PMI has more improved feature detection capability than MI and KCCA, especially in catching suspicious coincidences that are rare but potentially important not only for subsequent experimental studies but also for building computational predictive models which is demonstrated on two toy datasets and a real intrusion detection system dataset.

In many applications of pattern recognition in the bioinformatics and biomedical fields, input variables are organized into natural partitions that are called views in the literature. Mutual information can be used in selecting a minimal yet capable subset of views. Ignoring the presence of views, dismantling them, and treating their variables intermixed along with those of others at best results in a complex uninterpretable predictive system for researchers in these fields. Moreover, it would require measuring or computing majority of the views. We use the clustering indices of the views and rank the views according to the unique information they have with the target using minimum redundancy-maximum relevance (mRMR) approach. We also propose an ensemble approach to reduce the random variations in clusterings.

Spectral Clustering (SC) is a kernel method to cluster data objects using eigenvectors derived from the data. One fundamental issue that cause poor cuts in SC is its sensitivity to outliers. Another fundamental problem is how to determine the kernel bandwidth from the data. In fact, these two problems are also closely related. One cannot be solved before solving the other. The answer lies in robust and nonparametric estimators of the data density. We propose Reverse Soft K-Nearest Neighbor Density Estimation (RSKNN) that determines the density around a data sample, thus this sample's potential (other used terms are weight or entropy), using all the other samples' nearest neighbors' scatter properties on the contrary to the common practice of using the nearest neighbors of a sample itself to determine its own density. The basic idea behind this can be summarized as “every sample have k neighbors that are the nearest but not every point can be in many points' k-nearest neighborhood”. To demonstrate the use of it, we apply it to SC. Our package to use in SC consists of using RSKNN for estimating the density and the samples with high potential to be cluster centers helps in: 1) spectral decomposition phase to improve the generalization of the spectral cut criterion, 2) robust calculation of covariance matrix to be used in distance calculations, and 3) automatically determining the kernel bandwidth.

Feature selection and extraction are critical steps in many areas where pattern recognition techniques are applied. Feature selection and extraction are based on identifying and maximizing dependency relations. Gebelein's Maximal Correlation (GMC) is the most general form of dependence in that it does not make any statistical assumptions concerning the nature of the dependencies. Unfortunately, benefiting from such a useful measure in practice is generally impossible as there are only a few cases for which explicit formulae are available to calculate it. In this paper, we point out a parallel between GMC and the SINBAD algorithms, developed originally as a model of feature extraction for neurons in the cerebral cortex. We use SINBAD as a robust approximation to GMC to perform feature selection and extraction on a number of artificial and real datasets. We show that SINBAD estimates of GMC compare favorably to other well known feature selection and extraction methods based on mutual information, kernel canonical correlation analysis and principal component analysis.

Super-resolution is the creation of higher resolution views of pixel-based images through interpolating the original pixels. Natural images are highly redundant on a pixel-by-pixel scale due to local dependencies among pixels, such as lines and textures. Greater super-resolution can be achieved by taking advantage of these local features inherent in natural images. In order to discover and learn these statistically significant features, we used SINBAD method (Set of INteracting BAckpropagating Dendrites), which is a biologically inspired cortical model for perception. SINBAD method allowed us to infer missing pixel values better than standard backprop networks and polynomial interpolation techniques, which are insensitive to the lines or textures in the images. To further test which method preserves edges more accurately, we used Sobel edge detection. SINBAD cells picked local lines and edges as the easiest orderly features in un-preprocessed natural images. Thus, the SINBAD approach appears to follow the same route taken by the brain's processing of visual information.