A large population of users gets affected by sudden slowdown or shutdown of an enterprise application. System administrators and analysts spend considerable amount of time dealing with functional and performance bugs. These problems are particularly hard to detect and diagnose in most computer systems, since there is a huge amount of system generated supportability data (counters, logs etc.) that need to be analyzed. Most often, there isn't a very clear or obvious root cause. Timely identification of significant change in application behavior is very important to prevent negative impact on the service. In this paper, we present ADELE, an empirical, data-driven methodology for early detection of anomalies in data storage systems. The key feature of our solution is diligent selection of features from system logs and development of effective machine learning techniques for anomaly prediction. ADELE learns from system's own history to establish the baseline of normal behavior and gives accurate indications of the time period when something is amiss for a system. Validation on more than 4800 actual support cases shows~83% true positive rate and~12% false positive rate in identifying periods when the machine is not performing normally. We also establish the existence of problem “signatures” which help map customer problems to already seen issues in the field. ADELE's capability to predict early paves way for online failure prediction for customer systems.

System log files contains messages emitted from several modules within a system and carries valuable information about the system state such as device status and error conditions and also about the various tasks within the system such as program names, execution path, including function names and parameters, and the task completion status. For customers with remote support, the system collects and transmits these logs to a central enterprise repository, where these are monitored for alerts, problem forecasting, and troubleshooting. Very large log files limit the interpretability for the support engineers. For an expert, a large volume of log messages may not pose any problem; however, an inexperienced person may get flummoxed due to the presence of a large number of log messages. Often it is desired to present the log messages in a comprehensive manner where a person can view the important messages first and then go into details if required. In this article, we present a user-friendly log viewer where we first hide the unimportant or inconsequential messages from the log file. A user can then click a particular hidden view and get the details of the hided messages. Messages with low utility are considered inconsequential as their removal does not impact the end user for the aforesaid purpose such as problem forecasting or troubleshooting. We relate the utility of a message to the probability of its appearance in the due context. We present machine-learning-based techniques that computes the usefulness of individual messages in a log file. We demonstrate identification and discarding of inconsequential messages to shrink the log size to acceptable limits. We have tested this over real-world logs and observed that eliminating such low value data can reduce the log files significantly (30% to 55%), with minimal error rates (7% to 20%). When limited user feedback is available, we show modifications to the technique to learn the user intent and accordingly further reduce the error.

Storage workload identification is the task of characterizing a workload in a storage system (more specifically, network storage system—NAS or SAN) and matching it with the previously known workloads. We refer to storage workload identification as “workload identification” in the rest of this article. Workload identification is an important problem for cloud providers to solve because (1) providers can leverage this information to colocate similar workloads to make the system more predictable and (2) providers can identify workloads and subsequently give guidance to the subscribers as to associated best practices (with respect to configuration) for provisioning those workloads. Historically, people have identified workloads by looking at their read/write ratios, random/sequential ratios, block size, and interarrival frequency. Researchers are well aware that workload characteristics change over time and that one cannot just take a point in time view of a workload, as that will incorrectly characterize workload behavior. Increasingly, manual detection of workload signature is becoming harder because (1) it is difficult for a human to detect a pattern and (2) representing a workload signature by a tuple consisting of average values for each of the signature components leads to a large error. In this article, we present workload signature detection and a matching algorithm that is able to correctly identify workload signatures and match them with other similar workload signatures. We have tested our algorithm on nine different workloads generated using publicly available traces and on real customer workloads running in the field to show the robustness of our approach.

Performance problems are particularly hard to detect and diagnose in most computer systems, since there is no clear failure apart from the system being slow. In this paper, we present an empirical, data-driven methodology for detecting performance problems in data storage systems, and aiding in quick diagnosis once a problem is detected. The key feature of our solution is that it uses a combination of time-series analysis, domain knowledge and expert inputs to improve the overall efficacy. Our solution learns from a system's own history to establish the baseline of normal behavior. Hence it is not necessary to determine any static trigger-levels for metrics to raise alerts. Static triggers are ineffective since each system and its workloads are different from others. The method presented here (a) gives accurate indications of the time period when something goes wrong in a system, and (b) helps pin-point the most affected parts of the system to aid in diagnosis. Validation on more than 400 actual field support cases shows about 85% true positive rate with less than 10% false positive rate in identifying time periods of performance impact before or during the time a case was open. Results in a controlled lab environment are even better.

While biometrics-based identification is a key technology in many critical applications such as searching for an identity in a watch list or checking for duplicates in a citizen ID card system, there are many technical challenges in building a solution because the size of the database can be very large (often in 100s of millions) and the intrinsic errors with the underlying biometrics engines. Often multi-modal biometrics is proposed as a way to improve the underlying biometrics accuracy performance. In this paper, we propose a score based fusion scheme tailored for identification applications. The proposed algorithm uses a gradient descent method to learn weights for each modality such that weighted sum of genuine scores is larger than the weighted sum of all the impostor scores. During the identification phase, top K candidates from each modality are retrieved and a super-set of identities is constructed. Using the learnt weights, we compute the weighted score for all the candidates in the superset. The highest scoring candidate is declared as the top candidate for identification. The proposed algorithm has been tested using NIST BSSR-1 dataset and results in terms of accuracy as well as the speed (execution time) are shown to be far superior than the published results on this dataset.

We present an online adaptive clustering algorithm in a decision tree framework which has an adaptive tree and a code formation layer. The code formation layer stores the representative codes of the clusters and the tree adapts the separating hyperplanes between the clusters. The membership of a sample in a cluster is decided by the tree and the tree parameters are guided by stored codes. The model provides a hierarchical representation of the clusters by minimizing a global objective function as opposed to the existing hierarchical clusterings where a local objective function at every level is optimized. We show the results on real-life data.

In this paper, we present a least square kernel machine with box constraints (LSKMBC). The existing least square machines assume Gaussian hyperpriors and subsequently express the optima of the regularized squared loss as a set of linear equations. The generalized LASSO framework deviates from the assumption of Gaussian hyperpriors and employs a more general Huber loss function. In our approach, we consider uniform priors and obtain the loss functional for a given margin considered to be a model selection parameter. The framework not only differs from the existing least square kernel machines, but also it does not require Mercer condition satisfiability. Experimentally we validate the performance of the classifier and show that it is able to outperform SVM and LSSVM on certain real-life datasets.

We present a video summarization technique based on supervised learning. Within a class of videos of similar nature, user provides the desired summaries for a subset of videos. Based on this supervised information, the summaries for other videos in the same class are generated. We derive frame-transitional features and subsequently represent each frame transition as a state. We then formulate a loss functional to quantify the discrepency between state transitional probabilities in the original video and that in the intended summary video, and optimize this functional. We experimentally validate the performance of the technique using cross-validation scores on two different class of videos, and demonstrate that the proposed technique is able to produce high quality summarization capturing the user perception.

Analyzing syllable stress in spoken English has been an area of research for a long time. In this paper, we analyze the performance of a novel method for evaluating syllable stress in spoken English. Specifically, we study the problem of determining if a word is spoken with the correct syllable stress pattern. The proposed method uses generalized models for stressed and unstressed syllables to analyze the constituent syllables of a word and determines if the word is spoken correctly. The performance of the proposed method is reported in terms of classification results on human labeled word utterances and it is compared with that of the word-dependent models using various classifiers.

Recently we have shown that decision trees can be trained in the online adaptive (OADT) mode (Basak, 2004), leading to better generalization score. OADTs were bottlenecked by the fact that they are able to handle only two-class classification tasks with a given structure. In this article, we provide an architecture based on OADT, ExOADT, which can handle multiclass classification tasks and is able to perform function approximation. ExOADT is structurally similar to OADT extended with a regression layer. We also show that ExOADT is capable not only of adapting the local decision hyperplanes in the nonterminal nodes but also has the potential of smoothly changing the structure of the tree depending on the data samples. We provide the learning rules based on steepest gradient descent for the new model ExOADT. Experimentally we demonstrate the effectiveness of ExOADT in the pattern classification and function approximation tasks. Finally, we briefly discuss the relationship of ExOADT with other classification models.

In this paper, we design a content-based image retrieval system where multiple query examples can be used to indicate the need to retrieve not only images similar to the individual examples, but also those images which actually represent a combination of the content of query images. We propose a scheme for representing content of an image as a combination of features from multiple examples. This scheme is exploited for developing a multiple example-based retrieval engine. We have explored the use of machine learning techniques for generating the most appropriate feature combination scheme for a given class of images. The combination scheme can be used for developing purposive query engines for specialized image databases. Here, we have considered facial image databases. The effectiveness of the image retrieval system is experimentally demonstrated on different databases.

In this paper, an interactive method for ranking multiple-attribute items from user's feedback using a feedforward neural network (NN) has been presented. The task of ranking multiple-attribute items is relevant in different tasks of e-commerce including request for quotes (RFQ), negotiations, personalized catalogs, profiling, and customer modeling. Often, a linear parametric function is used to model the overall value function considering known individual attribute value functions, and the parameters of the linear function are estimated by linear programming (LP). In this paper, we propose an NN-based active learning method for evaluating and ranking the multiple-attribute items (or bids in RFQ) without imposing the restriction of linear dependence between the attribute value functions. Use of an NN also relaxes the constraint of known parametric form of individual attribute value functions, which is usually assumed in LP-based methods. A suitable objective error measure is defined in this context, and correspondingly, a feedforward NN is trained to obtain the ranking of the item set (or bids). Effectiveness of the method is validated on real-life data sets.

In this article, we provide an overview of the case adaptation process. We classify various existing case adaptation methods available in the literature. We consider three different aspects, namely, domain knowledge requirement, adaptive capabilities of the case adaptation methods, and the kind of adaptation knowledge required. We then derive certain findings about the nature of the case adaptation methods and their applicability in real-life tasks. © 2005 Wiley Periodicals, Inc. Int J Int Syst 20: 627–645, 2005.

We propose a method for hierarchical clustering based on the decision tree approach. As in the case of supervised decision tree, the unsupervised decision tree is interpretable in terms of rules, i.e., each leaf node represents a cluster, and the path from the root node to a leaf node represents a rule. The branching decision at each node of the tree is made based on the clustering tendency of the data available at the node. We present four different measures for selecting the most appropriate attribute to be used for splitting the data at every branching node (or decision node), and two different algorithms for splitting the data at each decision node. We provide a theoretical basis for the approach and demonstrate the capability of the unsupervised decision tree for segmenting various data sets. We also compare the performance of the unsupervised decision tree with that of the supervised one.

Decision trees and neural networks are widely used tools for pattern classification. Decision trees provide highly localized representation, whereas neural networks provide a distributed but compact representation of the decision space. Decision trees cannot be induced in the online mode, and they are not adaptive to changing environment, whereas neural networks are inherently capable of online learning and adpativity. Here we provide a classification scheme called online adaptive decision trees (OADT), which is a tree-structured network like the decision trees and capable of online learning like neural networks. A new objective measure is derived for supervised learning with OADT. Experimental results validate the effectiveness of the proposed classification scheme. Also, with certain real-life data sets, we find that OADT performs better than two widely used models: the hierarchical mixture of experts and multilayer perceptron.

In general, pattern classification algorithms assume that all the features are available during the construction of a classifier and its subsequent use. In many practical situations, data are recorded in different servers that are geographically apart, and each server observes features of local interest. The underlying infrastructure and other logistics (such as access control) in many cases do not permit continual synchronization. Each server thus has a partial view of the data in the sense that feature subsets (not necessarily disjoint) are available at each server. In this article, we present a classification algorithm for this distributed vertically partitioned data. We assume that local classifiers can be constructed based on the local partial views of the data available at each server. These local classifiers can be any one of the many standard classifiers (e.g., neural networks, decision tree, k nearest neighbor). Often these local classifiers are constructed to support decision making at each location, and our focus is not on these individual local classifiers. Rather, our focus is constructing a classifier that can use these local classifiers to achieve an error rate that is as close as possible to that of a classifier having access to the entire feature set. We empirically demonstrate the efficacy of the proposed algorithm and also provide theoretical results quantifying the loss that results as compared to the situation where the entire feature set is available to any single classifier.

A two-layer neural-network model is designed which accepts image coordinates as the input and learns the parametric form of conoidal shapes (lines/circles/ellipses) adaptively. It provides an efficient representation of visual information embedded in the connection weights and the parameters of the processing elements. It not only reduces the large space requirements of the classical Hough transform (HT), but also represents parameters with a higher precision. The performance of the methodology is compared with other existing algorithms and has been found to excel over those algorithms in many cases.

A single-layered Hough transform network is proposed that accepts image coordinates of each object pixel as input and produces a set of outputs that indicate the belongingness of the pixel to a particular structure (e.g., a straight line). The network is able to learn adaptively the parametric forms of the linear segments present in the image. It is designed for learning and identification not only of linear segments in two-dimensional images but also the planes and hyperplanes in the higher-dimensional spaces. It provides an efficient representation of visual information embedded in the connection weights. The network not only reduces the large space requirement, as in the case of classical Hough transform, but also represents the parameters with high precision.

A connectionist model along with its state dynamics is developed for detecting corner points in binary and gray images. For a given binary/gray image, each pixel in the image is assigned with some initial cornerity (our measurable quantity) which is a vector representing the direction and strength of the corner. These cornerities are then mapped onto a neural-network model which is essentially designed as a cooperative computational framework. The cornerity at each pixel is updated depending on the neighborhood information. After the network dynamics settles to stable state, the dominant points are obtained by finding out the local maxima in the cornerities. Theoretical investigations are made to ensure the stability and convergence of the network. It is found that the network is able to detect corner points even in the noisy images and for open object boundaries. The dynamics of the network is extended to accept the edge information from gray images also. The effectiveness of the model is experimentally demonstrated in synthetic and real-life binary and gray images.

The present article demonstrates a way of formulating neuro-fuzzy approaches for both feature selection and extraction under unsupervised learning. A fuzzy feature evaluation index for a set of features is defined in terms of degree of similarity between two patterns in both the original and transformed feature spaces. A concept of flexible membership function incorporating weighted distance is introduced for computing membership values in the transformed space. Two new layered networks are designed. The tasks of membership computation and minimization of the evaluation index, through unsupervised learning process, are embedded into them without requiring the information on the number of clusters in the feature space. The network for feature selection results in an optimal order of individual importance of the features. The other one extracts a set of optimum transformed features, by projecting -dimensional original space directly to n'-dimensional (n' < n) transformed space, along with their relative importance. The superiority of the networks to some related ones is established experimentally.

A new, efficient algorithm for blind separation of uniformly distributed sources is proposed. The mixing matrix is assumed to be orthogonal by prewhitening the observed signals. The learning rule adaptively estimates the mixing matrix by conceptually rotating a unit hypercube so that all output signal components are contained within or on the hypercube. Under some ideal constraints, it has been theoretically shown that the algorithm is very similar to an ideal O(1/T2) convergent algorithm, which is much faster than the existing O(1/T) convergent algorithms. The algorithm has been generalized to take care of the noisy signals by adaptively dilating the hypercube in conjunction with its rotation.

A general algorithm for blind separation of uniformly distributed signals is presented. First maximum likelihood equations are obtained for dealing with this task. It is difficult to obtain a closed form maximum likelihood solution for arbitrary mixing matrix. The learning rules are obtained based on the geometric interpretation of the maximum likelihood estimator. The algorithm, under special constraint of orthogonal mixing matrix, is the same as the O(1/T2) convergent algorithm. Special noise correction mechanisms are incorporated in the algorithm, and it has been found that the algorithm exhibits stable performance even in the presence of large amount of noise.

A connectionist model for categorization (self-organization) even in the presence of multiple or mixed patterns has been presented. During self-organization, the network automatically adjusts the number of nodes in the hidden and output layers, depending on the complexity or nature of overlap between the patterns. An ambiguity measure is given based on how well the features are being interpreted by the network. From the ambiguity measure a certainty factor about the decision of the network is derived. The effect of noise on the certainty factor is investigated. A vigilance threshold is used to decide whether the network's decision is correct or not. Functionally the network consists of two parts, one of them categorizes the incoming patterns and the other monitors the performance of categorization. The characteristics of the model has also been demonstrated experimentally on both 1D binary strings and image patterns even when they are corrupted by additive, subtractive, and mixed noise.

A connectionist system has been designed for learning and simultaneous recognition of flat industrial objects (based an the concepts of conventional and structured connectionist computing) by integrating the psychological hypotheses with the generalized Hough transform technique. The psychological facts include the evidence of separation of two regions for identification ("what it is") and pose estimation ("where it is"). The system uses the mechanism of selective attention for initial hypotheses generation. A special two-stage training paradigm has been developed for learning the structural relationships between the features and objects and the importance values of the features with respect to the objects. The performance of the system has been demonstrated on real-life data both for single and mixed (overlapped) instances of object categories. The robustness of the system with respect to noise and false alarming has been theoretically investigated.

In this paper two connectionist models for mid-level vision problems, namely, edge and line linking, have been presented. The processing elements (PE) are arranged in the form of two-dimensional lattice in both the models. The models take the strengths and the corresponding directions of the fragmented edges (or lines) as the input. The state of each processing element is updated by the activations received from the neighboring processing elements. In one model, each neuron interacts with its eight neighbors, while in the other model, each neuron interacts over a larger neighborhood. After convergence, the output of the neurons represent the linked edge (or line) segments in the image. The first model directly produces the linked line segments, while the second model produces a diffused edge cover. The linked edge segments are found by finding out the spine of the diffused edge cover. The experimental results and the proof of convergence of the network models have also been provided.>

Structural description of objects comprised descriptions of the parts and spatial relations between the parts. This paper presents a Hopfield net based scheme for matching structural shape descriptions. The current formulation of the matching scheme is general enough to take care of partial mismatch between the individual parts and spatial constraints between these parts. In addition, a transformation of the shape descriptions has been suggested with which shape descriptions containing asymmetrical spatial constraints between the parts can be matched using symmetric interconnection weights for the Hopfield net. The Hopfield net based formulation has been extended to consider the problem of finding the best match of the test shape descriptions with one of the stored prototypes. The matching scheme has been experimentally applied for recognition of hand-tools and symbols. In both cases, the network produced encouraging recognition results.

A connectionist model for learning and recognizing objects (or object classes) is presented. The learning and recognition system uses confidence values for the presence of a feature. The network can recognize multiple objects simultaneously when the corresponding overlapped feature train is presented at the input. An error function is defined, and it is minimized for obtaining the optimal set of object classes. The model is capable of learning each individual object in the supervised mode. The theory of learning is developed based on some probabilistic measures. Experimental results are presented. The model can be applied for the detection of multiple objects occluding each other.

A connectionist network is presented for simultaneous perception of multiple categories. These categories provide an adequate explanation of the input features originating from multiple classes. The network optimises an appropriately defined error function for making the inference. A supervised learning algorithm is presented for learning the association between the features and each individual category.>