Process behavior analysis (PBA) has been applied to service quality control for decades. In IT service delivery management, PBA has been widely and successfully applied to monitor key performance indicators (KPIs) over time to identify process anomalies. However, we notice that PBA is often limited to a relatively small number of high-level indicators instead of slicing and dicing data for each KPI to find the key dimensions that highlight specific problems. This is due to the fact that without applying domain knowledge to identify relevant alerts, monitoring all possible dimensions leads to too many irrelevant alerts. In this paper, we demonstrate that, using domain-specific rules for each dimension as well as for dimension combinations, we generate targeted alerts for specific consumers. While the presented framework is generic, we demonstrate specifically how targeted alerting can be applied to the incident management process, in particular to monitoring incident volumes over time. We showcase the value of our targeted alerting mechanism using data from 18 outsourcing clients.

This paper presents our initial efforts towards building a cognitive analytics framework for change management. We propose a novel predictive algorithm for change risk calculation based on historical change failures, server failures, change triggered incidents as well as expert user input. Our predictive algorithm provides significant improvement over traditional risk assessments in proactively capturing problematic changes when tested with real client account data.

IT service providers are faced with a dilemma when trying to ensure proper function and effective operation of their clients' infrastructure. On one hand, frequent changes to the IT infrastructure are required to ensure smooth operation; on the other hand, studies show that changes are responsible for 80% of all incidents that result in client outages. This paper proposes a novel methodology for investigating the role of change in incident prevention. We provide a detailed analysis of the change-incident space, offer algorithms on linking incidents to changes that caused them, and show how such data can be effectively used to build predictive models for incident prevention. We conclude by presenting our methodology applied to a real-world dataset and use cases.

Infrastructure as a Service (IaaS) clouds provide users with the ability to easily and quickly provision servers. A recent study found that one in three data center servers continues to consume resources without producing any useful work. A number of techniques have been proposed to identify such unproductive instances. However, those approaches adopt the strategy to identify idle cloud instances based on resource utilization. Resource utilization as indicator alone could be misleading, which is especially true for enterprise cloud environment. In this paper, we present Pleco, a tool that detects unproductive instances in IaaS clouds. Pleco captures dependency information between users and cloud instances by constructing a weighted reference model based on application knowledge. To handle cases of insufficient application knowledge, Pleco also supplements its dependency results with a machine learning model trained on resource utilization data. Pleco gives a confidence level and justification for each identified unproductive instances. Cloud administrators can then take different actions according to the information provided by Pleco. Pleco is lightweight and requires no modification to existing applications.

It is common knowledge in the IT service domain that changes to the system configuration are responsible for a major portion of incidents that result in client outages. However, it is typically very difficult to establish a relationship between changes and incidents as proper documentation takes lower priority at change creation time, as well as during incident management, in order to deal with the tremendous time pressure to quickly implement changes and resolve incidents. As a result, it is often not possible to leverage historical data to perform retrospective analysis to identify any emerging trends linking changes to incidents, or to build predictive models for proactive incident prevention at change creation time. In this paper, we present an approach for establishing causality between changes and incidents through an ensemble of statistics, data classification, and natural language processing techniques. We demonstrate our approach with a real world example.

Large enterprises are confronted with an increased need to fully understand the IT systems that support their business applications. Unfortunately, conventional communication trace-based approaches to discovering IT infrastructure dependencies are insufficient to gain a comprehensive mapping of business applications to IT infrastructure. This paper describes BizMap, a novel, information-discovery-based approach for deriving accurate business application topologies. BizMap uses a combination of information management, data analysis, and interactive visualization techniques to rapidly develop accurate topology models based on discovered information and configuration experts' insight. BizMap's evaluation is based on a comparison of the topologies created by BizMap's automated topology discovery with manually derived topologies with respect to performance and accuracy. It also includes a discussion of the user validation of the interactive BizMap steps.

Master data management (MDM) integrates data from multiple structured data sources and builds a consolidated 360-degree view of business entities such as customers and products. Today's MDM systems are not prepared to integrate information from unstructured data sources, such as news reports, emails, call-center transcripts, and chat logs. However, those unstructured data sources may contain valuable information about the same entities known to MDM from the structured data sources. Integrating information from unstructured data into MDM is challenging as textual references to existing MDM entities are often incomplete and imprecise and the additional entity information extracted from text should not impact the trustworthiness of MDM data. In this paper, we present an architecture for making MDM text-aware and showcase its implementation as IBM Info-Sphere MDM Extension for Unstructured Text Correlation, an add-on to IBM InfoSphere Master Data Management Standard Edition. We highlight how MDM benefits from additional evidence found in documents when doing entity resolution and relationship discovery. We experimentally demonstrate the feasibility of integrating information from unstructured data sources into MDM.

Large-scale sensor deployments and an increased use of privacy-preserving transformations have led to an increasing interest in mining uncertain time series data. Traditional distance measures such as Euclidean distance or dynamic time warping are not always effective for analyzing uncertain time series data. Recently, some measures have been proposed to account for uncertainty in time series data. However, we show in this paper that their applicability is limited. In specific, these approaches do not provide an intuitive way to compare two uncertain time series and do not easily accommodate multiple error functions.

The ability to trace the history of individual products, especially their movement through supply and distribution chains, is key to many solutions such as targeted recalls and counterfeit detection. In most traceability applications a number of independent organizations have to work together. EPCglobal has proposed an architecture for a network of RFID databases where each database provides a standardized query interface. That architecture facilitates simple retrieval of traceability data from individual repositories, but it does not support complex traceability queries or cross-organizational query processing. Theseos (R. Agrawal, 2006) provides traceability applications with the ability to execute complex traceability queries that may span multiple RFID databases.

Tracking and tracing individual items is a new and emerging trend in many industries. Driven by maturing technologies such as radio-frequency identification (RFID) and upcoming standards such as the electronic product code (EPC), a rapidly increasing number of enterprises are collecting vast amounts of tracking data. To enable traceability over the entire life-cycle of items data has to be shared across independent and possibly competing enterprises. The need to simultaneously compete and cooperate requires a traceability system design that allows companies to share their traceability data while maintaining complete sovereignty over what is shared and with whom. Based on an extensive study of traceability applications, we introduce the formal concept of traceability networks and highlight the technical challenges involved in sharing data in such a network. To address these challenges, we present an innovative combination of query processing techniques from P2P networks and distributed as well as parallel databases with confidentiality enforcement techniques

We implemented our new approach for efficient similarity search in large databases of tree structures. Our experiments show that filtering significantly accelerates the complex task of similarity search for tree-structured objects. Moreover, they show that no single feature of a tree is sufficient for effective filtering, but only the combination of structural and content-based filters yields good results.

In high-dimensional feature spaces traditional clustering algorithms tend to break down in terms of efficiency and quality. Nevertheless, the data sets often contain clusters which are hidden in various subspaces of the original feature space. In this paper, we present a feature selection technique called SURFING (subspaces relevant for clustering) that finds all subspaces interesting for clustering and sorts them by relevance. The sorting is based on a quality criterion for the interestingness of a subspace using the k-nearest neighbor distances of the objects. As our method is more or less parameterless, it addresses the unsupervised notion of the data mining task "clustering" in a best possible way. A broad evaluation based on synthetic and real-world data sets demonstrates that SURFING is suitable to find all relevant sub-spaces in high dimensional, sparse data sets and produces better results than comparative methods.

Many clustering algorithms tend to break down in high-dimensional feature spaces, because the clusters often exist only in specific subspaces (attribute subsets) of the original feature space. Therefore, the task of projected clustering (or subspace clustering) has been defined recently. As a solution to tackle this problem, we propose the concept of local subspace preferences, which captures the main directions of high point density. Using this concept, we adopt density-based clustering to cope with high-dimensional data. In particular, we achieve the following advantages over existing approaches: Our proposed method has a determinate result, does not depend on the order of processing, is robust against noise, performs only one single scan over the database, and is linear in the number of dimensions. A broad experimental evaluation shows that our approach yields results of significantly better quality than recent work on clustering high-dimensional data.

Several application domains such as molecular biology and geography produce a tremendous amount of data which can no longer be managed without the help of efficient and effective data mining methods. One of the primary data mining tasks is clustering. However, traditional clustering algorithms often fail to detect meaningful clusters because most real-world data sets are characterized by a high dimensional, inherently sparse data space. Nevertheless, the data sets often contain interesting clusters which are hidden in various subspaces of the original feature space. Therefore, the concept of subspace clustering has recently been addressed, which aims at automatically identifying subspaces of the feature space in which clusters exist. In this paper, we introduce SUBCLU (density-connected Subspace Clustering), an effective and efficient approach to the subspace clustering problem. Using the concept of density-connectivity underlying the algorithm DBSCAN [EKSX96], SUBCLU is based on a formal clustering notion. In contrast to existing grid-based approaches, SUBCLU is able to detect arbitrarily shaped and positioned clusters in subspaces. The monotonicity of density-connectivity is used to efficiently prune subspaces in the process of generating all clusters in a bottom up way. While not examining any unnecessary subspaces, SUBCLU delivers for each subspace the same clusters DBSCAN would have found, when applied to this subspace separately.

The detection of correlations between different features in a set of feature vectors is a very important data mining task because correlation indicates a dependency between the features or some association of cause and effect between them. This association can be arbitrarily complex, i.e. one or more features might be dependent from a combination of several other features. Well-known methods like the principal components analysis (PCA) can perfectly find correlations which are global, linear, not hidden in a set of noise vectors, and uniform, i.e. the same type of correlation is exhibited in all feature vectors. In many applications such as medical diagnosis, molecular biology, time sequences, or electronic commerce, however, correlations are not global since the dependency between features can be different in different subgroups of the set. In this paper, we propose a method called 4C (Computing Correlation Connected Clusters) to identify local subgroups of the data objects sharing a uniform but arbitrarily complex correlation. Our algorithm is based on a combination of PCA and density-based clustering (DBSCAN). Our method has a determinate result and is robust against noise. A broad comparative evaluation demonstrates the superior performance of 4C over competing methods such as DBSCAN, CLIQUE and ORCLUS.