Existing approaches to recommend services using natural language queries are supervised or unsupervised. Supervised approaches rely on a dataset with natural language queries annotated with categorizing labels. As the annotation process is manual and requires deep domain knowledge, these approaches are not readily applicable on new datasets. On the other hand, unsupervised approaches overcome the limitation. To date, unsupervised approaches are primarily based on matching keywords, entity relationships, topics and clusters. Keywords and entity relationships ignore the semantic similarity between a query and services. Topics and clusters capture the semantic similarity, but rely on mashups that explicitly capture relationships between services. Again, for new services, the information are not readily available. We propose NL2API, a framework that relies solely on service descriptions for recommending services. NL2API has the benefit of being immediately applicable as a bootstrap recommender for new datasets. To capture relationships among services, NL2API provides different approaches to construct communities where a community represents an abstraction over a group of services. Based on the communities and users' queries, NL2API applies a query matching approach to recommend top-k services. We evaluate NL2API on datasets collected from Programmable Web and API Harmony. Our evaluation shows that for sizable datasets such as Programmable Web NL2API outperforms baseline approaches.

IT compliance is an area of increasing attention and capital spend in enterprise IT environments. We present "Continuous Compliance", a framework that allows a managed IT services provider to automate the overall process of keeping IT assets conformant with enterprise policies, regulatory frameworks, and other best practices. Our framework applies to all cloud layers and service models: Infrastructure-, Platform-, and Software-as-a-Service. We describe our framework design, its operation, and the post-process analytics and reporting. We also examine remediation reports gathered from over 2,000 servers for a seven month period, graph the incidence of repeated remediations, and explore some reasons for gradually subsiding remediations.

Presented BlueWall: Software Defined Network Management in Hybrid Enterprise Cloud Environments for managing firewall request when servers (or services) are created (via APIs). Demonstrated hybrid network management design and self-service capabilities. Discussed challenges arising in network management in the hybrid enterprise cloud environments.

Presented BlueShift: Self-service for orchestrating automated tasks (via APIs) and human tasks for end-to-end transformation process including, application discovery, analysis, artifact transformation and enablement of cloud value-add services. Demonstrated transformation process for PlantsByWebSphere application, with Liberty Profile runtime as a target in BlueMix (cloud-foundry based platform). Discussed challenges arising in transformation from application complexity and non-functional requirements.

In the last few years, changing infrastructure and business requirements are forcing enterprises to rethink their networks. Enterprises look for network infrastructures that increase network efficiency, flexibility, and cost reduction. At the same time, the emergence of Cloud and mobile in enterprise networks has introduced tremendous variability in enterprise traffic patterns at the edge. This highly mobile and dynamic traffic presents a need for dynamic capacity management and adaptive traffic steering and appeals for new infrastructures and management solutions. In this context, passive optical networks (PON) have gained attention in the last few years as a promising solution for enterprise networks, as it can offer efficiency, security, and cost reduction. However, network management in PON is not yet automated and needs humain intervention. As such, capabilities for dynamic and adaptive PON are necessary. In this paper, we present a joint solution for PON capacity management both in deployment and in operation, as to maximize peak load tolerance by dynamically allocating capacity to fit varying and migratory traffic loads. To this end, we developed the novel approaches of capacity pool based deployment and dynamic traffic steering in PON. Compared with traditional edge network design, our approach significantly reduces the need for capacity over-provisioning. Compared with generic PON networks, our approach enables dynamic traffic steering through software-defined control. We implemented our design on a production grade PON testbed, and the results demonstrate the feasibility and flexibility of our approach.

With the promise of low-cost access to flexible and elastic resources, enterprises are increasingly migrating their existing workloads into the Cloud. Yet, the heterogeneity of the workloads and existing configuration of legacy IT infrastructure make it challenging to enable a one-click, seamless migration process. There are multiple tools available for migrating servers based on their existing configurations and multiple ways of dealing with data synchronization (post migration). In this paper, we present a Cloud Migration Orchestrator (CMO), based on business process management (BPM) approach to provide a systematic framework to automate and coordinate migration activities. CMO coordinates the process of migration, starting from discovery, provisioning, network configuration, execution of migration, cutover and validation. CMO integrates multiple migration technologies, to support different migration scenarios. We present and discuss our results from a preliminary deployment of CMO to migrate 25 VMware instances and discuss how this approach improves the effectiveness of migration, and seamlessly coordinates activities required to be executed.

Recent advances in the manufacturing and commercialization of passive optical components are now extending the capabilities of fiber to edge and campus networks. This paper presents a comparison between Passive Optical LAN (POL) and copper-based LAN solution, and demonstrate the benefits of PON such as reduced infrastructure footprint and cost, reduced power requirements, future-proof bandwidth, greener infrastructure, safer, higher security and higher reliability.

This paper presents Latent Association Analysis (LAA), a generative model that analyzes the topics within two document sets simultaneously, as well as the correlations between the two topic structures, by considering the semantic associations among document pairs. LAA defines a correlation factor that represents the connection between two documents, and considers the topic proportion of paired documents based on this factor. Words in the documents are assumed to be randomly generated by particular topic assignments and topic-to-word probability distributions. The paper also presents a new ranking algorithm, based on LAA, that can be used to retrieve target documents that are potentially associated with a given source document. The ranking algorithm uses the latent factor in LAA to rank target documents by the strength of their semantic associations with the source document. We evaluate the LAA algorithm with real datasets, specifically, the IT-Change and the IT-Solution document sets from the IBM IT service environment and the Symptom-Treatment document sets from Google Health. Experimental results demonstrate that the LAA algorithm significantly outperforms existing algorithms.

Cloud computing promises high scalability, flexibility and cost-effectiveness to satisfy emerging computing requirements. To efficiently provision computing resources in the cloud, system administrators need the capabilities of characterizing and predicting workload on the Virtual Machines (VMs). In this paper, we use data traces obtained from a real data center to develop such capabilities. First, we search for repeatable workload patterns by exploring cross-VM workload correlations resulted from the dependencies among applications running on different VMs. Treating workload data samples as time series, we develop a co-clustering technique to identify groups of VMs that frequently exhibit correlated workload patterns, and also the time periods in which these VM groups are active. Then, we introduce a method based on Hidden Markov Modeling (HMM) to characterize the temporal correlations in the discovered VM clusters and to predict variations of workload patterns. The experimental results show that our method can not only help better understand group-level workload characteristics, but also make more accurate predictions on workload changes in a cloud.

In recent years, the IT services industry is under continuous pressure to improve the quality of its services, while doing so at reduced prices. In order to determine quality, the IT industry seems to be converging towards a set of commonly used metrics, including but not limited to, equipment availability, the time to resolve incidents, etc. In order to improve the quality of their services, service providers need to adopt a consistent and continuous focus on their internal processes, the skills of their people, the organizational structure and so on. On the other hand, the IT service industry has been traditionally labor intensive. Even though a wide range of management tools is available to support even the most complex management tasks, the overall service management process still relies on the existence of humans to make important decisions, perform complex administration tasks and monitor the performance and effectiveness of the overall process. This paper studies the tradeoff between the cost of a service (as manifested by the staffing level) and the corresponding quality metrics (subject to the service level agreement). In particular, we study the cost of service quality through the use of an optimization model that takes into account the constraints and cost factors typically encountered in a service provider environment.

Mashups are powerful applications created from accessing and composing multiple and distributed information sources. Their ease-of-use and modularity allow users at any skill level to construct, share and integrate their own applications. However, data security concerns remain a hindering factor in its widespread adoption, in particular, for network management. In this paper, we propose a novel development methodology and system architecture called Maestro that allows developers to express their data privacy concerns and enforce policies during mashup executions. We evaluated Maestro by building two mashup applications for managing live networks and by running performance tests that show that our runtime has negligible overhead.

Problem resolution is a key issue in the IT service industry. A large service provider handles, on daily basis, thousands of tickets that report various types of problems from its customers. The efficiency of this process highly depends on the effective interactions among various expert groups, in search of the resolver to the reported problem. In fact, ticket transfer decisions reflect the expertise awareness between groups, thus encoding a sophisticated resolution social network. In this paper, we propose a computational framework to quantitatively assess expertise awareness, i.e., how well a group knows the expertise of others. An accurate assessment of expertise awareness could identify the weakest components in a resolution system. The framework, built on our previously developed resolution engine, is able to calculate the performance difference caused by excluding a node from the network. The difference exposes the awareness of this node to other nodes in the network. To our best knowledge, this is the first study on this problem from a computational perspective. We tested the proposed framework on a large set of real-world problem tickets and validated our discovery by carefully analyzing the tickets that are incorrectly transferred. Experimental results show that our framework can successfully capture groups that do not know others' expertise very well.

Many applications generate a large volume of parallel document collections. A parallel document collection consists of two sets of documents where the documents in each set correspond to each other and form semantic pairs (e.g., pairs of problem and solution descriptions in a help-desk setting). Although much work has been done on text mining, little previous work has attempted to mine such a novel kind of text data. In this paper, we propose a new probabilistic topic model, called Probabilistic Topic Mapping (PTM) model, to mine parallel document collections to simultaneously discover latent topics in both sets of documents as well as the mapping of topics in one set to those in the other. We evaluate the PTM model on one real parallel document collection in IT service domain. We show that PTM can effectively discover meaningful topics, as well as their mappings, and it's also useful for improving text matching and retrieval when there's a vocabulary gap.

Ticket resolution is a critical, yet challenging, aspect of the delivery of IT services. A large service provider needs to handle, on a daily basis, thousands of tickets that report various types of problems. Many of those tickets bounce among multiple expert groups before being transferred to the group with the right expertise to solve the problem. Finding a methodology that reduces such bouncing and hence shortens ticket resolution time is a long-standing challenge. In this paper, we present a unified generative model, the Optimized Network Model (ONM), that characterizes the lifecycle of a ticket, using both the content and the routing sequence of the ticket. ONM uses maximum likelihood estimation, to represent how the information contained in a ticket is used by human experts to make ticket routing decisions. Based on ONM, we develop a probabilistic algorithm to generate ticket routing recommendations for new tickets in a network of expert groups. Our algorithm calculates all possible routes to potential resolvers and makes globally optimal recommendations, in contrast to existing classification methods that make static and locally optimal recommendations. Experiments show that our method significantly outperforms existing solutions.

Strategic Outsourcing (SO) services allow clients to offload the operation of IT systems and processes to a specialized service provider, so they can focus on their core business functions. SO projects are multi-year engagements and are governed by a myriad of factors throughout their lifetime. The goal of service management is to ensure uninterrupted delivery of service from the provider to the client, while meeting a number of quality and performance goals. For the service provider, such objectives include continuous profitability of the contract, high execution quality and ultimately, high client satisfaction. The latter requires quality in delivery (as evidenced by a number of key performance indicators), controlled cost, and service responsiveness.

Services Quality is an area of opportunity for IT service providers to innovate and deliver outstanding results to their customers. The objective is to minimize the variance of key quality indicators and deliver predictable capabilities. In this paper we adopt the Lean Sigma methodology from the manufacturing domain as a quality control framework and propose an optimized system model for managing predictability and reducing cost in the IT incident management process. The model establishes guidelines for receiving, classifying and distributing work in a service delivery organization. It defines metrics, controls and management objectives. Using simulation, we conduct an extensive numerical study to show how the model behaves under different operational scenarios that reflect a diverse skill base, the presence of service level objectives, and incoming work with varying levels of complexity. In addition, we provide managerial insight into what drives performance and illustrate the trade-offs between different service delivery designs.

Large IT service providers track service requests and their execution through problem/change tickets. It is important to classify the tickets based on the problem/change description in order to understand service quality and to optimize service processes. However, two challenges exist in solving this classification problem: 1) ticket descriptions from different classes are of highly diverse characteristics, which invalidates most standard distance metrics; 2) it is very expensive to obtain high-quality labeled data. To address these challenges, we develop two seemingly independent methods 1) discriminative neighborhood metric learning (DNML) and 2) active learning with median selection (ALMS), both of which are, however, based on the same core technique: iterated representative selection. A case study on real IT service classification application is presented to demonstrate the effectiveness and efficiency of our proposed methods.

Motivated by the enormous amounts of data collected in a large IT service provider organization, this paper presents a method for quickly and automatically summarizing and extracting meaningful insights from the data. Termed Clustered Subset Selection (CSS), our method enables program-guided data explorations of high-dimensional data matrices. CSS combines clustering and subset selection into a coherent and intuitive method for data analysis. In addition to a general framework, we introduce a family of CSS algorithms with different clustering components such as k-means and Close-to-Rank-One (CRO) clustering, and Subset Selection components such as best rank-one approximation and Rank-Revealing QR (RRQR) decomposition.

IT problem management calls for quick identification of resolvers to reported problems. The efficiency of this process highly depends on ticket routing---transferring problem ticket among various expert groups in search of the right resolver to the ticket. To achieve efficient ticket routing, wise decision needs to be made at each step of ticket transfer to determine which expert group is likely to be, or to lead to the resolver.

Managing problem tickets is a key issue in IT service industry. A large service provider may handle thousands of problem tickets from its customers on a daily basis. The efficiency of processing these tickets highly depends on ticket routing---transferring problem tickets among expert groups in search of the right resolver to the ticket. Despite that many ticket management systems are available, ticket routing in these systems is still manually operated by support personnel. In this demo, we introduce EasyTicket, a ticket routing recommendation engine that helps automate this process. By mining ticket history data, we model an enterprise social network that represents the functional relationships among various expert groups in ticket routing. Based on this network, our system then provides routing recommendations to new tickets. Our experimental studies on 1.4 million real-world problem tickets show that on average, EasyTicket can improve the efficiency of ticket routing by 35%.

The continuous introduction of converged services such as VoIP and Video-On-Demand has created many operational challenges for service providers. In this paper, we describe how to use location-aware technologies, not only to integrate disparate management applications, but also to transform the underlying process to use geographical views as the focal point of management operations. Based on an engagement with a large Cable provider, we have designed and implemented 3i - Integrated Infrastructure Intelligence - to address key issues in the service assurance process. 3i is highly componentized and provides an intuitive way for creating role-based views through dynamic scoping, event aggregation and status projection, and location-driven active probing. We analyze the current service assurance process and compare it with the improved process after introducing 3i. Overall, the re-engineered process offers base execution improvements in alarm collection, problem drill-down and reporting, as well as complexity improvements throughout. 3i is a fully implemented tool and has demonstrated capabilities beyond its original intended scope as a decision support tool in planning and marketing functions.

An area of unique opportunity for SaaS is the mobile and ubiquitous computing space. The SaaS model has a number of advantages that are uniquely suited to a resource-constrained mobile computing environment. This short paper gives a high level overview of the SaaS which brings significant opportunities in this space and further presents the architectural considerations for SaaS providers and mobile computing environments

Clustered enterprise middleware systems employing dynamic workload scheduling are susceptible to a variety of application malfunctions that can manifest themselves in a counterintuitive fashion and cause debilitating damage. Until now, diagnosing problems in that domain involves investigating log files and configuration settings and requires in-depth knowledge of the middleware architecture and application design. This paper presents a method for problem determination using change point detection techniques and problem signatures consisting of a combination of changes (or absence of changes) in different metrics. We implemented this approach on a clustered middleware system and applied it to the detection of the storm drain condition: a debilitating problem encountered in clustered systems with counterintuitive symptoms. Our experimental results show that the system detects 93% of storm drain faults with no false positives.

Web applications have assumed a central role in the enterprise data center, and are becoming the foundation of an increasing number of critical business functions. The availability of such applications is a problem of critical importance. To address this problem, we present a health monitoring and control system for application server environments. We build on past work in the area of measuring system health by evaluating a number of parameters that convey health information. We propose a class-based model that assigns health policies to application servers to monitor health conditions of interest and correct problems before they cause catastrophic damage. We describe an implementation of the system, and provide a utility-based experimental evaluation.

Summary form only given. The vision of pervasive and ubiquitous computing was set forth in the early 90s, calling for a world of people and environments augmented with computational resources that provide access to information and services anytime, anywhere. Since then, part of this vision has been realized. The first part of this tutorial presents a comprehensive overview of pervasive computing and its evolution in the last 10 years. Particular attention is given to three areas where most research and development work has focused: natural interfaces (displays, speech, input devices), context awareness, and automated capture and access of live experiences. In addition, we explore a number of systems issues common in the development of pervasive applications, such as networking techniques for creating a pervasive communications infrastructure (especially in the presence of intermittent connectivity), middleware and operating system support, security and privacy. The second part provides an overview of the still nascent field of management of pervasive environments. We outline the challenges and present the latest work in the area: problem domains, architectures and implementations.

We are currently using the Marvel distributed management environment to provide management functionality for core Pronto services and service level router (SLR) functionality. The Marvel system, composed of a set of servers, is responsible for communicating with Pronto nodes, collecting information and providing the necessary controls for configuring SLR and hosting sites. Management information is provided to management clients according to three main views: platform management (network element layer), service management-network operator's view, service management-customer view.

Marvel is a distributed computing environment that allows the creation of scalable management services using intelligent agents and the World-Wide Web. Marvel is based on an information model that generates computed views of management information and a distributed computing model that makes this information available to a variety of client applications. Marvel does not replace existing element management agents but rather builds on top of them a hierarchy of servers that aggregate the underlying information in a synchronous or asynchronous fashion and present it in the form of Java-enriched Web pages. It uses a distributed database to reduce the cost associated with centralized network management systems and mobile agent technology to: (a) support thin clients by uploading the necessary code to access Marvel services; and (b) extend its functionality dynamically by downloading code that incorporates new objects and services. A prototype implementation in Java is presented together with results from its first application on a broadband home access network using cable modems.

Packet telephony is of increasing interest in both the telecommunications and Internet communities. The emergence of packet telephony will create new services, and presents an opportunity to rethink how conventional telephony services are implemented. In this paper, we present an architecture for telephony over packet networks (TOPS). TOPS allows users to move between terminals or to use mobile terminals while being reachable by the same name. TOPS users can have multiple terminals and control how calls are routed to them. TOPS allows for terminals with a range of capabilities such as support for video, whiteboard, and other media with a variety of coding formats. TOPS retains the necessary information on terminal capabilities to determine the appropriate type of communication to be established with the remote terminal. The architecture assumes that the underlying network supports the establishment of end-to-end connectivity between terminals, with an appropriate quality of service. The components of TOPS are a directory service, an application layer signaling protocol, and a logical channel abstraction for communication between end-systems. The directory service maps a user's name to a set of terminals where the user may be reached. A user can control the translation operation by specifying profiles that customize how his name is mapped to a set of terminals where he can be reached. Terminal capabilities are also stored in the directory service. The application layer signaling protocol establishes and maintains call state between communicating terminals. The logical channel abstraction provides a shared end-to-end context for a call's constituent media and control streams, while isolating the applications from the details of the network transport mechanisms. In addition to supporting simple point-to-point calls, the architecture supports both centralized and decentralized conferencing. We also introduce a simple encapsulation format for voice.

The configuration of virtual path (VP) connection services is expected to play an important role in the operation of large-scale asynchronous transfer mode (ATM) networks. A major research challenge is to understand the fundamental tradeoff between the network call throughput and the processing load on the signaling system and to provide an algorithm for VP capacity allocation that achieves an optimal network operating point while guaranteeing quality of service (QoS) at the call level and satisfies a priori bounds on the processing load of the call processors. We present a taxonomy of previous approaches to the problem and identify their strengths and weaknesses. Based on these observations, we provide an algorithm for the VP capacity allocation problem that satisfies nodal constraints on the call processing load and blocking constraints for each source-destination (SD) pair. The algorithm maximizes the network revenue under the above set of constraints and is parameterized by the number of traffic classes in the network, the method of presentation of networking resources, the admission control policy used in every link and VP, and the network routing scheme. Finally, we apply the algorithm to three sample networks and study several of its performance characteristics. In one case, we applied the calculated VP distribution to the Xunet ATM testbed and verified experimentally the predicted performance.