Having precise specifications of service APIs is essential for many Software Engineering activities. Unfortunately, available documentation of services is often inadequate and/or imprecise and, hence, cannot be fully relied upon. Generating service documentation manually is a tedious and error-prone task, especially in light of changes to services. Therefore, there is a need for automated support in generating service documentation. In this work, we present a novel approach to infer the API of a service by analyzing recorded messages sent to and received from this service. Our approach includes a novel, two-level clustering technique to cluster messages, a step that many existing approaches to infer message formats fail to perform precisely in the presence of significant variation of payload information of the available messages. We have evaluated our approach on message traces from four different real-world services. The experimental result shows that our approach is more effective than existing techniques in extracting correct message formats from recorded messages.

We present GHTraffic, a dataset of significant size comprising HTTP transactions extracted from GitHub data and augmented with synthetic transaction data. The dataset facilitates reproducible research on many aspects of service-oriented computing. This paper discusses use cases for such a dataset and extracts a set of requirements from these use cases. We then discuss the design of GHTraffic, and the methods and tool used to construct it. We conclude our contribution with some selective metrics that characterise GHTraffic.

APIs play a significant role in the sharing, utilization and integration of information and service assets for enterprises, delivering significant business value. However, the documentation of service APIs can often be incomplete, ambiguous, or even non-existent, hindering API-based application development efforts. In this paper, we introduce an approach to automatically mine the fine-grained message formats required in defining the APIs of services and applications from their interaction traces, without assuming any prior knowledge. Our approach includes three major steps with corresponding techniques: (1) classifying the interaction messages of a service into clusters corresponding to message types, (2) identifying the keywords of messages in each cluster, and (3) extracting the format of each message type. We have applied our approach to network traces collected from four real services which used the following application protocols: REST, SOAP, LDAP and SIP. The results show that our approach achieves much greater accuracy in extracting message formats for service APIs than current state-of-art approaches.

The landscape of modern enterprise IT environments is that a large number of distributed software systems interact and cooperate with each other to support daily business operations. With the prevalence of cloud computing, the level of system connectivity increases further. The large scale of such an environment makes it difficult to test a system's quality attributes such as performance and scalability before it is actually deployed in the production environment. Under the currently dominant iterative and incremental software development paradigm, this difficulty is even more pronounced when the quality attributes of an enterprise system need to be examined and evaluated at early stages but a large part of its operating environment is not available or accessible. In this paper, we present a Coloured Petri nets (CPN) based system behaviour emulation approach and a lightweight emulated testing framework for provisioning a virtual deployment testing environment for an enterprise software system, so that its quality attributes, especially scalability, can be evaluated without physically connecting to the real production environment. It is worth noting that the focus of this work is on the testing environment which enables virtual system deployment and testing, instead of being on the research topic of deployment test. CPN and its associated design and simulation tools have been used and integrated to model and execute the behaviour of those cooperating endpoint systems that an enterprise software system interacts with. We have successfully implemented an industry standard protocol Lightweight Directory Access Protocol in our approach and applied it in testing the scalability of a real-world enterprise application, CA Technologies’ IdentityManager. A thorough in-lab performance study has also been conducted to examine the capacity and scalability of this approach.

Summary The high volume of monitoring information generated by large‐scale cloud infrastructures poses a challenge to the capacity of cloud providers in detecting anomalies in the infrastructure. Traditional anomaly detection methods are resource‐intensive and computationally complex for training and/or detection, what is undesirable in very dynamic and large‐scale environment such as clouds. Isolation‐based methods have the advantage of low complexity for training and detection and are optimized for detecting failures. In this work, we explore the feasibility of Isolation Forest, an isolation‐based anomaly detection method, to detect anomalies in large‐scale cloud data centers. We propose a method to code time‐series information as extra attributes that enable temporal anomaly detection and establish its feasibility to adapt to seasonality and trends in the time‐series and to be applied online and in real‐time.

The emergence of cloud computing has made dynamic provisioning of elastic capacity to applications on-demand. Cloud data centers contain thousands of physical servers hosting orders of magnitude more virtual machines that can be allocated on demand to users in a pay-as-you-go model. However, not all systems are able to scale up by just adding more virtual machines. Therefore, it is essential, even for scalable systems, to project workloads in advance rather than using a purely reactive approach. Given the scale of modern cloud infrastructures generating real time monitoring information, along with all the information generated by operating systems and applications, this data poses the issues of volume, velocity, and variety that are addressed by Big Data approaches. In this paper, we investigate how utilization of Big Data analytics helps in enhancing the operation of cloud computing environments. We discuss diverse applications of Big Data analytics in clouds, open issues for enhancing cloud operations via Big Data analytics, and architecture for anomaly detection and prevention in clouds along with future research directions.

Cloud computing is a solution for addressing challenges such as licensing, distribution, configuration, and operation of enterprise applications associated with the traditional IT infrastructure, software sales and deployment models. Migrating from a traditional model to the Cloud model reduces the maintenance complexity and cost for enterprise customers, and provides on-going revenue for Software as a Service (SaaS) providers. Clients and SaaS providers need to establish a Service Level Agreement (SLA) to define the Quality of Service (QoS). The main objectives of SaaS providers are to minimize cost and to improve Customer Satisfaction Level (CSL). In this paper, we propose customer driven SLA-based resource provisioning algorithms to minimize cost by minimizing resource and penalty cost and improve CSL by minimizing SLA violations. The proposed provisioning algorithms consider customer profiles and providers' quality parameters (e.g., response time) to handle dynamic customer requests and infrastructure level heterogeneity for enterprise systems. We also take into account customer-side parameters (such as the proportion of upgrade requests), and infrastructure-level parameters (such as the service initiation time) to compare algorithms. Simulation results show that our algorithms reduce the total cost up to 54 percent and the number of SLA violations up to 45 percent, compared with the previously proposed best algorithm.

When operating in volatile environments, service-based systems (SBSs) that are dynamically composed from component services must be monitored in order to guarantee timely and successful delivery of outcomes in response to user requests. However, monitoring consumes resources and very often impacts on the quality of the SBSs being monitored. Such resource and system costs need to be considered in formulating monitoring strategies for SBSs. The critical path of a composite SBS, i.e., the execution path in the service composition with the maximum execution time, is of particular importance in cost-effective monitoring as it determines the response time of the entire SBS. In volatile operating environments, the critical path of an SBS is probabilistic, as every execution path can be critical with a certain probability, i.e., its criticality. As such, it is important to estimate the criticalities of different execution paths when deciding which parts of the SBS to monitor. Furthermore, cost-effective monitoring also requires management of the trade-off between the benefit and cost of monitoring. In this paper, we propose CriMon, a novel approach to formulating and evaluating monitoring strategies for SBSs. CriMon first calculates the criticalities of the execution paths and the component services of an SBS and then, based on those criticalities, generates the optimal monitoring strategy considering both the benefit and cost of monitoring. CriMon has two monitoring strategy formulation methods, namely local optimisation and global optimisation. In-lab experimental results demonstrate that the response time of an SBS can be managed cost-effectively through CriMon-based monitoring. The effectiveness and efficiency of the two monitoring strategy formulation methods are also evaluated and compared.

A Service Level Agreement (SLA) is a legal contract between parties to ensure the Quality of Service (QoS) are provided the providers to the customers. A SLA negotiation between participants assists in defining the QoS requirements of critical service-based processes. However, the negotiation process for customers is a significant task particularly when there are multiple SaaS providers in the Cloud market, as service cost and quality are constantly changing and consumers have varying needs. Therefore, we propose a novel automated negotiation framework where a SaaS broker is utilized as the one-stop-shop for customers to achieve the required service efficiently when negotiating with multiple providers. The automated negotiation framework facilitates intelligent bilateral bargaining of SLAs between a SaaS broker and multiple providers to achieve different objectives for different participants. To maximize profit and improve customer satisfaction levels for the broker, we propose the design of counter offer generation strategies and decision making heuristics that take into account time, market constraints and trade-off between QoS parameters. Our negotiation heuristics are evaluated by extensive experimental studies of our framework using data from a real Cloud provider.

For a large and evolving software system, the project team could receive many bug reports over a long period of time. It is important to achieve a quantitative understanding of bug-fixing time. The ability to predict bug-fixing time can help a project team better estimate software maintenance efforts and better manage software projects. In this paper, we perform an empirical study of bug-fixing time for three CA Technologies projects. We propose a Markov-based method for predicting the number of bugs that will be fixed in future. For a given number of defects, we propose a method for estimating the total amount of time required to fix them based on the empirical distribution of bug-fixing time derived from historical data. For a given bug report, we can also construct a classification model to predict slow or quick fix (e.g., below or above a time threshold). We evaluate our methods using real maintenance data from three CA Technologies projects. The results show that the proposed methods are effective.

Enterprise software systems operate in large-scale, heterogeneous, distributed environments which makes assessment of non-functional properties, such as scalability and robustness, of those systems particularly challenging. Enterprise environment emulators can provide test-beds representative of real environments using only a few physical hosts thereby allowing assessment of the non-functional properties of enterprise software systems. To date, analysing outcomes of these tests has been an ad hoc and somewhat tedious affair; largely based on manual and/or script-assisted inspection of interaction logs. Quokka visualises emulations significantly aiding analysis and comprehension. Emulated interactions can be viewed live (in real-time) as well as be replayed at a later stage, furthermore, basic charts are used to aggregate and summarise emulations, helping to identify performance and scalability issues.

The loss of control over information assets is a major security and privacy concern in the Cloud. Service consumers typically have no insights which controls protect their information assets and how effectively. To tackle this challenge, we propose an approach where service providers and consumers conjointly manage security requirements for a Cloud service following the ISO 27001 standard for information security management. We have developed a security management platform that provides tool support for service providers and consumers (i) to specify and consolidate security requirements and (ii) to collect, measure, analyse and report information about the effectiveness of implemented controls. By involving service consumers in management activities following an international standard, our approach helps service providers to increase transparency and traceability of their security measures whereas service consumers gain much-needed insights in the protection of their information assets. The applicability of our approach is demonstrated with an example scenario.

Information technology is playing a more and more critical role in many large organizations and enterprise software systems have become increasingly integrated to better support the goals of these organizations. As a consequence, it is a major engineering challenge to assure quality of a software system that is to be deployed in an enterprise environment containing many thousands of interconnected systems. To address this challenge, we propose the use of a scalable emulation environment enabling real-time system testing in large-scale settings. In this work, we discuss the main concepts of our approach, present Reac2o, a prototype implementation of such an emulation environment, and illustrate its applicability in the context of identity management.

Current Web Service security standards have inadequate support for end-to-end protection of data when some receivers of the data are unknown to the sender. This paper presents an approach to aid collaborative partner services in properly protecting each other's data. Our approach allows each partner to derive an adequate protection mechanism with minimum performance overhead for each message it sends based on those of the corresponding messages it receives. We modify the message handling mechanisms of Web Service engines to dynamically gather protection requirements for a given outgoing message by aggregating requirements from original owners of message data.

Role based access control (RBAC) is a powerful security administration concept that can simplify permission assignment management. Migration to and maintenance of RBAC requires role engineering, the identification of a set of roles that offer administrative benefit. However, establishing that RBAC is desirable in a given enterprise is lacking in current role engineering processes. To help identify the practical need for RBAC, we propose RoleVAT, a Role engineering tool for the Visual Assessment of user and permission Tendencies. User and permission clusters can be visually identified as potential user groups or roles. The benefit and impact of this visual analysis in enterprise environments is discussed and demonstrated through testing on real life as well as synthetic datasets. Our experimental results show the effectiveness of RoleVAT as well as interesting user and role tendencies in real enterprise environments.