Context: Coordination in large-scale software development is critical yet difficult, as it faces the problem of dependency management and resolution. In this work, we focus on managing requirement dependencies that in Agile software development (ASD) come in the form of user stories. Objective: This work studies decisions of large-scale Agile teams regarding identification of dependencies between user stories. Our goal is to explain detection of dependencies through users’ behavior in large-scale, distributed projects. Method: We perform empirical evaluation on a large real-world dataset from an Agile software organization, provider of a leading software for Agile project management. We mine the usage data of the Agile Lifecycle Management (ALM) tool to extract large-scale development project data for more than 70 teams running over a five-year period. Results: Our results demonstrate that dependencies among user stories are not frequently observed (the problem affects around 10% of user stories), however, their implications on large-scale ASD are considerable. Dependencies have impact on software releases and increase work coordination complexity for members of different teams. Conclusion: Requirement dependencies undermine Agile teams’ autonomy and are difficult to manage at scale. We conclude that leveraging ALM monitoring data to automatically detect dependencies could help Agile teams address work coordination needs and manage risks related to dependencies in a timely manner.

Industry in all sectors is experiencing a profound digital transformation that puts software at the core of their businesses. To react to continuously changing user requirements and dynamic markets, companies need to build robust workflows that allow them to increase their agility in order to remain competitive. This increasingly rapid transformation, especially in domains such as Internet of things or cloud computing, poses significant challenges to guarantee high‐quality software, since dynamism and agile short‐term planning reduce the ability to detect and manage risks. In this study, the authors describe the main challenges related to managing risk in agile software development, building on the experience of more than 20 agile coaches operating continuously for 15 years with hundreds of teams in industries in all sectors. They also propose a framework to manage risks that consider those challenges and supports collaboration, agility, and continuous development. An implementation of that framework is then described in a tool that handles risks and mitigation actions associated with the development of multi‐cloud applications. The methodology and the tool have been validated by a team of evaluators that were asked to consider its use in developing an urban smart mobility service and an airline flight scheduling system.

Compliance with the new European General Data Protection Regulation (Regulation (EU) 2016/679, GDPR) and security assurance are currently two major challenges of Cloud‐based systems. GDPR compliance implies both privacy and security mechanisms definition, enforcement and control, including evidence collection. This study presents a novel DevOps framework aimed at supporting Cloud consumers in designing, deploying and operating (multi)Cloud systems that include the necessary privacy and security controls for ensuring transparency to end‐users, third parties in service provision (if any) and law enforcement authorities. The framework relies on the risk‐driven specification at design time of privacy and security level objectives in the system service level agreement and in their continuous monitoring and enforcement at runtime.

Performing diagnostics in IT systems is an increasingly complicated task, and it is not doable in satisfactory time by even the most skillful operators. Systems and their architecture change very rapidly in response to business and user demand. Many organizations see value in the maintenance and management model of NoOps that stands for No Operations. One of the implementations of this model is a system that is maintained automatically without any human intervention. The path to NoOps involves not only precise and fast diagnostics but also reusing as much knowledge as possible after the system is reconfigured or changed. The biggest challenge is to leverage knowledge on one IT system and reuse this knowledge for diagnostics of another, different system. We propose a framework of weighted graphs which can transfer knowledge, and perform high-quality diagnostics of IT systems. We encode all possible data in a graph representation of a system state and automatically calculate weights of these graphs. Then, thanks to the evaluation of similarity between graphs, we transfer knowledge about failures from one system to another and use it for diagnostics. We successfully evaluate the proposed approach on Spark, Hadoop, Kafka and Cassandra systems.

Although crowdsourcing has been proven efficient as a mechanism to solve independent tasks for on-line production, it is still unclear how to define and manage workflows in complex tasks that require the participation and coordination of different workers. Despite the existence of different frameworks to define workflows, we still lack a commonly accepted solution that is able to describe the most common workflows in current and future platforms. In this paper, we propose CrowdWON, a new graphical framework to describe and monitor crowd processes, the proposed language is able to represent the workflow of most well-known existing applications, extend previous modelling frameworks, and assist in the future generation of crowdsourcing platforms. Beyond previous proposals, CrowdWON allows for the formal definition of adaptative workflows, that depend on the skills of the crowd workers and/or process deadlines. CrowdWON also allows expressing constraints on workers based on previous individual contributions. Finally, we show how our proposal can be used to describe well known crowdsourcing workflows.

Internet and communication technologies have lowered the costs for communities to collaborate, leading to new services and collectively built infrastructures like community networks. Community networks get formed when individuals and local organisations from a geographic area team up to create and run a community-owned IP network to satisfy the community's demand for ICT, such as facilitating Internet access and providing services of local interest. To address the limitation and enhance utility of community networks, we deploy collaborative clouds in community networks that allow interesting applications to be developed for serving local needs of communities. Such collaborative clouds employ resources contributed by the members of the community network for provisioning infrastructure and software services, and adapt to the specific social, economic and technical characteristics of the community networks. We need to support mechanisms that provide assistance in cloud service selection while taking into account different aspects pertaining to associated risks in community clouds, quality concerns of the users and cost limitations specifically in multi-clouds ecosystems. This paper proposes a risk-cost-quality based decision support system to assist the community cloud users to select the most appropriate cloud services meeting their needs. The proposed framework not only increases the ease of adoption of community clouds by providing assistance to users in cloud service selection, but also provides insights into the improvement of community clouds based on user behaviour.

The growth in the number of cloud computing users has led to the availability of a variety of cloud based services provided by different vendors. This has made the task of selecting suitable set of services quite difficult. There has been a lot of research towards the development of suitable decision support system (DSS) to assist users in making an optimal selection of cloud services. However, existing decision support systems cannot address two crucial issues: firstly, the involvement of both business and technical perspectives in decision making simultaneously and, secondly, the multiple-clouds services based selection using single DSS. In this paper, we tackle these issues in the light of solving the problem of cloud service discovery. In particular, we present the following novel contributions: Firstly, we present critical analysis of the state-of-the-art in decision support systems. Based on our analysis, we identify critical shortcomings in the existent tools and develop the set of requirements which should be met by a potential DSS. Secondly, we present a new holistic framework for the development of DSS which allows a pragmatic description of user requirements. Additionally, the data gathering and analysis is studied as an integral part of the proposedDSS and therefore, we present concrete algorithms to assess the data for an optimal service discovery. Thirdly, we assess our framework for applicability to cloud service selection using an industrial case study. We also demonstrate the implementation and performance of our proposed framework using a prototype which serves as a proof of concept. Overall, this paper provides novel and holistic framework for development of a multiple cloud service discovery based decision support system.

The increasing adoption of crowdsourcing for commercial and industrial purposes rises the need for creating sophisticated mechanisms in crowd-based digital platforms for efficient worker management. One of the main challenges in this area is worker motivation and skill set control and its impact on the output quality. The quality delivered by the workers in the crowd depends on different aspects such as their skills, experience, commitment, etc. The lack of generic and detailed proposals to incentive workers and the need for creating ad-hoc solutions depending on the domain make it difficult to evaluate the best rewarding functions in each scenario. In this paper, we make a step further in this direction and propose the use of aggregation functions to evaluate the professional skills of crowd-workers based on the quality of their past tasks. Additionally, we present a real industrial crowdsourcing solution for software localisation in which the proposed solutions are put into practice with real text translations quality measures.

Anonymization techniques that provide k-anonymity suffer from loss of quality when data dimensionality is high. Microaggregation techniques are not an exception. Given a set of records, attributes are grouped into non-intersecting subsets and microaggregated independently. While this improves quality by reducing the loss of information, it usually leads to the loss of the k-anonymity property, increasing entity disclosure risk. In spite of this, grouping attributes is still a common practice for data sets containing a large number of records. Depending on the attributes chosen and their correlation, the amount of information loss and disclosure risk vary. However, there have not been serious attempts to propose a way to find the best way of grouping attributes. In this paper, we present GOMM, the Genetic Optimizer for Multivariate Microaggregation which, as far as we know, represents the first proposal using evolutionary algorithms for this problem. The goal of GOMM is finding the optimal, or near-optimal, attribute grouping taking into account both information loss and disclosure risk. We propose a way to map attribute subsets into a chromosome and a set of new mutation operations for this context. Also, we provide a comprehensive analysis of the operations proposed and we show that, after using our evolutionary approach for different real data sets, we obtain better quality in the anonymized data comparing it to previously used ad-hoc attribute grouping techniques. Additionally, we provide an improved version of GOMM called D-GOMM where operations are dynamically executed during the optimization process to reduce the GOMM execution time.

With the unstoppable growth of applications requiring data to be represented as graphs, the interest for keeping this type of data private also grows. While many efforts have been made in order to anonymize tabular data, anonymizing graphs is a recen

The increasing amount of graph like data from social networks, science and the web has grown an interest in analyzing the relationships between different entities. New specialized solutions in the form of graph databases, which are generic and able to adapt to any schema as an alternative to RDBMS, have appeared to manage attributed multigraphs efficiently. In this paper, we describe the internals of DEX graph database, which is based on a representation of the graph and its attributes as maps and bitmap structures that can be loaded and unloaded efficiently from memory. We also present the internal operations used in DEX to manipulate these structures. We show that by using these structures, DEX scales to graphs with billions of vertices and edges with very limited memory requirements. Finally, we compare our graph-oriented approach to other approaches showing that our system is better suited for out-of-core typical graph-like operations.

As the complexity of enterprise systems increases, the need for monitoring and analyzing such systems also grows. A number of companies have built sophisticated monitoring tools that go far beyond simple resource utilization reports. For example, based on instrumentation and specialized APIs, it is now possible to monitor single method invocations and trace individual transactions across geographically distributed systems. This high-level of detail enables more precise forms of analysis and prediction but comes at the price of high data rates (i.e., big data). To maximize the benefit of data monitoring, the data has to be stored for an extended period of time for ulterior analysis. This new wave of big data analytics imposes new challenges especially for the application performance monitoring systems. The monitoring data has to be stored in a system that can sustain the high data rates and at the same time enable an up-to-date view of the underlying infrastructure. With the advent of modern key-value stores, a variety of data storage systems have emerged that are built with a focus on scalability and high data rates as predominant in this monitoring use case. In this work, we present our experience and a comprehensive performance evaluation of six modern (open-source) data stores in the context of application performance monitoring as part of CA Technologies initiative. We evaluated these systems with data and workloads that can be found in application performance monitoring, as well as, on-line advertisement, power monitoring, and many other use cases. We present our insights not only as performance results but also as lessons learned and our experience relating to the setup and configuration complexity of these data stores in an industry setting.

The need for managing massive attributed graphs is becoming common in many areas such as recommendation systems, proteomics analysis, social network analysis or bibliographic analysis. This is making it necessary to move towards parallel systems that allow managing graph databases containing millions of vertices and edges. Previous work on distributed graph databases has focused on finding ways to partition the graph to reduce network traffic and improve execution time. However, partitioning a graph and keeping the information regarding the location of vertices might be unrealistic for massive graphs. In this paper, we propose Parallel-GDB, a new system based on specializing the local caches of any node in this system, providing a better cache hit ratio. ParallelGDB uses a random graph partitioning, avoiding complex partition methods based on the graph topology, that usually require managing extra data structures. This proposed system provides an efficient environment for distributed graph databases.

Finding decompositions of a graph into a family of clusters is crucial to understanding its underlying structure. While most existing approaches focus on partitioning the nodes, real-world datasets suggest the presence of overlapping communities. We present OCA, a novel algorithm to detect overlapped communities in large data graphs. It outperforms previous proposals in terms of execution time, and efficiently handles large graphs containing more than 108nodes and edges.

With the increase of available public data sources and the interest for analyzing them, privacy issues are becoming the eye of the storm in many applications. The vast amount of data collected on human beings and organizations as a result of cyberinfrastructure advances, or that collected by statistical agencies, for instance, has made traditional ways of protecting social science data obsolete. This has given rise to different techniques aimed at tackling this problem and at the analysis of limitations in such environments, such as the seminal study by Aggarwal of anonymization techniques and their dependency on data dimensionality. The growing accessibility to high-capacity storage devices allows keeping more detailed information from many areas. While this enriches the information and conclusions extracted from this data, it poses a serious problem for most of the previous work presented up to now regarding privacy, focused on quality and paying little attention to performance aspects. In this workshop, we want to gather researchers in the areas of data privacy and anonymization together with researchers in the area of high performance and very large data volumes management. We seek to collect the most recent advances in data privacy and anonymization (i.e. anonymization techniques, statistic disclosure techniques, privacy in machine learning algorithms, privacy in graphs or social networks, etc) and those in High Performance and Data Management (i.e. algorithms and structures for efficient data management, parallel or distributed systems, etc).

Efficiency of memory-intensive operations is a key factor in obtaining good performance during multi-join query processing. The pipelined execution of these queries forces the operations in the query plan to be processed concurrently. Making a wrong decision regarding the amount of memory allocated for such operations can have a drastic impact on the response time. However, some of the execution algorithms used at run time interrupt the pipelined execution, ensuring that some operations are never executed concurrently. Because of this, it is essential to explore new approaches in order to improve memory exploitation.

In this demonstration we show the Bibliographic Exploration tool BIBEX. BIBEX is based on the graph database query engine DEX and integrates both the Citeseer and DBLP databases. BIBEX can be found in our web site at www.dama.upc.edu/bibex.

Link and graph analysis tools are important devices to boost the richness of information retrieval systems. Internet and the existing social networking portals are just a couple of situations where the use of these tools would be beneficial and enriching for the users and the analysts. However, the need for integrating different data sources and, even more important, the need for high performance generic tools, is at odds with the continuously growing size and number of data repositories.

Record Linkage (RL) is an important component of data cleansing and integration. For years, many efforts have focused on improving the performance of the RL process, either by reducing the number of record comparisons or by reducing the number of attribute comparisons, which reduces the computational time, but very often decreases the quality of the results. However, the real bottleneck of RL is the post-process, where the results have to be reviewed by experts that decide which pairs or groups of records are real links and which are false hits. In this paper, we show that exploiting the relationships (e.g. foreign key) established between one or more data sources, makes it possible to find a new sort of semantic blocking method that improves the number of hits and reduces the amount of review effort.

The use of evolutionary algorithms has been proposed as a powerful random search strategy to solve the join order problem. Specifically, genetic programming used in query optimization has been proposed as an alternative to the limitations of dynamic programming with large join queries. However, very little is known about the impact and behavior of the genetic operations used in this type of algorithms. In this paper, we present an analysis that helps us to understand the effect of these operations during the optimization execution. Specifically, we study five different aspects: the age of the members in the population in terms of generations, the number of query execution plans (QEP) discarded without producing new offsprings, the average QEP life time in generations, the efficiency of the genetic operations and the evolution of the best cost. All in all, our analysis allows us to understand the impact of crossovers compared to mutation operations and the dynamically changing effects of these operations.

The use of clusters of symmetric multiprocessor (SMP) configurations in database processing has become a key factor in allowing greater scalability. It has also posed many challenges in the implementation of one of the most costly operations within relational algebra: the join operation. When massive data is involved, usually the join cannot be performed in-memory and is processed out of core. In this case, performance depends on an effective use of the memory hierarchy, such that I/O and memory contention are minimized. In this paper, we propose a parallel algorithm for out of core join processing that dynamically adapts its behavior to the resources available in the system. We evaluate and compare our proposal against other parallel approaches in a real SMP cluster in a major commercial database, the IBM/spl reg/ DB2P Universal Database7 product (DB2 UDB). Results show that our proposal outperforms previous work significantly.