In systems of systems (SoSs), independent constituent systems operated by different organizations cooperate towards a common goal. This empirical paper is the first to explore what these organizations can learn from high-reliability organization (HRO) theory when collaborating on software performance testing, a part of software performance engineering (SPE). Our case study is based on the Norwegian tax report SoS. Our overall research objective is to understand the organizational factors of SoS software performance testing. To understand these organizational factors , we also analyze the technical factors of SoS software performance. Three months before the opening of the tax return system, we observed meetings where three independent organizations coordinated their performance testing. We also attended conference meetings during, and retrospectives after, the opening. These observations were transcribed and deductively coded for the five HRO principles. Further, we analyzed measurement logs from the organizations participating in the tax return system. The technical root cause of a 90-minute outage in the tax report SoS was a load balancer primary memory shortage caused by too many unserved users. However, the organizational root cause is more interesting: incompatible worldviews reflecting an inadequate holistic SoS management. HRO theory can benefit organizations preparing for workload peaks in mission-critical SoSs. Using the five HRO principles, the constituent organizations can develop a collective mind, bridging incompatible worldviews in SoS software performance testing.

Abstract Context and Motivation: In 2016, the European Union introduced ‘innovation partnerships’ to facilitate innovative development of the EU through public procurement. Requirements engineering is one of the main challenges in the public procurement of innovative products. Nevertheless, there is little empirical research on public procurement, particularly managing requirements in the pre-tender dialogue phase between potential suppliers and problem owners. Question/Problem: This paper investigates the market dialogue phase of an innovation partnership project in Norway. We aim to understand critical factors of the dialogue phase that clarify and focus needs and requirements. This leads to the research question: How can we clarify and focus needs and requirements for a new solution in the market dialogue phase? Principal Ideas/Results: We have conducted a case study at a major Norwegian hospital. The objective of this innovation partnership is to make the emergency room in a Norwegian hospital more efficient. The case study illustrates how requirements have been developed by the joint effort of the procurement team, the active engagement of potential suppliers, and the learning and mutual trust between them. By discussing the vision and getting feedback on opportunities and limitations in existing and projected technologies, the procurement team has refined their ambition and focused on the core of the innovation. Contribution: This paper contributes to the literature on requirement engineering in public procurement by describing how requirements are focused during the dialogue phase of an innovation partnership facilitated by a cross-functional procurement team with sufficient competencies, resources, and trust.

Eliciting scalability requirements during agile software development is complicated and poorly described in previous research. This article presents a lightweight artifact for eliciting scalability requirements during agile software development: the ScrumScale model. The ScrumScale model is a simple spreadsheet. The scalability concepts underlying the ScrumScale model are clarified in this design science research, which also utilizes coordination theory. This paper describes the open banking case study, in which a legacy banking system becomes open. This challenges the scalability of this legacy system. The first step in understanding this challenge is to elicit the new scalability requirements. In the open banking case study, key stakeholders from TietoEVRY spent 55 h eliciting the scalability requirements of TietoEVRY’s open banking project. According to TietoEVRY, the ScrumScale model provided a systematic way of producing scalability requirements. For TietoEVRY, the scalability concepts behind the ScrumScale model also offered significant advantages in dialogs with other stakeholders.

Architectural technical debt can be generated by changes in the business and the environment of an organization. In this paper, we emphasize the change in scalability requirements due to new regulations. Scalability is the ability of a system to handle an increased workload. For complex systems that are abruptly exposed via open interfaces and hence a greater workload, the scalability requirements may quickly increase, leading to technical debt. We term this scalability debt. This paper describes scalability triage, a light-weight, novel technique for identifying scalability threats as a form of technical debt. We illustrate this technique with an open banking case from a large software organization. Open banking is partly caused by the new European PSD2 regulative that enforce banks to open interfaces to unknown third-party actors. Banking systems are well-established, mature systems. However, with the advent of open banking and PSD2, the workload may quickly rocket. This leads to tougher scalability requirements and accumulated architectural debt, despite previously sound architectural decisions. Using scalability triage, such risks may be identified fast. It will then be possible to prevent this form of technical debt with timely reengineering.

Abstract Scalability engineering is currently not well integrated into agile development techniques. This paper extends agile development techniques so that scalability can be handled in an incremental and iterative development process. By scalability we mean the ability of a system to handle increasing workload. We propose the ScrumScale Method which includes scalability engineering in Scrum. This extension should also be applicable to other agile techniques. For scalability testing, we indicate how quality thresholds should be scaled up or down according to the degree of completeness of the product, test hardware, test software, test data and test workload. Using action research, we have conducted three pilots in three Norwegian software organizations. These three pilots have different architectures and operate in different markets yet have in common scalability challenges.

Many software organisations struggle to provide appropriate levels of scalability in their software systems. Agile development rests on pragmatic and value-centred approaches to requirement capture that allows customers and vendors to interact in the process of producing the software system that best mets the real needs of the customers. In collaboration with Norwegian software organisations we have observed that setting scalability requirements is hard. Organisations struggle because they lack a conceptually sound language for expressing scalability requirements. To improve current practice, we propose a light-weight and flexible approach to specifying scalability requirements. Flexibility ensures that a more extensive characterisation can be used if higher precision is required, and more information becomes available.

Scalability and elasticity are key capabilities to tackle the variable workload of an application. Cloud elasticity offers opportunities to manage dynamically the underlying resources of an application and improve its scalability. However, managing scalability of cloud-based systems might lead to a management overhead. Self-adaptive systems are a well-known approach to tame this complexity. In this position paper, we propose an approach for the continuous design and management of scalability in multi-cloud systems. Our approach is based on a three-layer architecture and relies on two existing frameworks, namely ScaleDL and CloudML

This work-in-progress paper introduces the EU FP7 STREP CloudScale. The contribution of this paper is an overall description of CloudScale's engineering approach for the design and evolution of scalable cloud applications and services. An Electronic Health Record (EHR) system serves as a motivation scenario. The overall CloudScale method describes how CloudScale will identify and gradually solve scalability problems in this existing applications. CloudScale will also enable the modelling of design alternatives and the analysis of their effect on scalability and cost. Best practices for scalability will further guide the design process. The CloudScale method is supported by three integrated tools and a scalability description modelling language. CloudScale will be validated by two case studies.

A method for scalability testing of the Microsoft Lync 2010 communication system is presented, exploring the relation between system size and system load. The method can be used for optimal provisioning, balancing user Quality of Experience (QoE) with equipment volume and energy consumption. Observing a standard edition of Lync, on a virtualised platform using the VMware hypervisor, the method indicated linear scalability. QoE was mainly limited by the Mean Opinion Score (MOS). This MOS limit corresponded to a Lync front end server utilisation of about 60%.

Many software engineering courses include all-term projects to convey principles relating to large-scale multi-person development. But even such projects will easily be too small and simple, unless a sufficient amount of study time is allocated to them. This time may be hard to find, especially in strictly programmed profession studies where a lot of general theory courses have to be taken. This paper reports on the experiences from a software engineering project where the solution to the above problem has been to have several courses share one project. This had some advantages. First of all, it allows time for a bigger and more complex project with reasonable sacrifices of "own time" in each of the participating courses. Equally important, it is possible to show connections between the courses. In spite of these advantages, there have also been problems with the project, still leaving room for improvement.