• Reviews runtime composition in the dynamic System of Systems (SoS) • Identifies key challenges: modeling, orchestration, resilience, heterogeneity • Synthesizes seven solution strategies from recent SoS literature • Maps tools and evaluation methods used in runtime SoS research • Reveals gaps in integration, benchmarking, and socio-technical alignment Modern Systems of Systems (SoSs) increasingly operate in dynamic environments (e.g., smart cities, autonomous vehicles) where runtime composition —the on-the-fly discovery, integration, and coordination of constituent systems (CSs)—is crucial for adaptability. Despite growing interest, the literature lacks a cohesive synthesis of runtime composition in dynamic SoSs. This study synthesizes research on runtime composition in dynamic SoSs and identifies core challenges, solution strategies, supporting tools, and evaluation methods. We conducted a Systematic Literature Review (SLR), screening 1,774 studies published between 2019 and 2024 and selecting 80 primary studies for thematic analysis (TA). Challenges fall into four categories: modeling and analysis, resilient operations, system orchestration, and heterogeneity of CSs. Solutions span seven areas: co-simulation and digital twins, semantic ontologies, integration frameworks, adaptive architectures, middleware, formal methods, and AI-driven resilience. Service-oriented frameworks for composition and integration dominate tooling, while simulation platforms support evaluation. Interoperability across tools, limited cross-toolchain workflows, and the absence of standardized benchmarks remain key gaps. Evaluation approaches include simulation-based, implementation-driven, and human-centered studies, which have been applied in domains such as smart cities, healthcare, defense, and industrial automation. The synthesis reveals tensions, including autonomy versus coordination, the modeling-reality gap, and socio-technical integration. It calls for standardized evaluation metrics, scalable decentralized architectures, and cross-domain frameworks. The analysis aims to guide researchers and practitioners in developing and implementing dynamically composable SoSs.

The recent advancements of Small Language Models (SLMs) have opened new possibilities for efficient code generation. SLMs offer lightweight and cost-effective alternatives to Large Language Models (LLMs), making them attractive for use in resource-constrained environments. However, empirical understanding of SLMs, particularly their capabilities, limitations, and performance trade-offs in code generation remains limited. This study presents a comprehensive empirical evaluation of 20 open-source SLMs ranging from 0.4B to 10B parameters on five diverse code-related benchmarks (HumanEval, MBPP, Mercury, HumanEvalPack, and CodeXGLUE). The models are assessed along three dimensions: i) functional correctness of generated code, ii) computational efficiency and iii) performance across multiple programming languages. The findings of this study reveal that several compact SLMs achieve competitive results while maintaining a balance between performance and efficiency, making them viable for deployment in resource-constrained environments. However, achieving further improvements in accuracy requires switching to larger models. These models generally outperform their smaller counterparts, but they require much more computational power. We observe that for 10% performance improvements, models can require nearly a 4x increase in VRAM consumption, highlighting a trade-off between effectiveness and scalability. Besides, the multilingual performance analysis reveals that SLMs tend to perform better in languages such as Python, Java, and PHP, while exhibiting relatively weaker performance in Go, C++, and Ruby. However, statistical analysis suggests these differences are not significant, indicating a generalizability of SLMs across programming languages. Based on the findings, this work provides insights into the design and selection of SLMs for real-world code generation tasks.

Background: Large language models (LLM) suffer from various forms of biases due to the biased datasets used to train the models. At the same time, human cognitive biases have an equal propensity to express themselves when using LLMs for software engineering tasks. Software testing is a critical phase of the software development life cycle. Confirmation bias is reported to have deteriorated software testing by designing more specification-consistent test cases compared to specificationinconsistent test cases. However, there is a lack of debiasing (mitigation) strategies in this regard. Aims: In this paper, first, we investigate whether the LLM model suffers from confirmation bias while performing software testing tasks. Second, we propose a vision of debasing confirmation bias in software testing via LLM. Method: We conducted an empirical study to detect confirmation bias by an LLM (ChatGPT4.0) in the design of functional test cases. Based on empirical findings, we used the analytical paradigm to design a multi-agent system. Results: We present a vision for debiasing confirmation bias in functional software testing by leveraging LLMs via a multi-agent approach. Conclusions: The proposed vision may improve the performance of LLMs in terms of reduced confirmation bias and serve as a debiasing technique for functional software testing.

There is an increasing need for practitioners to address legislative and ethical issues in both the development and deployment of data-driven applications with AI/ML due to growing concerns and regulations, such as GDPR and the EU AI Act. Thus, the field needs a systematic framework for assessing risks and helping to stay compliant with regulations in designing and deploying software systems. Clear and concise descriptions of risks associated with each model and data source are needed to guide the design without acquiring deep knowledge of the regulations. In this paper, we propose a reference architecture for an ethical orchestration system that manages distributed AI/ML applications on the cloud–edge continuum and present a proof-of-concept implementation of the main ideas of the architecture. Our starting point is the methods already in use in the industry, such as model cards, and we extend the idea of model cards to data source cards and software component cards, which provide practitioners and the automated system with relevant information in actionable form. With the metadata card based orchestration system and information about the risk levels of the target infrastructure, the users can create deployments of distributed AI/ML systems that fulfill the regulatory and other requirements.

Containerization in multi-cloud environments has received significant attention in recent years both from academic research and industrial development perspectives. However, there exists no effort to systematically investigate the state of research on this topic. The aim of this research is to systematically identify and categorize the multiple aspects of containerization in multi-cloud environment. We conducted the Systematic Mapping Study (SMS) on the literature published between January 2013 and July 2024. One hundred twenty one studies were selected and the key results are: (1) Four leading themes on containerization in multi-cloud environment are identified: ‘Scalability and High Availability’, ‘Performance and Optimization’, ‘Security and Privacy’, and ‘Multi-Cloud Container Monitoring and Adaptation’. (2) Ninety-eight patterns and strategies for containerization in multi-cloud environment were classified across 10 subcategories and 4 categories. (3) Ten quality attributes considered were identified with 47 associated tactics. (4) Four catalogs consisting of challenges and solutions related to security, automation, deployment, and monitoring were introduced. The results of this SMS will assist researchers and practitioners in pursuing further studies on containerization in multi-cloud environment and developing specialized solutions for containerization applications in multi-cloud environment.

WebAssembly (Wasm) is a binary instruction format designed for secure and efficient execution within sandboxed environments - predominantly web apps and browsers - to facilitate performance, security, and flexibility of web programming languages. In recent years, Wasm has gained significant attention from the academic research community and industrial development projects to engineer high-performance web applications. Despite the offered benefits, developers encounter a multitude of issues rooted in Wasm (e.g., faults, errors, failures) and are often unaware of their root causes that impact the development of web applications. To this end, we conducted an empirical study that mines and documents practitioners’ knowledge expressed as 385 issues from 12 open-source Wasm projects deployed on GitHub and 354 question-answer posts via Stack Overflow. Overall, we identified 120 types of issues, which were categorized into 19 subcategories and 9 categories to create a taxonomical classification of issues encountered in Wasm-based applications. Furthermore, root cause analysis of the issues helped us identify 278 types of causes, which have been categorized into 29 subcategories and 10 categories as a taxonomy of causes. Our study led to first-of-its-kind taxonomies of the issues faced by developers and their underlying causes in Wasm-based applications. The issue-cause taxonomies - identified from GitHub and SO, offering empirically derived guidelines - can guide researchers and practitioners to design, develop, and refactor Wasm-based applications.

Artificial Intelligence has demonstrated its significance in software engineering through notable improvements in productivity, accuracy, collaboration, and learning outcomes.This study examines the impact of generative AI tools, specifically ChatGPT, on the software development experiences of undergraduate students. Over a three-month project with seven students, ChatGPT was used as a support tool. The research focused on assessing ChatGPT’s effectiveness, benefits, limitations, and its influence on learning. Results showed that ChatGPT significantly addresses skill gaps in software development education, enhancing efficiency, accuracy, and collaboration. It also improved participants’ fundamental understanding and soft skills. The study highlights the importance of incorporating AI tools like ChatGPT in education to bridge skill gaps and increase productivity, but stresses the need for a balanced approach to technology use. Future research should focus on optimizing ChatGPT’s appli cation in various development contexts to maximize learning and address specific challenges.

The ever-growing complexity and dynamic nature of modern System of Systems (SoS) necessitate efficient communication mechanisms to ensure interoperability and collaborative functioning among constituent systems (CS), referred to as holons in the holonic architecture of SoS. This paper proposes a novel approach to enhance humand-to-holon and holon-to-holon communication within the holonic architecture through the integration of Natural Language Processing (NLP) techniques. Our proposed framework utilizes advancements in NLP, specifically Large Language Models (LLMs), enabling holons to understand and act on natural language instructions. This enables more intuitive holon-to-holon and human-to-holon interactions, leading to better coordination among diverse systems. The framework’s practical application is demonstrated through an Unmanned Vehicle Fleet (UVF) case study, showcasing its potential in enhancing communication and coordination in complex SoS. Additionally, we propose evaluat ion strategies to assess the efficiency and effectiveness of this framework, and identify areas for improvement. This work sets the stage for future exploration and prototype implementation, paving the way for further advancements in SoS communication and collaboration.

Architecting software-intensive systems can be a complex process. It deals with the daunting tasks of unifying stakeholders’ perspectives, designers’ intellect, tool-based automation, pattern-driven reuse, and so on, to sketch a blueprint that guides software implementation and evaluation. Despite its benefits, architecture-centric software engineering (ACSE) suffers from a multitude of challenges. ACSE challenges could stem from a lack of standardized processes, socio-technical limitations, and scarcity of human expertise etc. that can impede the development of existing and emergent classes of software. Software Development Bots (DevBots) trained on large language models can help synergise architects’ knowledge with artificially intelligent decision support to enable rapid architecting in a human-bot collaborative ACSE. An emerging solution to enable this collaboration is ChatGPT, a disruptive technology not primarily introduced for software engineering, but is capable of articulating and refining architectural artifacts based on natural language processing. We detail a case study that involves collaboration between a novice software architect and ChatGPT to architect a service-based software. Future research focuses on harnessing empirical evidence about architects’ productivity and explores socio-technical aspects of architecting with ChatGPT to tackle challenges of ACSE.

With the advances in machine learning, there is a growing interest in AI-enabled tools for autocompleting source code.GitHub Copilot, also referred to as the "AI Pair Programmer", has been trained on billions of lines of open source GitHub code, and is one of such tools that has been increasingly used since its launch in June 2021.However, little effort has been devoted to understanding the practices and challenges of using Copilot in programming with auto-completed source code.To this end, we conducted an empirical study by collecting and analyzing the data from Stack Overflow (SO) and GitHub Discussions.More specifically, we searched and manually collected 169 SO posts and 655 GitHub discussions related to the usage of Copilot.We identified the programming languages, IDEs, technologies used with Copilot, functions implemented, benefits, limitations, and challenges when using Copilot.The results show that when practitioners use Copilot: (1) The major programming languages used with Copilot are JavaScript and Python, (2) the main IDE used with Copilot is Visual Studio Code, (3) the most common used technology with Copilot is Node.js,(4) the leading function implemented by Copilot is data processing, (5) the significant benefit of using Copilot is useful code generation, and (6) the main limitation encountered by practitioners when using Copilot is difficulty of integration.Our results suggest that using Copilot is like a double-edged sword, which requires developers to carefully consider various aspects when deciding whether or not to use it.Our study provides empirically grounded foundations and basis for future research on the role of Copilot as an AI pair programmer in software development.

With the advances in machine learning, there is a growing interest in AI-enabled tools for autocompleting source code. GitHub Copilot, also referred to as the “AI Pair Programmer”, has been trained on billions of lines of open source GitHub code, and is one of such tools that has been increasingly used since its launch in June 2021. However, little effort has been devoted to understanding the practices, challenges, and expected features of using Copilot in programming for auto-completed source code from the point of view of practitioners. To this end, we conducted an empirical study by collecting and analyzing the data from Stack Overflow (SO) and GitHub Discussions. More specifically, we searched and manually collected 303 SO posts and 927 GitHub discussions related to the usage of Copilot. We identified the programming languages, Integrated Development Environments (IDEs), technologies used with Copilot, functions implemented, benefits, limitations, and challenges when using Copilot. The results show that when practitioners use Copilot: (1) The major programming languages used with Copilot are JavaScript and Python, (2) the main IDE used with Copilot is Visual Studio Code, (3) the most common used technology with Copilot is Node.js, (4) the leading function implemented by Copilot is data processing, (5) the main purpose of users using Copilot is to help generate code, (6) the significant benefit of using Copilot is useful code generation, (7) the main limitation encountered by practitioners when using Copilot is difficulty of integration, and (8) the most common expected feature is that Copilot can be integrated with more IDEs. Our results suggest that using Copilot is like a double-edged sword, which requires developers to carefully consider various aspects when deciding whether or not to use it. Our study provides empirically grounded foundations that could inform software developers and practitioners, as well as provide a basis for future investigations on the role of Copilot as an AI pair programmer in software development.

Quantum computing systems rely on the principles of quantum mechanics to perform a multitude of computationally challenging tasks more efficiently than their classical counterparts. The architecture of software-intensive systems can empower architects who can leverage architecture-centric processes, practices, description languages to model, develop, and evolve quantum computing software (quantum software for short) at higher abstraction levels. We conducted a Systematic Literature Review (SLR) to investigate (i) architectural process, (ii) modelling notations, (iii) architecture design patterns, (iv) tool support, and (iv) challenging factors for quantum software architecture. Results of the SLR indicate that quantum software represents a new genre of software-intensive systems; however, existing processes and notations can be tailored to derive the architecting activities and develop modelling languages for quantum software. Quantum bits (Qubits) mapped to Quantum gates (Qugates) can be represented as architectural components and connectors that implement quantum software. Tool-chains can incorporate reusable knowledge and human roles (e.g., quantum domain engineers, quantum code developers) to automate and customise the architectural process. Results of this SLR can facilitate researchers and practitioners to develop new hypotheses to be tested, derive reference architectures, and leverage architecture-centric principles and practices to engineer emerging and next generations of quantum software.

Artificial intelligence (AI) solutions and technologies are being increasingly adopted in smart systems contexts; however, such technologies are concerned with ethical uncertainties. Various guidelines, principles, and regulatory frameworks are designed to ensure that AI technologies adhere to ethical well-being. However, the implications of AI ethics principles and guidelines are still being debated. To further explore the significance of AI ethics principles and relevant challenges, we conducted a survey of 99 randomly selected representative AI practitioners and lawmakers (e.g., AI engineers and lawyers) from 20 countries across five continents. To the best of our knowledge, this is the first empirical study that unveils the perceptions of two different types of population (AI practitioners and lawmakers) and the study findings confirm that transparency, accountability, and privacy are the most critical AI ethics principles. On the other hand, lack of ethical knowledge, no legal frameworks, and lacking monitoring bodies are found to be the most common AI ethics challenges. The impact analysis of the challenges across principles reveals that conflict in practice is a highly severe challenge. Moreover, the perceptions of practitioners and lawmakers are statistically correlated with significant differences for particular principles (e.g. fairness and freedom) and challenges (e.g. lacking monitoring bodies and machine distortion). Our findings stimulate further research, particularly empowering existing capability maturity models to support ethics-aware AI systems’ development and quality assessment.

Ethics in AI becomes a global topic of interest for both policymakers and academic researchers. In the last few years, various research organizations, lawyers, think tankers, and regulatory bodies get involved in developing AI ethics guidelines and principles. However, there is still debate about the implications of these principles. We conducted a systematic literature review (SLR) study to investigate the agreement on the significance of AI principles and identify the challenging factors that could negatively impact the adoption of AI ethics principles. The results reveal that the global convergence set consists of 22 ethical principles and 15 challenges. Transparency, privacy, accountability and fairness are identified as the most common AI ethics principles. Similarly, lack of ethical knowledge and vague principles are reported as the significant challenges for considering ethics in AI. The findings of this study are the preliminary inputs for proposing a maturity model that assesses the ethical capabilities of AI systems and provides best practices for further improvements.

Due to its enormous benefits, the research and industry communities have shown an increasing interest in the Microservices Architecture (MSA) style over the last few years. Despite this, there is a limited evidence-based and thorough understanding of the types of issues (e.g., faults, errors, failures, mistakes) faced by microservices system developers and causes that trigger the issues. Such evidence-based understanding of issues and causes is vital for long-term, impactful, and quality research and practice in the MSA style. To that end, we conducted an empirical study on 1,345 issue discussions extracted from five open source microservices systems hosted on GitHub. Our analysis led to the first of its kind taxonomy of the types of issues in open source microservices systems, informing that the problems originating from Technical debt (321, 23.86%), Build (145, 10.78%), Security (137, 10.18%), and Service execution and communication (119, 8.84%) are prominent. We identified that “General programming errors”, “Poor security management”, “Invalid configuration and communication”, and “Legacy versions, compatibility and dependency” are the predominant causes for the leading four issue categories. Study results streamline a taxonomy of issues, their mapping with underlying causes, and present empirical findings that could facilitate research and development on emerging and next-generation microservices systems.

Microservices is an architectural style that provides several benefits to develop applications as small, independent, and modular services. Building Microservices Architecture (MSA)-based applications is immensely supported by using software testing fundamentals. With the increasing interest in the development of MSA-based applications, it is important to systematically identify, analyze, and classify the publication trends, research themes, approaches, tools, and challenges in the context of testing MSA-based applications. The search yielded 2,481 articles, and 33 articles were finally selected as the primary studies with snowballing. The key findings are that (i) 5 research themes characterize testing approaches in MSA-based applications; (ii) integration and unit testing are the most popular testing approaches; and (iii) addressing the challenges in automated and inter-communication testing is gaining the interest of the community. Additionally, it emerges that there is a lack of dedicated tools to support testing for MSA-based applications, and the reasons and solutions behind the challenges in testing MSA-based applications need to be further explored.

This proposal is design to address the proposed research work on agile software development and architecture co-existence. The objective of this research is to answer how architecting activities emerge and evolve with agile software development in industry. The architecting activities are architectural analysis (AA), architectural synthesis (AS), architectural evaluation (AE), architectural implementation (AI), architectural maintenance and evolution (AME), architectural recovery (AR), architectural description (ADp), architectural understanding (AU), architectural impact analysis (AIA), architectural reuse (ARu) and architectural refactoring (ARf). This research objective could achieve by using multiple research methods. We are planning to use comprehensively report the pure ‘state- of- practice’ for architecting activities in ASD from industry and practitioners point of views. Therefore, we decided to use the case studies, survey and semi structure interview as research methods. The result of this research work can provide the baseline information for architecture evolution frameworks for agile software development, challenges and solutions in ASD for SA activities, expected evolvable dimensions of the software system, methods that may help for minimizing the architectural and agile co-existence issues and architectural technical debt in agile software development.