ABSTRACT The video gaming industry has evolved into an industry that drives high revenue and tangentially faces some of the toughest problems in the software engineering domain. With more games being released with either remote multiplayer components or cloud streaming foundations, maintaining an engaging player experience that lives up to a high‐quality bar is a problem that many game studios must overcome. Traditionally, studios have relied on brute‐force manual testing of their software to find impacting bugs. However, as studios and publishers focus on shortening the delivery timeline of their products to consumers, we have seen a rise in the use of testing tools, automation and enhanced processes that focus on achieving a high level of quality. In this systematic literature review, we provide a review of the tools and processes and discuss how these tools enhance the quality experience or help decrease the cost of testing while allowing for faster releases. We conclude by observing some potential areas of opportunity, such as suggesting an approach to help validate the severity of bugs found through community validation and providing a mechanism to create a way to track bugs across heterogeneous devices.

This innovative practice full paper delves into the transformative role of Learning Assistants (LAs) in Computer Science education, focusing on enhancing student engagement and improving learning outcomes. The LA model, which aligns with Vygotsky's Social Constructivist Learning Theory, fosters an environment of student-centered learning and social interaction. In a pilot study conducted in Spring 2024 at a public university, the LA program is implemented in two computer science courses. A quasi-experimental design has been used to evaluate the impact of LA-facilitated team activities on student learning outcomes. The study compares a control group receiving traditional instruction with an experimental group participating in LA-facilitated team activities. The experimental group engaged in weekly team-based activities, guided by LAs and faculty, to reinforce class concepts and promote collaboration among team members. Student engagement and learning have been evaluated using feedback from students and LAs collected through Discussion Boards (DBs). Preliminary findings suggest that LA-facilitated in-class activities promote active learning and enhance problem-solving skills. LAs provide valuable support and guidance to students, particularly those struggling to understand complex concepts. The study tested a working model of AIELA, an innovative AI-powered chatbot that assists human LAs in supporting students through knowledge-reinforcing questions and multimodal data analysis, powered by OpenAI API's gpt-4-turbo model. This research is a step towards embracing the challenges of modern CS education, inspiring further innovation in this critical field. The findings will benefit educators seeking innovative strategies to enrich student engagement and learning in engineering and computing disciplines.

Model-driven software engineering (MDSE) has emerged as a popular and commonly used method for designing software systems in which models are the primary development artifact over the last decade. MDSE has resulted in the trend toward further automating the software process. However, the generation of test cases from user requirements still lags in reaching the required level of automation. Given that most user requirements are written in natural language, the recent advances in natural language processing (NLP) provide an opportunity to further automate the test generation process.In this paper, we exploit the advances in MDSE and NLP to generate abstract test cases from user requirements written in structured natural language and the respective data model. We accomplish this by creating meta-models for user requirements and abstract test cases and defining the appropriate transformation rules. To support this transformation, helper methods are defined to extract the relevant information from user requirements related to testing. To show the feasibility of the approach, we developed a prototype and conducted a case study with use cases and test cases from a Payroll Management System.

There continues to be an increase in enrollments in various computing programs at academic institutions due to many job opportunities available in the information, communication, and technology sectors. This enrollment surge has presented several challenges in many Computer Science (CS), Information Technology (IT), and Software Engineering (SE) programs at universities and colleges. One such challenge is that many instructors in CS/IT/SE programs continue to use learning approaches that are not learner centered and therefore are not adequately preparing students to be proficient in the ever-changing computing industry. To mitigate this challenge, instructors need to use evidence-based pedagogical approaches, e.g., active learning, to improve student learning and engagement in the classroom and equip students with the skills necessary to be lifelong learners. This article presents an approach that combines learning and engagement strategies (LESs) in learning environments using different teaching modalities to improve student learning and engagement. We describe how LESs are integrated into face-to-face (F2F) and online class activities. The LESs currently used are collaborative learning , gamification , problem-based learning , and social interaction . We describe an approach used to quantify each LES used during class activities based on a set of characteristics for LESs and the traditional lecture-style pedagogical approaches. To demonstrate the impact of using LESs in F2F class activities, we report on a study conducted over seven semesters in a software testing class at a large urban minority serving institution. The study uses a posttest-only study design, the scores of two midterm exams, and approximate class times dedicated to each LES and traditional lecture style to quantify their usage in a face-to-face software testing class. The study results showed that increasing the time dedicated to collaborative learning, gamification, and social interaction and decreasing the traditional lecture-style approach resulted in a statistically significant improvement in student learning, as reflected in the exam scores.

The behavior, attributes and properties of a software system is represented in a set of requirements that are written in structured natural language and are usually ambiguous. In large development projects, different modeling techniques are used to create and manage these requirements which aid in the analysis of the problem domain. Requirements are later used in the development process to create test cases, which is still mainly a manual process. To automate this process, we plan to use several of the techniques used in model-driven software development and Natural Language Processing(NLP). The approach under consideration is to use a model-to-model transformation to convert requirements into test cases with the support of Stanford CoreNLP techniques. Key to this transformation process is the use of meta-modeling for requirements and test cases. In this paper we focus on creating a comprehensive meta-model for requirements that can represent both use cases and user stories and performing preliminary analysis of the requirements using NLP. In later work we will develop a set of transformation rules to convert requirements into partial test cases. To show the feasibility of our approach we develop a prototype that can accept a cross-section of requirements written as both use cases and user stories.

Recently there has been an increase in the number of software applications using the microservices architectural pattern. This is due to the benefits derived over the more traditional N-tier patterns that use monolithic designs for each of the tiers. The value of using the microservices architectural pattern, particularly in the cloud, has been pioneered by companies such as Netflix and Google. These companies have created protocols and tools to support the development of cloud-based applications. However, the testing of microservices applications continues to be challenging due to the added complexity of network communication between the collaborating services. In this paper we provide a comparison of several open-source tools used to support the testing of microservices, describe polyglot Ridesharing microservices reference prototype, and present the results of a study that measures the overhead to test the Ridesharing application. We also describe our experiences in configuring and running the testing tools on the prototype.

In recent years there has been a focus on the automatic generation of test cases using white box testing techniques, however the same cannot be said for the generation of test cases at the system-level from natural language system requirements. Some of the white-box techniques include: the use of constraint solvers for the automatic generation of test inputs at the white box level; the use of control flow graphs generated from code; and the use of path generation and symbolic execution to generate test inputs and test for path feasibility. Techniques such as boundary value analysis (BVA) may also be used for generating stronger test suites. However, for black box testing we rely on specifications or implicit requirements and spend considerable time and effort designing and executing test cases. This paper presents an approach that leverages natural language processing and machine learning techniques to capture black box system behavior in the form of constraints. Constraint solvers are then used to generate test cases using BVA and equivalence class partitioning. We also conduct a proof of concept that applies this approach to a simplified task management application and an enterprise job recruiting application.

Having to compete harder than ever for students' attention against cool social media sites, mobile apps, video games, messaging, etc., instructors are struggling to find new ways to motivate and engage learners. Gamification offers a promising framework for educational interventions that can increase students' motivation and engagement. This BOF will bring together instructors who have already explored the use of various game driven strategies and elements to increase the motivation of their students and also those who are looking for promising interventions to do that. The facilitators conducted a similar BOF at SIGCSE 2018, which was well attended, well received and sparked an interesting and lively discussion. This BOF will continue the conversation on what might be useful ways of incorporating game elements in Computer Science courses, what tools might be needed to support such efforts, how can the student experience of gamified learning be personalized, etc. Two NSF sponsored projects related to the use of gamification in STEM courses will seed the discussion. We hope to collectively contribute to the ongoing important discussion on successful strategies for applying of gamification in Computer Science education.

The low performance and drop-outs in Computer Science classes are frequently attributed to lack of engagement and motivation. Meanwhile, gamification - the application of game design principles and game mechanics to a non-game context -- increasingly attracts the interest of educators due to its potential to foster motivation and behavioral changes in learning contexts. Course gamification is about using game elements, such as instant feedback, freedom to fail, freedom of choice, achievements, leveling, progress mechanics, badges, and leaderboards, as interventions in the learning process. This BOF will provide a platform for a discussion of when and how course gamification strategies can be beneficial for improving student motivation, engagement and achievements in Computer Science classes. Questions to be discussed include: What game elements could be useful in gamifying a computer science course? How can an instructor incorporate game elements in the design of their courses? What support the instructor may need for gamifying their courses? Could some game elements have potential drawbacks on student motivation and performance? The discussion leaders and attendees with experience in applying gamification in their courses will share examples of successes or failures, challenges, problems, tips, and techniques for gamifying learning with each other and those new to the use of gamification. Two NSF sponsored projects related to the use of gamification in STEM courses will seed the discussion. As an outcome of the discussion, we hope to collectively identify some lessons learned and challenges to be overcome for a successful application of gamification in Computer Science education.

Middleware was introduced to facilitate the development of sophisticated applications based on a uniform methodology and industry standards. However, early research and practice suggested that no one-size-fits-all approach was suitable for all application domains and scenarios. This gave rise to industry initiatives to standardize domain-specific middleware services and profiles, as well as research efforts on configurable, reflective, and adaptive middleware. The industry's approach led to easy deployment, although with a level of flexibility limited by the extent of existing profiles. The approach of the research community, on the other hand, enabled high flexibility, allowing any middleware configuration to be defined. Nevertheless, creating sound configurations using this approach is a challenging task, limiting the target audience to expert engineers. As a consequence, both initiatives do not scale with the current proliferation of specialized application domains. In this paper, we target this problem with an approach that leverages model-driven engineering for the construction of domain-specific middleware platforms. A set of high-level, yet expressive, building blocks is defined in the form of a metamodel, which is used to create models that specify the desired middleware configuration. We argue that this approach enables the rapid development of middleware platforms to match the proliferation of application domains, at the same time as it does not require per-application middleware construction or even highly skilled middleware engineers. We present the current state of our research and discuss research directions to fully realize the approach.

Software continues to affect a major part of our daily lives, including the way we use our phones, home appliances, medical devices, and cars. The pervasiveness of software has led to a growing demand for software developers over the next decade. To ensure the high quality of software developed in industry, students being trained in software engineering also need to be trained on how to use testing techniques and supporting tools effectively at all levels of development. In this article, we investigate how testing tools are used in the software project of an undergraduate testing course. We also investigate how a cyberlearning environment—the Web-Based Repository of Software Testing Tutorials (WReSTT)—is used to supplement the learning materials presented in class, particularly the tutorials on different software testing tools. The results of a study spanning three semesters of the undergraduate course suggest that (1) the use of code coverage tools motivates students to improve their test suites; (2) the number of bugs found when using coverage tools slightly increased, which is similar to the results found in the research literature; and (3) students find WReSTT to be a useful resource for learning about software testing techniques and the use of code coverage tools.

No abstract available.

As software becomes more ubiquitous and complex, the cost of software bugs continues to grow at a staggering rate. To remedy this situation, there needs to be major improvement in the knowledge and application of software validation techniques. Although there are several software validation techniques, software testing continues to be one of the most widely used in industry. The high demand for software engineers in the next decade has resulted in more software engineering (SE) courses being offered in academic institutions. However, due to the number of topics to be covered in SE courses, little or no attention is given to software testing, resulting in students entering industry with little or no testing experience. We propose a minimally disruptive approach of integrating software testing into SE courses by providing students access to a collaborative learning environment containing learning materials on testing techniques and testing tools. In this article, we describe the learning environment and the studies conducted to measure the benefits accrued by students using the learning environment in the SE courses.

There continues to be a lack of adequate training for students in software testing techniques and tools at most academic institutions. Several educators and researchers have investigated innovative approaches that integrate testing into programming and software engineering (SE) courses with some success. The main problems are getting other educators to adopt their approaches and ensuring students continue to use the techniques they learned in previous courses. In this paper we present a study that evaluates a non-intrusive approach to integrating software testing techniques and tools in SE courses. The study uses a Web-Based Repository of Software Testing Tools (WReSTT) that contains tutorials on software testing concepts and tools. The results of the study show that (1) students who use WReSTT in the classroom can improve their understanding and use of testing techniques and tools, (2) students find WReSTT a useful learning resource, and (3) the collaborative learning environment motivates students to complete assignments.

SUMMARY Advances in communication devices and technologies are dramatically expanding our communication capabilities and enabling a wide range of multimedia communication applications. The current approach to develop communication‐intensive applications results in products that are fragmented, inflexible, and incapable of responding to changing end‐users' communication needs. These limitations have resulted in the need for a new development approach of building communication applications that are driven by end‐users and that support the dynamic nature of communication‐based collaboration. To address this need, the Communication Virtual Machine (CVM) technology has been developed to support rapid specification and automatic realization of user‐centric communication applications based on a domain‐specific modeling approach. The CVM technology consists of a domain‐specific modeling language (DSML), the Communication Modeling Language (CML), that is used to create communication models, and a semantic rich platform, the CVM, that realized the created communication models. In this paper, we report on our experiences of applying a systematic approach to engineering CML and the synthesis of CML models in CVM. Based on a feature model describing the taxonomy of the user‐centric communication domain in a network independent manner, we develop the meta‐model of CML and its different concrete syntaxes. We also present a behavioral specification (dynamic semantics) of CML that enables the dynamic synthesis of user‐centric communication models into an executable form called communication control script. We validated the CML semantics using Kermeta, a meta‐programming environment for engineering DSMLs, and evaluated the practicality of our approach using a CVM prototype and a set of experiments. Copyright © 2011 John Wiley & Sons, Ltd.

SUMMARY One of the characteristics of the increasingly widespread use of object‐oriented libraries and the resulting intensive use of inheritance is the proliferation of dependencies on abstract classes. Since abstract classes cannot be instantiated, they cannot be tested in isolation using standard execution‐based testing strategies. A standard approach to testing abstract classes is to instantiate a concrete descendant class and test the features that are inherited. This paper presents a structured approach that supports the testing of features in abstract classes, paying particular attention to ensuring that the features tested are those defined in the abstract class. Two empirical studies are performed on a suite of large Java programs and the results presented. The first study analyses the role of abstract classes from a testing perspective. The second study investigates the impact of the testing strategy on the programs in this suite to demonstrate its feasibility and to comment on the pragmatics of its use. Copyright © 2010 John Wiley & Sons, Ltd.

Rapid advances in electronic communication devices and technologies have resulted in a shift in the way communication applications are being developed. The emerging development strategies provide end-users with a greater ability to manipulate the underlying communication technologies by providing the appropriate level of abstraction, referred to as user-centric communication. In communication-intensive domains such as telemedicine and disaster management, the user-centric communication strategies still lack the ability to coordinate the various communication services in collaborative processes. In this paper, we present a domain-specific modeling language (DSML), Workflow Communication Modeling Language (WF-CML), that supports the rapid realization of collaborative user-centric communication applications. WF-CML is an extension of CML with communication specific abstractions of workflow concepts. To realize WF-CML models, the dynamic synthesis process in the Communication Virtual Machine (CVM) prototype was extended to coordinate the negotiation and media transfer processes based on events generated during the collaboration. We also present a comparative study to show the advantage of using WF-CML over a general-purpose workflow language and execution environment.

One of the main concerns in the software industry continues to be the development of high quality software. This concern will be exacerbated as software systems become more complex. The training of software developers continues to grow in academia since more institutions are offering software engineering (SE) courses. However, the list of topics that are expected to be covered in this course leaves little or no time for topics that focus on developing quality software, such as software testing and the use of testing tools.

The use of domain-specific modeling languages (DSMLs) results in higher productivity during the development process. This is accomplished by raising the level of abstraction during design and focusing on domain concepts rather than low-level implementation details. Unlike other development paradigms, little work has been done in determining and measuring the claimed benefits of using DSMLs. In this paper, we propose a new approach to determine the effort involved in creating and using DSML models to develop applications and to manage the behavior of applications at runtime. The approach involves a classification of the effort involved, and definition of relevant metrics to measure the effort for each category. A case study is presented that shows how we applied the proposed metrics during the development and execution of an application using three different DSMLs. Categories and Subject Descriptors

Information systems that use the mediation architecture are well suited to accessing heterogeneous data in a distributed environment. These mediation systems utilize a central security mediator that is primarily concerned with securing the internal structure of the mediation system. The current problem with this approach is that clients are unable to have authority and administer the security of their data during the collaboration with the mediation system. This creates a black box scenario from the perspective of the client towards the mediation system, which can have great consequences in transactions that deal with sensitive data within an untrusted environment. In this paper, we present a solution in the form of a decentralized mediator that provides all parties involved in a transaction within the mediation system autonomy over their security that is security model independent and dynamic at runtime. Our approach allows the client to manage the access to data during a transaction with the mediation system.

As researchers and members of the IT industry move towards a vision of computing systems that manage themselves, it is imperative to investigate ways to dynamically validate these systems to avoid the high cost of system failures. Although research continues to advance in many areas of autonomic computing, there is a lack of development in the area of testing these types of systems at runtime. Self-managing features in autonomic systems dynamically invoke changes to the structure and behavior of components that may already be operating in an unpredictable environment; further emphasizing the need for runtime testing. In this paper we propose a framework that dynamically validates changes in autonomic computing systems. Our framework extends the current structure of autonomic computing systems to include self-testing as an implicit characteristic. We validate our framework by creating a prototype of an autonomic container that incorporates the ability to self-test

One of the characteristics of the increasingly widespread use of object-oriented libraries and the resulting intensive use of inheritance is the proliferation of dependencies on abstract classes. Such classes defer the implementation of some features, and are typically used as a specification or design tool. However, since their features are not fully implemented, abstract classes cannot be instantiated, and thus pose challenges for execution-based testing strategies. This paper presents a structured approach that supports the testing of features in abstract classes. Core to the approach is a series of static analysis steps that build a comprehensive view of the inter-class dependencies in the system under test. We then leveraged this information to define a test order for the methods in an abstract class that minimizes the number of stubs required during testing, and clearly identifies the required functionality of these stubs. Our approach is based on a comprehensive taxonomy of object-oriented classes that provides a framework for our analysis. First we describe the algorithms to calculate the inter-class dependencies and the test-order that minimizes stub creation. Then we give an overview of our tool, AbstractTestJ that implements our approach by generating a test order for the methods in an abstract Java class. Finally, we harness this tool to provide an analysis of 12 substantial Java applications that demonstrates both the feasibility of our approach and the importance of this technique.

There continue to be advances in the automation of Web application development, however testing these applications remains mainly a manual process. In this paper we present a methodology to test page flows using traditional test coverage criteria in conjunction with an automated testing tool. The criteria is applied in the context of page flows and transformed into: all pages, all actions, all links, and all forwards. We define a state-based model of the application using information from the page flow and then use this model as the basis for generating a script to be executed by the automated testing tool. This finite state machine (FSM) also models the various combinations of inputs associated with the user interface of the application. During execution of the script, test cases are randomly generated using the FSM along with textual input from a pre-defined data pool. The adequacy of the test coverage based on the criteria for the page flow is determined by analyzing the elements covered during execution of the test script

The convergence of data, voice and multimedia communication over digital networks, coupled with continuous improvement in network capacity and reliability has significantly enriched the ways we communicate. However, the stovepipe approach used to develop today's communication applications and tools results in rigid technology, limited utility, lengthy and costly development cycle, difficulty in integration, and hinders innovation. In this paper, we present a fundamentally different approach, which we call communication virtual machine (CVM) to address these problems. CVM provides a user-centric, model-driven approach for conceiving, synthesizing and delivering communication solutions across application domains. We argue that CVM represents a far more effective paradigm for engineering communication solutions. The concept, architecture, modeling language, prototypical design and implementation of CVM are discussed

The object-oriented (OO) paradigm provides several benefits during analysis and design of large-scale software systems, but scores lower in terms of testability. The low testability score for OO software is due mainly to the composition of OO systems exhibiting the characteristics of abstraction, encapsulation, genericity, inheritance, polymorphism, concurrency and exception handling. To address the difficulty of testing the features of a class, a plethora of implementation-based testing techniques (IBTTs) have been developed. However, no one IBTT has emerged as the preferred technique to test the implementation of a class. In this paper we present a technique that automatically identify those IBTTs that are most suitable for testing a class based on the characteristics of that class. Our approach uses a taxonomy of OO classes that is used to succinctly abstract the characteristics of a class under test (CUT). We have implemented a tool that automates the process of mapping IBTTs to a class. In addition to identifying the IBTTs that would be best suited for testing a class, our tool provides feedback to the tester facilitating the identification of the characteristics of the class that are not suitably tested by any of the IBTTs in the list. We provide results of a study supporting the notion that using more than on IBTT during testing improves test coverage of a CUT.

In this paper, we describe a class abstraction technique (CAT) for Java programs that support the testing process by capturing aspects of software complexity based on the combination of class characteristics. These class characteristics relate to properties of the class features such as concurrency, polymorphism, exception handling, and accessibility as well as relationships between classes. Our taxonomy (CAT) for Java allows us to generate a finite number of possible class groups (taxa). Each class C in a Java program is cataloged into a group that summarizes the dependencies with other types realized through declarations and definitions in C. We also provide a high-level design for a tool to catalog Java classes based on our taxonomy.

In this paper we present the design and implementation of a class ordering system that is driven by a parameterized cost model. The parameters to the model assign weights to the edge types that describe the relationships between the classes in the graphical representation of the program. The nodes in the graph are classes and the edges express relationships between the classes. Previous research has included three or four edge types in the graph. However, to accommodate the full complement of C++ language constructs, which include template classes and functions and nested classes, we extend the graph to include six edge types. The parameters to the cost model can be tuned to remove certain types of edges in an attempt to reduce the cost of the testing effort or to reduce the cost of breaking cycles in the graph. Our case study indicates that inclusion of inheritance edges in cycle breaking considerations may reduce the number of edge removals by a factor of two or more.