ABSTRACT The video gaming industry has evolved into an industry that drives high revenue and tangentially faces some of the toughest problems in the software engineering domain. With more games being released with either remote multiplayer components or cloud streaming foundations, maintaining an engaging player experience that lives up to a high‐quality bar is a problem that many game studios must overcome. Traditionally, studios have relied on brute‐force manual testing of their software to find impacting bugs. However, as studios and publishers focus on shortening the delivery timeline of their products to consumers, we have seen a rise in the use of testing tools, automation and enhanced processes that focus on achieving a high level of quality. In this systematic literature review, we provide a review of the tools and processes and discuss how these tools enhance the quality experience or help decrease the cost of testing while allowing for faster releases. We conclude by observing some potential areas of opportunity, such as suggesting an approach to help validate the severity of bugs found through community validation and providing a mechanism to create a way to track bugs across heterogeneous devices.

Model-driven software engineering (MDSE) has emerged as a popular and commonly used method for designing software systems in which models are the primary development artifact over the last decade. MDSE has resulted in the trend toward further automating the software process. However, the generation of test cases from user requirements still lags in reaching the required level of automation. Given that most user requirements are written in natural language, the recent advances in natural language processing (NLP) provide an opportunity to further automate the test generation process.In this paper, we exploit the advances in MDSE and NLP to generate abstract test cases from user requirements written in structured natural language and the respective data model. We accomplish this by creating meta-models for user requirements and abstract test cases and defining the appropriate transformation rules. To support this transformation, helper methods are defined to extract the relevant information from user requirements related to testing. To show the feasibility of the approach, we developed a prototype and conducted a case study with use cases and test cases from a Payroll Management System.

The behavior, attributes and properties of a software system is represented in a set of requirements that are written in structured natural language and are usually ambiguous. In large development projects, different modeling techniques are used to create and manage these requirements which aid in the analysis of the problem domain. Requirements are later used in the development process to create test cases, which is still mainly a manual process. To automate this process, we plan to use several of the techniques used in model-driven software development and Natural Language Processing(NLP). The approach under consideration is to use a model-to-model transformation to convert requirements into test cases with the support of Stanford CoreNLP techniques. Key to this transformation process is the use of meta-modeling for requirements and test cases. In this paper we focus on creating a comprehensive meta-model for requirements that can represent both use cases and user stories and performing preliminary analysis of the requirements using NLP. In later work we will develop a set of transformation rules to convert requirements into partial test cases. To show the feasibility of our approach we develop a prototype that can accept a cross-section of requirements written as both use cases and user stories.

In recent years there has been a focus on the automatic generation of test cases using white box testing techniques, however the same cannot be said for the generation of test cases at the system-level from natural language system requirements. Some of the white-box techniques include: the use of constraint solvers for the automatic generation of test inputs at the white box level; the use of control flow graphs generated from code; and the use of path generation and symbolic execution to generate test inputs and test for path feasibility. Techniques such as boundary value analysis (BVA) may also be used for generating stronger test suites. However, for black box testing we rely on specifications or implicit requirements and spend considerable time and effort designing and executing test cases. This paper presents an approach that leverages natural language processing and machine learning techniques to capture black box system behavior in the form of constraints. Constraint solvers are then used to generate test cases using BVA and equivalence class partitioning. We also conduct a proof of concept that applies this approach to a simplified task management application and an enterprise job recruiting application.

Agile development emphasizes collaborations among customers, business analysts, domain experts, developers, and testers. However, the large scale and rapid pace of many agile projects presents challenges during testing activities. Large sets of test artifacts must be comprehensible and available to various stakeholders, traceable to requirements, and easily maintainable as the software evolves. In this paper we describe Legend, a toolset that leverages domain-specific language to streamline functional testing in agile projects. Some key features of the toolset include test template generation from user stories, model-based automation, test inventory synchronization, and centralized test tagging.