Despite the advantages of using expert evaluation as a method within games user research (GUR) (i.e. provides stakeholders low cost, rapid feedback), it does not always accurately reflect the general player’s experience. Testing the game out with real users (also called playtesting) helps bridge this gap by giving game developers an in-depth look into the player experience. However, playtesting is resource intensive and time consuming, making it difficult to implement within the tight time frames of industry game development. AI can help to mitigate some of these issues by providing an automated way to simulate player behaviour and experience. In this paper, we introduce a tool called PathOS+—a playtesting interface which uses AI playtesting data to help enhance expert evaluation. Results from a study conducted with expert participants shows how PathOS+ could contribute to game design and assist developers and researchers in conducting expert evaluations. This is an important contribution as it provides game user researchers and designers with a fast, low-cost and effective game evaluation approach which has the potential to make game evaluation more accessible to indie and smaller game studios.

Game studios aim to develop titles that deliver a fun and engaging experience for players. Playtesting promises to help identify opportunities to improve player experience and assist developers in achieving their design intent. However, a lack of research on the added value of playtesting means that many studios are still uncertain about its commercial viability and impact on product success. This gap in understanding is further complicated by the vague definition of "success" afforded by sales figures and review scores. In this paper, we assess reported feature quality of three commercial titles by analyzing playtesting reports and game reviews. By comparing themes and design issues expressed in game reviews to the results of pre-release playtesting for each game, we aim to highlight the value of playtesting and propose a set of guidelines for selecting playtest methods based on the needs of a given game evaluation. Through the real-world case studies presented, this paper contributes to the growing domain of games user research and highlights the value of playtesting in game development.