For several years empirical studies have spanned the spectrum of research from software productivity, quality, reliability, performance to human computer interaction. Analyses have involved software systems ranging from desktop software to telecommunication switching systems. But surprising there has been little work done on the emerging digital game industry, one of the fastest growing domains today. To the best of our knowledge, our work is one of the first empirical analysis of a large commercially successful game system. In this paper, we introduce an analysis of the significant user data generated in the gaming industry by using a successful game: Project Gotham Racing 4.

The software engineering community has had seminal papers on data analysis for software productivity, quality, reliability, performance etc. Analyses have involved software systems ranging from desktop software to telecommunication switching systems. Little work has been done on the emerging digital game industry. In this paper we explore how data can drive game design and production decisions in game development. We define a mixture of qualitative and quantitative data sources, broken down into three broad categories: internal testing, external testing, and subjective evaluations. We present preliminary results of a case study of how data collected from users of a released game can inform subsequent development.

Automatic recording of user behavior within a system (instrumentation) to develop and test theories has a rich history in psychology and system design. Often, researchers analyze instrumented behavior in isolation from other data. The problem with collecting instrumented behaviors without attitudinal, demographic, and contextual data is that researchers have no way to answer the 'why' behind the 'what'. We have combined the collection and analysis of behavioral instrumentation with other HCI methods to develop a system for Tracking Real-Time User Experience (TRUE). Using two case studies as examples, we demonstrate how we have evolved instrumentation methodology and analysis to extensively improve the design of video games. It is our hope that TRUE is adopted and adapted by the broader HCI community, becoming a useful tool for gaining deep insights into user behavior and improvement of design for other complex systems.