This article introduces model checking as an alternative method to estimate the latency and parallelism of asynchronous Realtime Interactive Systems (RISs). Five typical concurrency and synchronization schemes often found in concurrent Virtual Reality (VR) and computer game systems are identified as use-cases. These use-cases guide the development a) of software primitives necessary for the use-case implementation based on asynchronous RIS architectures and b) of a graphical editor for the specification of various concurrency and synchronization schemes (including the use-cases) based on these primitives. Several model-checking tools are evaluated against typical requirements in the RIS area. As a result, the formal model checking language Rebeca and its model checker RMC are applied to the specification of the use-cases to estimate latency and parallelism for each case. The estimations are compared to measured results achieved by classical profiling from a real-world application. The estimated results of the latencies by model checking approximated the measured results adequately with a minimal difference of 3.9% in the best case and -26.8% in the worst case. It also detected a problematic execution path not covered by the stochastic nature of the measured profiling samples. The estimated results of the degree of parallelization by model checking are approximated with an minimal difference of 9.3% and a maximal difference of -28.8%. Finally, the effort of model checking is compared to the effort of implementing and profiling a RIS.

This article describes a set of metrics for a message-passing-based asynchronous Realtime Interactive System (RIS). Current trends in concurrent RISs are analyzed, several profiling tools are outlined, and common metrics are identified. A set of nine metrics is presented in a unified and formalized way. The implementation of a profiler that measures and calculates these metrics is illustrated. The implementation of an instrumentation and a visualization tool are described. A case study shows how this approach proved beneficial during the optimization of latency of an actual system.

This article evaluates the utility of three technical design approaches implemented during the development of a Realtime Interactive Systems (RIS) architecture focusing on the areas of Virtual and Augmented Reality (VR and AR), Robotics, and Human-Computer Interaction (HCI). The design decisions are (1) the choice of the Scala programming language, (2) the implementation of the actor computational model, and (3) the central incorporation of ontologies as a base for semantic modeling, required for several Artificial Intelligence (AI) methods. A white-box expert review is applied to a detailed use case illustrating an interactive and multimodal game scenario, which requires a number of complex functional features like speech and gesture processing and instruction mapping. The review matches the three design decisions against three comprehensive non-functional requirements from software engineering: Reusability, scalability, and extensibility. The qualitative evaluation is condensed to a semi-quantitative summary, pointing out the benefits of the chosen technical design.

We present SiXton 's Curse-a computer game-to illustrate the ben efits of a novel simulation platform. Simulator X [2] targets vir tual, augmented, and mixed reality applications as well as computer games. The game simulates a medieval village called SiXton that can be explored and experienced using gestures and speech for in put. SiXton's Curse utilizes multiple independent components for physical simulation, sound and graphics rendering, artificial intelli gence, as well as for multi-modal interaction (MMI). The compo nents are already an integral part of Simulator X's current version. Building on Hewitt's actor model, the Simulator X platform enables the developer to easily exploit the capabilities of modern hardware architectures. A state variable concept is implemented on top of the actor model to grant uniform and easy access to global states and values by using the internal mechanisms of the actor model. Communication via an asynchronous messaging in terface reduces component coupling. The scalability of the actor model provides a uniform concurrency paradigm on different levels of granularity as well as exchangeability of architectural elements and components.

This article presents a platform for software technology research in the area of intelligent Realtime Interactive Systems. Simulator X is targeted at Virtual Reality, Augmented Reality, Mixed Reality, and computer games. It provides a foundation and testbed for a variety of different application models. The current research architecture is based on the actor model to support fine grained concurrency and parallelism. Its design follows the minimize coupling and maximize cohesion software engineering principle. A distributed world state and execution scheme is combined with an object-centered world view based on an entity model. Entities conceptually aggregate properties internally represented by state variables. An asynchronous event mechanism allows intra- and interprocess communication between the simulation actors. An extensible world interface uses an ontology-based semantic annotation layer to provide a coherent world view of the resulting distributed world state and execution scheme to application developers. The world interface greatly simplifies configurability and the semantic layer provides a solid foundation for the integration of different Artificial Intelligence components. The current architecture is implemented in Scala using the Java virtual machine. This choice additionally fosters low-level scalability, portability, and reusability.

Describes how a physical simulation can be integrated with our Responsive Workbench system to support complex assembly tasks involving multiple hands and users. Our system uses the CORIOLIS physical simulation package extended to meet the real-time requirements for our highly interactive virtual environment. We develop a new set of interface tools that exploit the natural properties of physical simulation (i.e. the superposition of forces). Our tools are based on sets of springs connecting the user's hand to a virtual object. Visualizing these springs provides "visual force feedback" of the applied forces and facilitates the prediction of the objects' behavior. Our force-based interaction concept allows multiple hands and users to manipulate a single object without the need for locking the object.

We present Avocado, our object-oriented framework for the development of distributed, interactive virtual environment applications. Data distribution is achieved by transparent replication of a shared scene graph among the participating processes of a distributed application. A sophisticated group communication system is used to guarantee state consistency even in the presence of late joining and leaving processes. We also describe how the familiar data flow graph found in modern stand-alone 3D-application toolkits extends nicely to the distributed case.