While augmented and virtual reality technologies are becoming mainstream, it is still technically challenging and time-consuming to create new applications. Many designers draw from traditional low-fidelity prototyping methods that do not lend themselves well to designing in 3D. Developers use high-end programming frameworks such as Unity and Unreal which require significant hardware/software setups and coding skills. We see a gap in the medium-fidelity range where there is an opportunity for new tools to leverage the advantages of 360° content for AR/VR prototyping. Existing tools, however, have only limited support for 3D geometry, spatial and proxemic interactions, puppeteering, and storytelling. We present 360theater, a new method and a tool for rapid prototyping of AR/VR experiences, which takes dioramas into the virtual realm by enhancing 360° video capture with 3D geometry and simulating spatial interactions via Wizard of Oz. Our comparative evaluation of techniques with novice and experienced AR/VR designers shows that 360theater can close the gap and achieve a higher fidelity and more realistic AR/VR prototypes than comparable methods.

Significant tool support exists for the development of mixed reality (MR) applications; however, there is a lack of tools for analyzing MR experiences. We elicit requirements for future tools through interviews with 8 university research, instructional, and media teams using AR/VR in a variety of domains. While we find a common need for capturing how users perform tasks in MR, the primary differences were in terms of heuristics and metrics relevant to each project. Particularly in the early project stages, teams were uncertain about what data should, and even could, be collected with MR technologies. We designed the Mixed Reality Analytics Toolkit (MRAT) to instrument MR apps via visual editors without programming and enable rapid data collection and filtering for visualizations of MR user sessions. With MRAT, we contribute flexible interaction tracking and task definition concepts, an extensible set of heuristic techniques and metrics to measure task success, and visual inspection tools with in-situ visualizations in MR. Focusing on a multi-user, cross-device MR crisis simulation and triage training app as a case study, we then show the benefits of using MRAT, not only for user testing of MR apps, but also performance tuning throughout the design process.

What is Mixed Reality (MR)? To revisit this question given the many recent developments, we conducted interviews with ten AR/VR experts from academia and industry, as well as a literature survey of 68 papers. We find that, while there are prominent examples, there is no universally agreed on, one-size-fits-all definition of MR. Rather, we identified six partially competing notions from the literature and experts' responses. We then started to isolate the different aspects of reality relevant for MR experiences, going beyond the primarily visual notions and extending to audio, motion, haptics, taste, and smell. We distill our findings into a conceptual framework with seven dimensions to characterize MR applications in terms of the number of environments, number of users, level of immersion, level of virtuality, degree of interaction, input, and output. Our goal with this paper is to support classification and discussion of MR applications' design and provide a better means to researchers to contextualize their work within the increasingly fragmented MR landscape.

Designers and researchers often rely on simple gesture recognizers like Wobbrock et al.'s $1 for rapid user interface prototypes. However, most existing recognizers are limited to a particular input modality and/or pre-trained set of gestures, and cannot be easily combined with other recognizers. In particular, creating prototypes that employ advanced touch and mid-air gestures still requires significant technical experience and programming skill. Inspired by $1's easy, cheap, and flexible design, we present the GestureWiz prototyping environment that provides designers with an integrated solution for gesture definition, conflict checking, and real-time recognition by employing human recognizers in a Wizard of Oz manner. We present a series of experiments with designers and crowds to show that GestureWiz can perform with reasonable accuracy and latency. We demonstrate advantages of GestureWiz when recreating gesture-based interfaces from the literature and conducting a study with 12 interaction designers that prototyped a multimodal interface with support for a wide range of novel gestures in about 45 minutes.

360-degree video is increasingly used to create immersive user experiences; however, it is typically limited to a single user and not interactive. Recent studies have explored the potential of 360 video to support multi-user collaboration in remote settings. These studies identified several challenges with respect to 360 live streams, such as the lack of gaze awareness, out-of-sync views, and missed gestures. To address these challenges, we created 360Anywhere, a framework for 360 video-based multi-user collaboration that, in addition to allowing collaborators to view and annotate a 360 live stream, also supports projection of annotations in the 360 stream back into the real-world environment in real-time. This enables a range of collaborative augmented reality applications not supported with existing tools. We present the 360Anywhere framework and tools that allow users to generate applications tailored to specific collaboration and augmentation needs with support for remote collaboration. In a series of exploratory design sessions with users, we assess 360Anywhere's power and flexibility for three mobile ad-hoc scenarios. Using 360Anywhere, participants were able to set up and use fairly complex remote collaboration systems involving projective augmented reality in less than 10 minutes.

Augmented Reality (AR) developers face a proliferation of new platforms, devices, and frameworks. This often leads to applications being limited to a single platform and makes it hard to support collaborative AR scenarios involving multiple different devices. This paper presents XD-AR, a cross-device AR application development framework designed to unify input and output across hand-held, head-worn, and projective AR displays. XD-AR's design was informed by challenging scenarios for AR applications, a technical review of existing AR platforms, and a survey of 30 AR designers, developers, and users. Based on the results, we developed a taxonomy of AR system components and identified key challenges and opportunities in making them work together. We discuss how our taxonomy can guide the design of future AR platforms and applications and how cross-device interaction challenges could be addressed. We illustrate this when using XD-AR to implement two challenging AR applications from the literature in a device-agnostic way.

In Europe there are more than 580 000 people who suffer from aphasia - an acquired speech and language disorder that occurs because of brain damage, primarily as a result of a stroke. Especially with regards to demographic change, health care systems have to face present and future challenges to improve aphasia therapy. Thereby, immediate therapeutic measures are decisive for best possible and long-term success in language therapy. Regarding essential requirements, on the one hand, therapy intensity and frequency have to be increased significantly while on the other hand, measures need to be adjusted along everyday activities. A very promising approach to meet this requirements are augmented reality applications. They can be used to create a highly natural exercise situation, in which patients interact and practice with their personal possessions at home. This facilitates the successful and continuous transfer of learnings for the patient, contrary to being solely dependent on clinical therapy units. This paper gives an overview of the concept of a real-time software providing augmented and dynamic language therapy, which is interactive and utilizes simple user interface design, for home-based training.

Over the past 20 years, search engines have become emph{the} entry point of the WWW. Due to evolving needs for different and new kinds of information, the interfaces of search engine results pages (SERPs) change over time. Thus, their usability must be continuously evaluated to ensure user satisfaction and competitive edge. As no complete solution exists, we present emph{S.O.S.:} the emph{SERP Optimization Suite}. Our approach comprises (a) emph{WaPPU}, which is a near real-time tool for evaluating web interfaces based on usability scores, and (b) a emph{catalog of best practices} that maps bad scores to potential causes and corresponding adjustments for optimization. During a case study in which we assessed and optimized a real-world SERP, S.O.S. has proven its feasibility and effectiveness by significantly improving the SERP's usability.

Web designers currently face the increased proliferation and diversity of new touch devices which pose major challenges to the design task. This paper presents W3Touch - an interface instrumentation toolkit for web designers to collect user performance data for different device characteristics in order to help them identify potential design problems for touch interaction. Web designers can visualise the data aggregated by W3Touch and use simple metrics to automate the adaptation process for many different viewing and interaction contexts. In a series of experiments with web designers and users, we show that W3Touch is able to detect interaction problems that are hard to find using conventional methods and demonstrate how the tool was successfully used to automate the desktop-to-mobile migration of Wikipedia as an example.

It is crucial for the success of a search-driven web application to answer users' queries in the best possible way. A common approach is to use click models for guessing the relevance of search results. However, these models are imprecise and waive valuable information one can gain from non-click user interactions. We introduce TellMyRelevance!---a novel automatic end-to-end pipeline for tracking cursor interactions at the client, analyzing these and learning according relevance models. Yet, the models depend on the layout of the search results page involved, which makes them difficult to evaluate and compare. Thus, we use a Random Mouse Cursor as an extension to our pipeline for generating layout-dependent baselines. Based on these, we can perform evaluations of real-world relevance models. A large-scale interaction log analysis showed that we can learn relevance models whose predictions compare favorably to predictions of an existing state-of-the-art click model.