Optical see-through head-mounted displays (OST HMDs) are a popular output medium for mobile Augmented Reality (AR) applications. To date, they lack efficient text entry techniques. Smartphones are a major text entry medium in mobile contexts but attentional demands can contribute to accidents while typing on the go. Mobile multi-display ecologies, such as combined OST HMD-smartphone systems, promise performance and situation awareness benefits over single-device use. We study the joint performance of text entry on mobile phones with text output on optical see-through head-mounted displays. A series of five experiments with a total of 86 participants indicate that, as of today, the challenges in such a joint interactive system outweigh the potential benefits.

Drone assisted navigation aids for supporting walking activities of visually impaired have been established in related work but fine-point object grasping tasks and the object localization in unknown environments still presents an open and complex challenge. We present a drone-based interface that provides fine-grain haptic feedback and thus physically guides them in hand-object localization tasks in unknown surroundings. Our research is built around community groups of blind or visually impaired (BVI) people, which provide in-depth insights during the development process and serve later as study participants. A pilot study infers users’ sensibility to applied guiding stimuli forces and the different human-drone tether interfacing possibilities. In a comparative follow-up study, we show that our drone-based approach achieves greater accuracy compared to a current audio-based hand guiding system and delivers overall a more intuitive and relatable fine-point guiding experience.

We present a dynamic error prediction system for industrial production machines. We implemented a flexible data collection tool to create error warnings for a production line, which aims to improve the already existing static alarm models. For industrial machines, there are threshold-based alarm models set by prior experiences and observations of the operator. For machines without standardized interfaces and communication protocols, which are not Industry 4.0 compatible, it represents a challenge to implement and add a dynamic and opportunistic system behavior. Machines need to learn from past errors autonomously and adapt the production properties dynamically. We implemented a framework that makes production machines conform to the Internet of Things (IoT) concepts, by making previously non-IoT enabled resources available to get new insights into the production processes.The system component recognition and the database setup is done fully automatically by our developed system.We designed and applied a feature-based data drift model in a real-world industrial setting to determine data deviation between normal and erroneous work-pieces in real-time to predict upcoming erroneous behavior. The drift analysis flagged and predicted work-pieces as erroneous several minutes before the pre-defined machine alarms would have been raised. The resulting flagged sensors and values can be compared to the system determined errors to get new insights into the abnormal machine behavior. For the reduction of downtime, the most valuable immediate result of the system is the ability to notify the operator earlier and reduce overall downtime.

Alphanumeric text entry is a challenge for Virtual Reality (VR) applications. VR enables new capabilities, impossible in the real world, such as an unobstructed view of the keyboard, without occlusion by the user's physical hands. Several hand representations have been proposed for typing in VR on standard physical keyboards. However, to date, these hand representations have not been compared regarding their performance and effects on presence for VR text entry. Our work addresses this gap by comparing existing hand representations with minimalistic fingertip visualization. We study the effects of four hand representations (no hand representation, inverse kinematic model, fingertip visualization using spheres and video inlay) on typing in VR using a standard physical keyboard with 24 participants. We found that the fingertip visualization and video inlay both resulted in statistically significant lower text entry error rates compared to no hand or inverse kinematic model representations. We found no statistical differences in text entry speed.

We study the performance and user experience of two popular mainstream text entry devices, desktop keyboards and touchscreen keyboards, for use in Virtual Reality (VR) applications. We discuss the limitations arising from limited visual feedback, and examine the efficiency of different strategies of use. We analyze a total of 24 hours of typing data in VR from 24 participants and find that novice users are able to retain about 60% of their typing speed on a desktop keyboard and about 40-45% of their typing speed on a touchscreen keyboard. We also find no significant learning effects, indicating that users can transfer their typing skills fast into VR. Besides investigating baseline performances, we study the position in which keyboards and hands are rendered in space. We find that this does not adversely affect performance for desktop keyboard typing and results in a performance trade-off for touchscreen keyboard typing.

We present HeadPhones (Headtracking + smartPhones), a novel approach for the spatial registration of multiple mobile devices into an ad hoc multi-display environment. We propose to employ the user's head as external reference frame for the registration of multiple mobile devices into a common coordinate system. Our approach allows for dynamic repositioning of devices during runtime without the need for external infrastructure such as separate cameras or fiducials. Specifically, our only requirements are local network connections and mobile devices with built-in front facing cameras. This way, HeadPhones enables spatially-aware multi-display applications in mobile contexts. A user study and accuracy evaluation indicate the feasibility of our approach.

In this study, we develop an electromagnetic-based haptic interface to provide controlled magnetic forces to the operator through a wearable haptic device (an orthopedic finger splint with single dipole moment) without position feedback. First, we model the electromagnetic forces exerted on a single magnetic dipole attached to the wearable haptic device, and derive magnetic force-current mapping for the dipole moment. Second, this mapping is used as basis for parameter selection of the electromagnetic coils of the haptic interface, dipole moment of the wearable haptic device, and the operating workspace of the system. The electromagnetic-based haptic interface enables three-dimensional (3D) virtual object rendering in mid-air within a workspace of 150 mm × 150 mm × 20 mm, using magnetic forces in excess of 50 mN. Participants experimentally demonstrate a 61% success rate in distinguishing the geometry of 4 representative 3D virtual objects. However, our statistical analysis shows that the ability of the participants to distinguish between geometries is not statistically significant, for 95% confidence level.

While we witness significant changes in display technologies, to date, the majority of display form factors remain flat. The research community has investigated other geometric display configuration given the rise to cubic displays that create the illusion of a 3D virtual scene within the cube. We present a self-contained mobile perspective cubic display (mpCubee) assembled from multiple smartphones. We achieve perspective correct projection of 3D content through head-tracking using built-in cameras in smartphones. Furthermore, our prototype allows to spatially manipulate 3D objects on individual axes due to the orthogonal configuration of touch displays.

We present a demonstration of HeadPhones (Headtracking + smart-Phones), a novel approach for the spatial registration of multiple mobile devices into an ad hoc multi-display environment. We propose to employ the user's head as external reference frame for the registration of multiple mobile devices into a common coordinate system. Our approach allows for dynamic repositioning of devices during runtime without the need for external infrastructure such as separate cameras or fiducials. Specifically, our only requirements are local network connections and mobile devices with built-in front facing cameras. This way, HeadPhones enables spatially-aware multi-display applications in mobile contexts.

We present a novel approach for extending the input space around unmodified mobile devices. Using built-in front-facing cameras of unmodified handheld devices, GlassHands estimates hand poses and gestures through reflections in sunglasses, ski goggles or visors. Thereby, GlassHands creates an enlarged input space, rivaling input reach on large touch displays. We introduce the idea along with its technical concept and implementation. We demonstrate the feasibility and potential of our proposed approach in several application scenarios, such as map browsing or drawing using a set of interaction techniques previously possible only with modified mobile devices or on large touch displays. Our research is backed up with a user study.

Data glasses do carry promising potential for hands-free interaction, but also raise various concerns amongst their potential users. In order to gain insights into the nature of those concerns, we investigate how potential usage scenarios are perceived by device users and their peers. We present results of a two-step approach: a focus group discussion with 7 participants, and a user study with 38 participants. In particular, we look into differences between the usage of data glasses and more established devices such as smart phones. We provide quantitative measures for scenario-related social acceptability and point out factors that can influence user attitudes. Based on our quantitative and qualitative results, we derive design implications that might support the development of head-worn devices and applications with an improved social acceptability.

Immersive AR environments are situated information spaces that employ the Augmented Reality display paradigm. They tightly couple abstract, digital information and physical elements present in the user's real-world environment. As a consequence of this mixed digital-physical spatiality, interface design has the chance to build upon the user's cognitive map (spatial mental model) of her environment. The proposed PhD project investigates how knowledge about a user's spatial mental model of an augmented scene can be translated to best practices in the design of remotely shared AR environments.

Mobile location recognition by capturing images of the environment (visual localization) is a promising technique for indoor navigation in arbitrary surroundings. However, it has barely been investigated so far how the user interface (UI) can cope with the challenges of the vision-based localization technique, such as varying quality of the query images. We implemented a novel UI for visual localization, consisting of Virtual Reality (VR) and Augmented Reality (AR) views that actively communicate and ensure localization accuracy. If necessary, the system encourages the user to point the smartphone at distinctive regions to improve localization quality. We evaluated the UI in a experimental navigation task with a prototype, informed by initial evaluation results using design mockups. We found that VR can contribute to efficient and effective indoor navigation even at unreliable location and orientation accuracy. We discuss identified challenges and share lessons learned as recommendations for future work.

In this paper, we review the use of gameful design in the automotive domain. Outside of vehicles the automotive industry is mainly using gameful design for marketing and brand forming. For in-vehicle applications and for applications directly connected to real vehicles, the main usage scenarios of gameful design are navigation, eco-driving and driving safety. The objective of this review is to answer the following questions: (1) What elements of gameful design are currently used in the automotive industry? (2) What other automotive applications could be realized or enhanced by applying gameful design? (3) What are the challenges and limitations of gameful design in this domain especially for in-vehicle applications? The review concludes that the use of gameful design for in-vehicle applications seems to be promising. However, gamified applications related to the serious task of driving require thought-out rules and extensive testing in order to achieve the desired goal.

Self-reporting techniques, such as data logging or a diary, are frequently used in long-term studies, but prone to subjects' forgetfulness and other sources of inaccuracy. We conducted a six-week self-reporting study on smartphone usage in order to investigate the accuracy of self-reported information, and used logged data as ground truth to compare the subjects' reports against. Subjects never recorded more than 70% and, depending on the requested reporting interval, down to less than 40% of actual app usages. They significantly overestimated how long they used apps. While subjects forgot self-reports when no automatic reminders were sent, a high reporting frequency was perceived as uncomfortable and burdensome. Most significantly, self-reporting even changed the actual app usage of users and hence can lead to deceptive measures if a study relies on no other data sources.

The ongoing evolution of mobile phones to "pocket computers" generated a demand for more and more applications to be ported to the mobile phone. Because a full security assessment for a whole mobile operating system would be prohibitively costly, currently security critical applications can not be implemented. We address this challenge by introducing security zones to enable applications with high security demands like driving licenses, health insurance cards, or passports on mobile phones. This zone concept creates the need for visualization of the current zone and a way to switch between zones. In this paper we discuss several possible ways of achieving this.

We consider the reasons why we organise books in a physical environment and investigate whether situating interactions with a smartphone could improve the user experience of e-readers. Our prototype uses the Kinect depth sensor to detect the position of a user in relation to sections of a physical bookshelf. We also built a mobile application that allows users to browse and organise digital books by moving between each section. We present our initial observations of a user study that evaluated search and categorisation tasks with our prototype. Our findings motivate reasons to explore digital books in a physical environment and indicate issues to consider when designing situated interactions with e-readers.

User authentication on publicly exposed terminals with established mechanisms, such as typing the credentials on a virtual keyboard, can be insecure e.g. due to shoulder surfing or due to a hacked terminal. In addition, username and password entry can be time-consuming and thus improvable with relation to usability. As security and comfort are often competing with each other, novel authentication and authorization methods especially for public terminals are desirable. In this paper, we present an approach on a distributed authentication and authorization system, where the user can be easily identified and enabled to use a service with his smartphone. The smartphone (as personal and private device the user is always in control of) can provide a highly secure authentication token that is renewed and exchanged in the background without the user's participation. The claimed improvements were supported by a user survey with an implementation of a digital room management system as an example for a public display. The proposed authentication procedure would increase security and yet enable fast authentication within publicly exposed terminals.

Saving money is usually a tedious task that requires a high degree of self-control for many of us. Some people have one or more specific savings targets in mind and thus need to prioritize them. We propose connecting a savings box with a personal smartphone. Thus, people become motivated to keep track of their savings for multiple targets. Using a savings box capable of counting money and connecting it to an app, we believe people stick to their plans to save with higher motivation and are happier with their behavior. In this paper, we present first evidence for the success of this concept. We gathered feedback through an online user study in which participants were shown a video prototype. We propose further research directions with our SmartPiggy, to confirm the feasibility of behavioral economics in HCI.

We present a visual odometry system for indoor navigation with a focus on long-term robustness and consistency. As our work is targeting mobile phones, we employ monocular SLAM to jointly estimate a local map and the device's trajectory. We specifically address the problem of estimating the scale factor of both, the map and the trajectory. State-of-the-art solutions approach this problem with an Extended Kalman Filter (EKF), which estimates the scale by fusing inertial and visual data, but strongly relies on good initialization and takes time to converge. Each visual tracking failure introduces a new arbitrary scale factor, forcing the filter to re-converge. We propose a fast and robust method for scale initialization that exploits basic geometric properties of the learned local map. Using random projections, we efficiently compute geometric properties from the feature point cloud produced by the visual SLAM system. From these properties (e.g., corridor width or height) we estimate scale changes caused by tracking failures and update the EKF accordingly. As a result, previously achieved convergence is preserved despite re-initializations of the map. To minimize the time required to continue tracking after failure, we perform recovery and re-initialization in parallel. This increases the time available for recovery and hence the likelihood for success, thus allowing almost seamless tracking. Moreover, fewer re-initializations are necessary. We evaluate our approach using extensive and diverse indoor datasets. Results demonstrate that errors and convergence times for scale estimation are considerably reduced, thus ensuring consistent and accurate scale estimation. This enables long-term odometry despite of tracking failures which are inevitable in realistic scenarios.

Vision-based approaches for mobile indoor localization do not rely on the infrastructure and are therefore scalable and cheap. The particular requirements to a navigation user interface for a vision-based system, however, have not been investigated so far.

We present GymSkill, a personal trainer for ubiquitous monitoring and assessment of physical activity using standard fitness equipment. The system records and analyzes exercises using the sensors of a personal smartphone attached to the gym equipment. Novel fine-grained activity recognition techniques based on pyramidal Principal Component Breakdown Analysis (PCBA) provide a quantitative analysis of the quality of human movements. In addition to overall quality judgments, GymSkill identifies interesting portions of the recorded sensor data and provides suggestions for improving the individual performance, thereby extending existing work. The system was evaluated in a case study where 6 participants performed a variety of exercises on balance boards. GymSkill successfully assessed the quality of the exercises, in agreement with the professional judgment provided by a physician. User feedback suggests that GymSkill has the potential to serve as an effective tool for motivating and supporting lay people to overcome sedentary, unhealthy lifestyles. GymSkill is available in the Android Market as `VMI Fit'.

We introduce and define Cognitive Objects for human-robot interaction and human-computer interaction and disambiguate them against existing ‘Smart Objects’. Cognitive Objects are physical real-world objects used in manipulation tasks by humans and robots. As such, they incorporate self-awareness, reduce ambiguity and uncertainty in object recognition, and provide services to both humans and robots during their usage in real-world environments. We distinguish Cognitive Objects from other ‘Smart Objects’ and computationally enriched artifacts, outline their characteristics, and describe their potential impact on human-computer and human-robot interaction with real-world objects.

Advances in the fields of distributed computing, wireless sensor networks, ubiquitous computing and many more have facilitated the development of a plenitude of Intelligent Environments (IE). The development processes, underlying tools and methods have however not yet been in the focus of researchers. A significant amount of time has to be spent for building a novel ecology of distributed systems to form an Intelligent Environment to conduct novel research in. A sophisticated middleware can shift the focus from building to research. Yet, a middleware is only one of the many requirements for leveraging the development process of Intelligent Environments. In this paper, we present a complete tool chain to simplify research on Intelligent Environments and to ease their creation. This tool chain consists out of middleware, simulation, visualization and prototyping tools. We start by modeling the physical environment using a CAD software and visualization tool including physical properties. The tool chain is complemented by a middleware and a sophisticated simulation environment allowing e.g. to simulate mobile phone interactions triggering, and the deployment to the real-world events. For developers and researchers of Intelligent Environments, creation and maintenance of Intelligent Environments is significantly improved. Development is speeded up as algorithms, interaction can be already evaluated and tested in simulation before the deployment to the real world.

"There's An App For That" -- but how do we actually develop them? While smartphones and tablet PCs are getting more and more popular and their application scenarios are growing, we still develop them using only a standard integrated development environment. Although context-based services and apps do, next to network connectivity, require lots of sensor data, the tools for providing realistic sensor data during development are still immature.

With fast proceeding demographic change, lack of qualified nursing staff and costs explosion concerning our health and welfare systems, new methods have to be developed to support elderly people in their home environment with ambient integrated and intelligent assistance/service technologies. Service units and related typologies presented in this paper are adding new layers of functionalities to conventional home care environments. To simplify implementation, service units shift the complexity arising with more and more complex high-tech buildings to a compact entity/unit which could be industrially prefabricated/customized and equipped with various assistance technologies in a controlled factory environment. Moreover, service units can be installed within new or existing buildings rapidly and with minimized effort as an additional building sub-module.

While there is more to context than location, localization and positioning must continue to be improved. Location-aware applications, such as Google Latitude, are enjoying great popularity. Location-based applications and services are widely used on platforms such as the iPhone. The localization itself thus remains an issue, especially in indoor scenarios. In this paper we discuss Digital Enhanced Cordless Telecommunications (DECT) and its potential for positioning and localization. While the technology itself is not new, the application of DECT for localization has so far not been thoroughly investigated, especially for combined indoor and outdoor usage. We have collected and analyzed large data sets of DECT and WLAN fingerprints in urban, sub-urban and rural areas, both for indoor and outdoor situations. We show that DECT could improve the accuracy and robustness of existing localization schemes based on WLAN or GSM.

In this paper, we present a concept for an open vehicular data interface and describe it's components and architecture. We discuss the enabled applications in the context of advanced driver assistance systems with a focus on human-machine interfaces, vehicle-to-x (V2X) communication and context inference systems. We conclude by a presentation of the initial implementation and deployed system.

Current driver assistance systems such as Adaptive Cruise Control (ACC) and in particular future assistance systems such as Collision Warning make high demands on reliability of detection and ranging methods for vehicles within the local vicinity. Autonomous systems such as Radar which are already integrated into a multitude of vehicles meet these requirements to only a limited extent. As an alternative, cooperative systems for detection and ranging will be enabled by future Vehicle-2-Vehicle communication. But cooperative detection and ranging also has drawbacks regarding reliability due to positioning and transmission errors if it is applied in a standalone way. Thus, the solution presented in this paper is a hybrid approach combining autonomous and cooperative methods for detection and ranging within a common architecture. A particle filter is used for state estimation. The results are a higher detection effectiveness and a lower position error compared to using standalone autonomous or cooperative detection and ranging methods.

Today's automotive sensor systems for in-vehicle based target tracking, i.e. radar, lidar, camera, are limited to a field of view which is restricted by distance, angle and line-of-sight. Future driver assistance systems such as predictive collision avoidance or situation-aware adaptive cruise control require a more complete and accurate situation awareness in order to detect hazardous and inefficient situations in time. Therefore, we introduce multi-target tracking including vehicle-2-vehicle communications as a complementing sensor for future driver assistance systems. The paper presents first simulation results of our algorithm which show promising outcomes.

Today's automotive sensor systems for in-vehicle based target tracking, i.e. radar, lidar, camera, are limited to a field of view which is restricted by distance, angle and line-of-sight. Future driver assistance systems such as predictive collision avoidance or situation-aware adaptive cruise control require a more complete and accurate situation awareness in order to detect hazardous and inefficient situations in time. Therefore, we introduce multi-target tracking including vehicle-2-vehicle communications as a complementing sensor for future driver assistance systems. The demonstration presents set-up, functionality and results of multi-sensor multi-target tracking based on particle filtering.

In this paper, we report on a user-centered, iterative design process for augmenting sports equipment with ubiquitous computing technology. In several design iterations, a fully working system for training and physiotherapy has been developed and deployed using hard- and software components of ubiquitous computing technology. We report on the design and development process that led to this system. Based on our experience we generalize the specific processes to general ubicomp systems. The validity of our approach has been verified by a larger user study

In this paper we present a toolkit for realizing capacitive sensing applications for human-computer interaction in pervasive computing systems. We argue that capacitive sensors - due to their unique properties - are well suited for many pervasive and ubiquitous computing applications and scenarios. We describe the CapToolKit designed to rapidly realize prototypes and systems that are able to detect the presence of humans and objects. Our toolkit also allows the integration of 3D interaction with everyday objects as well as instrumented environments. We illustrate its capabilities by presenting several applications implemented using CapToolKit. The entire system will be open-sourced to allow utilization of our technology within other research projects. By building on the existing toolkit researchers are provided a foundation for developing their own sensor systems, algorithms, and applications

We report on approaches for context-awareness in a kitchen environment. Two devices, an augmented cutting board and a sensor-enriched knife, enable the environment to determine the type of food handled during the preparation of meals.