Modern vehicles are using AI and increasingly sophisticated sensor suites to improve Advanced Driving Assistance Systems (ADAS) and support automated driving capabilities. Heads-Up-Displays (HUDs) provide an opportunity to visually inform drivers about vehicle perception and interpretation of the driving environment. One approach to HUD design may be to reveal to drivers the vehicle’s full contextual understanding, though it is not clear if the benefits of additional information outweigh the drawbacks of added complexity, or if this balance holds across drivers. We designed and tested an Augmented Reality (AR) HUD in an online study (N = 298), focusing on the influence of HUD visualizations on drivers’ situation awareness and perceptions. Participants viewed two driving scenes with one of three HUD conditions. Results were nuanced: situation awareness declined with increasing driving context complexity, and contrary to expectation, also declined with the presence of a HUD compared to no HUD. Significant differences were found by varying HUD complexity, which led us to explore different characterizations of complexity, including counts of scene items, item categories, and illuminated pixels. Our analysis finds that driving style interacts with driving context and HUD complexity, warranting further study.

Glanceability and low access time are arguably the key assets of a smartwatch. Smartwatches are designed for, and excel at micro-interactions- simple tasks that only take seconds to complete. However, if a user desires to transition to a task requiring sustained usage, we show that there are additional factors that prevent possible longer usage of the smartwatch. In this paper, we conduct a study with 18 participants to empirically demonstrate that interacting with the smartwatch on the wrist leads to fatigue after only a few minutes. In our study, users performed three tasks in two different poses while using a smartwatch. We demonstrate that only after three minutes of use, the change in perceived exertion of the user was anchored as "somewhat strong" on the Borg CR10 survey scale. These results place an upper bound for smartwatch usage that needs to be considered in application and interaction design.

An infrastructure-less indoor localization system is proposed based on fingerprints of light signals acquired at high frequencies. In contrast to other systems that modulate lights, the proposed system distinguishes lights by learning from training samples. Due to slight differences in the electronic components used in the construction of compact fluorescent light (CFL) and light emitting diode (LED) bulbs, the optical signals emitted by each light bulb have slight differences with other light bulbs even within the same brand and model. Light signals are digitized with a fast and accurate analog-to-digital converter (ADC) at up to 1 mega-samples/second, segmented, and mapped into the frequency domain using the Fast Fourier Transform (FFT). Spectral features based on the FFT are filtered, normalized, and used as training data for supervised machine learning algorithms. Results are provided for two classifiers of varying complexity: (1) A k-Nearest Neighbor (KNN) classifier; (2) A Convolutional Neural Net (CNN) classifier. A hardware system for indoor localization was designed to analyze the performance of the classifiers. Under certain restrictions, results show that light bulbs may be identified with high accuracy without special infrastructure for modulation. Identifying a light bulb is meant to be synonymous with identifying its associated location.

In this paper we develop a model capable of classifying drivers from their driving behaviors sensed by only low level sensors. The sensing platform consists of data available from the diagnostic outlet (OBD) of the car and smartphone sensors. We develop a window based support vector machine model to classify drivers. We test our model with two datasets collected under both controlled and naturalistic conditions. Furthermore, we evaluate the model using each sensor source (car and phone) independently and combining both the sensors. The average classification accuracies attained with data collected from three different cars shared between couples in a naturalistic environment were 75.83%, 85.83% and 86.67% using only phone sensors, only cars sensors and combined car and phone sensors respectively.

As a new generation of smartwatches enters the market, one common use is for displaying information such as notifications. While some content might warrant immediately interrupting a user, there is also information that might be important to display yet less urgent. It would be useful to show this content on the watch but not immediately draw the user's attention away from their primary task. In this paper, we investigate how fast three visual parameters draw a user's attention. In particular, we present data from a smartwatch user study where we examine the size, frequency, and color of a visual prompt and the associated impact on reaction time. We find statistically significant differences for size and frequency where smaller and slower result in the less immediate reactions. We also present reaction time distributions that a designer can use to tailor expected notification response times to match their content.

Tracking multiple people in an indoor environment enables various smart building applications such as HVAC energy saving, patient/child monitoring, etc. Researchers have explored various sensing methods including vision, motion, and RF, which either require specific installation requirements or high deployment density. We introduce a passive sparse sensing method based on ambient structural vibration induced by foot strikes. Our system tracks multiple people based on the premise that human foot strikes have spatio-temporal variation, and hence do not fully overlap. The system achieved less than 0.4m accuracy in both one and two persons stepping conditions.

Driven by curiosity and our own three diverse smartphone application usage datasets, we sought to unpack the nuances of mobile device use by revisiting two recent Mobile HCI studies [1, 17]. Our goal was to add to our broader understanding of smartphone usage by investigating if differences in mobile device usage occurred not only across our three datasets, but also in relation to prior work. We found differences in the top-10 apps in each dataset, in the durations and types of interactions as well as in micro-usage patterns. However, it proved very challenging to attribute such differences to a specific factor or set of factors: was it the time frame in which the studies were executed? The recruitment procedure? The experimental method? Using our somewhat troubled analysis, we discuss the challenges and issues of conducting mobile research of this nature and reflect on caveats related to the replicability and generalizability of such work.

We examine the nature of smartwatches and explore their associated user interface design space in this paper. Several smartwatches are using small graphical displays and as such are adopting similar forms. However, there are indications that other designs could be feasible. We discuss how smartwatches might use non-graphical displays and still offer "smart" capabilities. To demonstrate feasibility, we present two smartwatch prototypes and show how LED arrays can be used to dynamically support several functions needed by smartwatch applications. Finally, we discuss some tradeoffs associated with this approach and point to additional opportunities for investigating smartwatch designs.

We introduce Corona, a novel spatial sensing technique that implicitly locates adjacent mobile devices in the same plane by examining asymmetric Bluetooth Low Energy RSSI distributions. The underlying phenomenon is that the off-center BLE antenna and asymmetric radio frequency topology create a characteristic Bluetooth RSSI distribution around the device. By comparing the real-time RSSI readings against a RSSI distribution model, each device can derive the relative position of the other adjacent device. Our experiments using an iPhone and iPad Mini show that Corona yields position estimation at 50% accuracy within a 2cm range, or 85% for the best two candidates. We developed an application to combine Corona with accelerometer readings to mitigate ambiguity and enable cross-device interactions on adjacent devices.

Loupe is a novel interactive device with a near-eye virtual display similar to head-up display glasses that retains a handheld form factor. We present our hardware implementation and discuss our user interface that leverages Loupe's unique combination of properties. In particular, we present our input capabilities, spatial metaphor, opportunities for using the round aspect of Loupe, and our use of focal depth. We demonstrate how those capabilities come together in an example application designed to allow quick access to information feeds.

While much progress has been made in wearable computing in recent years, input techniques remain a key challenge. In this paper, we introduce uTrack, a technique to convert the thumb and fingers into a 3D input system using magnetic field (MF) sensing. A user wears a pair of magnetometers on the back of their fingers and a permanent magnet affixed to the back of the thumb. By moving the thumb across the fingers, we obtain a continuous input stream that can be used for 3D pointing. Specifically, our novel algorithm calculates the magnet's 3D position and tilt angle directly from the sensor readings. We evaluated uTrack as an input device, showing an average tracking accuracy of 4.84 mm in 3D space - sufficient for subtle interaction. We also demonstrate a real-time prototype and example applications allowing users to interact with the computer using 3D finger input.

We present Facet, a multi-display wrist worn system consisting of multiple independent touch-sensitive segments joined into a bracelet. Facet automatically determines the pose of the system as a whole and of each segment individually. It further supports multi-segment touch, yielding a rich set of touch input techniques. Our work builds on these two primitives to allow the user to control how applications use segments alone and in coordination. Applications can expand to use more segments, collapses to encompass fewer, and be swapped with other segments. We also explore how the concepts from Facet could apply to other devices in this design space.

The in-vehicle experience offers a unique challenge for delivering the right amount of information to the driver at the right time. The level of attention required to successfully manage the driving task is often in variable. An ideal in vehicle information delivery system would deliver content to the driver only during low task demand times, such as waiting at a stop light, when the driver's safety would be minimally compromised. The system would also have to respond to sudden changes in the situation such as driver interruption or distraction and terminate gracefully, allowing the driver to refocus on the driving task. In this paper, we present an embedded natural language processing (NLP) system that delivers speech synthesized summarized text content into tailored time slices. The system is also designed to respond dynamically to interruptions. We anticipate that this system could safely deliver speech synthesized content to drivers and allow them to make the most of their time on the road. We have implemented this system on an Atom Z530 processor with 1GB of RAM, a processor comparable to those found in factory installed In-Vehicle Infotainment (IVI) systems and have evaluated it in a laboratory test using a standard NLP corpus to demonstrate this potential.

Collaborative spaces supporting personal mobile devices provide for a powerful integration of personalized content with supportive embedded infrastructure. Social, spatial, and informational considerations have a salient impact on such modern collaborative spaces. The design, implementation, and evaluation of a collaborative workspace prototype that directly supports the integrated use of mobile devices not only yields insights into the basic capabilities behind such a space, but also a deeper understanding of the different composition control mechanisms available. Specifically, such environments can effectively work with existing laptop platforms, and show increased promise for supporting future generations of smaller mobile devices.

With the continuing miniaturization of powerful computation into mobile devices, there exists an opportunity for re-envisioning how we interact with our personal technology.In addition to a core computational/interaction component such as a mobile phone, there could be substantial benefit to a user by offering an ensemble of multiple mobile devices that can be used together. Such devices could provide novel input or output capabilities, or distribute user interactions in a more effective way. Our goal with this proposed workshop is to foster discussion about what possibilities such collections of devices might offer.

We investigate the effect of placement and user mobility on the time required to access an on-body interface. In our study, a wrist-mounted system was significantly faster to access than a device stored in the pocket or mounted on the hip. In the latter two conditions, 78% of the time it took to access the device was spent retrieving the device from its holder. As mobile devices are beginning to include peripherals (for example, Bluetooth headsets and watches connected to a mobile phone stored in the pocket), these results may help guide interface designers with respect to distributing functions across the body between peripherals.

By analyzing features of users' typing, Automatic Whiteout++ detects and corrects up to 32.37% of the errors made by typists while using a mini-QWERTY (RIM Blackberry style) keyboard. The system targets "off-by-one" errors where the user accidentally presses a key adjacent to the one intended. Using a database of typing from longitudinal tests on two different keyboards in a variety of contexts, we show that the system generalizes well across users, model of keyboard, user expertise, and keyboard visibility conditions. Since a goal of Automatic Whiteout++ is to embed it in the firmware of mini-QWERTY keyboards, it does not rely on a dictionary. This feature enables the system to correct errors mid-word instead of applying a correction after the word has been typed. Though we do not use a dictionary, we do examine the effect of varying levels of language context in the system's ability to detect and correct erroneous keypresses.

The wristwatch is a device that is quick to access, but is currently under-utilized as a platform for interaction. We investigate interaction on a circular touchscreen wristwatch, empirically determining the error rate for variously-sized buttons placed around the rim. We consider three types of inter-target movements, and derive a mathematical model for error rate given a movement type and angular and radial button widths.

People use photographs for numerous reasons with one of the most common uses of both analog and digital photographs is as an artifact to share and discuss with others. While the practice of photo sharing has been thoroughly examined in the HCI community, there is currently very little research on easily capturing and sharing content within groups of collocated mobile users. In this paper we present the design, implementation, and evaluation of a mobile photo sharing application, Mobiphos, that gives a group of collocated users the ability to capture and simultaneously share photos in real-time with each other.

MacKenzie and Soukoreff have previously introduced a Fitts' Law-based performance model of expert two-thumb text entry on mini-QWERTY keyboards [4]. In this work we validate the original model using results from a longitudinal study of mini-QWERTY keyboards, and update the model to account for observed inter-key time data.

In this paper we present a 20-participant controlled experiment to evaluate and compare a head-down visual display and a synthesized speech audio display for comprehending text while mobile. Participants completed reading comprehension trials while walking a path and sitting. We examine overall performance and perceived workload for four conditions: audio-walking, audio-sitting, visual-walking, and visual-sitting. Results suggest audio is an acceptable modality for mobile comprehension of text. Participants' comprehension scores for the audio-walking condition were comparable to the scores for the visual-walking condition. More importantly, participants saw improvements in their ability to navigate the environment when using the audio display.

The HandyKey Twiddler™ is a one-handed chording mobile keyboard that employs a 3 × 4 button design, similar to that of a standard mobile telephone. We present a longitudinal study of novice users' learning rates on the Twiddler. Ten participants typed for 20 sessions using 2 different text entry methods. Each session was composed of 20 min of typing with multitap and 20 min of one-handed chording on the Twiddler. We found that users initially had a faster average typing rate with multitap; however, after 4 sessions the difference became negligible, and by the 8th session participants typed faster with chording on the Twiddler. Five participants continued our study and achieved an average rate of 47 words per minute (wpm) after approximately 25 hr of practice in varying conditions. One participant achieved an average rate of 67 wpm, equivalent to the typing rate of the 2nd author, who has been a Twiddler user for 10 years.We analyze the effects of learning on various aspects of chording, provide evidence that lack of visual feedback does not hinder expert typing speed, and examine the potential use of multicharacter chords (MCCs) to increase text entry speed. Finally, we explore improving novice user's experience with the Twiddler through the use of a chording tutorial.

We present a Wizard of Oz evaluation of dual--purpose speech, a technique designed to provide support during a face--to--face conversation by leveraging a user's conversational speech for input. With a dual--purpose speech interaction, the user's speech is meaningful in the context of a human--to--human conversation while providing useful input to a computer. For our experiment, we evaluate the ability to schedule appointments with our calendaring application, the Calendar Navigator Agent. We examine the relative difference between using speech for input compared to traditional pen input on a PDA. We found that speech is more direct and our participants can use their conversational speech for computer input. In doing so, we reduce the manual input needed to operate a PDA while engaged in a calendaring conversation.

An experienced user of the Twiddler, a one--handed chording keyboard, averages speeds of 60 words per minute with letter--by--letter typing of standard test phrases. This fast typing rate coupled with the Twiddler's 3x4 button design, similar to that of a standard mobile telephone, makes it a potential alternative to multi--tap for text entry on mobile phones. Despite this similarity, there is very little data on the Twiddler's performance and learnability. We present a longitudinal study of novice users' learning rates on the Twiddler. Ten participants typed for 20 sessions using two different methods. Each session is composed of 20 minutes of typing with multi--tap and 20 minutes of one--handed chording on the Twiddler. We found that users initially have a faster average typing rate with multi--tap; however, after four sessions the difference becomes negligible, and by the eighth session participants type faster with chording on the Twiddler. Furthermore, after 20 sessions typing rates for the Twiddler are still increasing.

In this paper, we explore the concept of dual-purpose speech: speech that is socially appropriate in the context of a human-to-human conversation which also provides meaningful input to a computer. We motivate the use of dual-purpose speech and explore issues of privacy and technological challenges related to mobile speech recognition. We present three applications that utilize dual-purpose speech to assist a user in conversational tasks: the Calendar Navigator Agent, DialogTabs, and Speech Courier. The Calendar Navigator Agent navigates a user's calendar based on socially appropriate speech used while scheduling appointments. DialogTabs allows a user to postpone cognitive processing of conversational material by proving short-term capture of transient information. Finally, Speech Courier allows asynchronous delivery of relevant conversational information to a third party.