Virtual Reality (VR) is a promising technology platform for immersive visual analytics. However, the design space of VR analytics interface design is vast and difficult to explore using traditional A/B comparisons in formal or informal controlled experiments— a fundamental part of an iterative design process. A key factor that complicates such comparisons is the dataset. Exposing participants to the same dataset in all conditions introduces an unavoidable learning effect. On the other hand, using different datasets for all experimental conditions introduces the dataset itself as an uncontrolled variable, which reduces internal validity to an unacceptable degree. In this paper, we propose to rectify this problem by introducing a generative process for synthesizing clustered datasets for VR analytics experiments. This process generates datasets that are distinct while simultaneously allowing systematic comparisons in experiments. A key advantage is that these datasets can then be used in iterative design processes. In a two-part experiment, we show the validity of the generative process and demonstrate how new insights in VR-based visual analytics can be gained using synthetic datasets.

In this paper, we introduce a generic Bayesian framework for inferring the intent of a tracked object, as early as possible, based on the available partial sensory observations. It treats the prediction problem, i.e. not estimating the object state such as position, within an object tracking formulation. This leads to a low-complexity implementation of the inference routine with minimal training requirements. The proposed approach utilises suitable stochastic, namely linear Gaussian, models to capture long term dependencies in the object trajectory as dictated by intent. Numerical examples are shown to demonstrate the efficacy of this framework.

This letter presents an alternative, more consistent, construction for bridging distributions, which enables inferring the destination of a tracked object from the available partial sensory observations. Two algorithms are then introduced to sequentially estimate the probability of all possible endpoints within a generic Bayesian framework. They capture the influence of intended destination on the object's motion via suitably adapted stochastic models. Whilst the bridging approach has low training requirements, the proposed formulation can lead to more efficient predictors, e.g. around 65% less computations for certain models. Synthetic and real data is used to illustrate the effectiveness of the introduced algorithms.

The objective of this paper is twofold. First, it presents a brief overview of existing driver and passenger identification or recognition approaches, which rely on smartphone data. This includes listing the typically available sensory measurements and highlighting a few key practical considerations for automotive settings. Second, a simple identification method that utilizes the smartphone inertial measurements and, possibly, doors signal is proposed. It is based on analyzing the user behavior during entry, namely, the direction of turning, and extracting relevant salient features, which are distinctive depending on the side of entry to the vehicle. This is followed by applying a suitable classifier and decision criterion. Experimental data is shown to demonstrate the usefulness and effectiveness of the introduced probabilistic, low-complexity, identification technique.

Predictive touch is an HMI technology that relies on inferring, early in the pointing gesture, the interface item a driver or passenger intends to select on an in-vehicle display [1, 2]. It simplifies and expedites the selection task, thereby reducing the associated interaction effort. This paper presents two studies on drivers using predictive touch and focuses on evaluating the best means to facilitate selecting the intended on-display item. This includes immediate midair selection with the system autonomously auto-selecting the predicted interface component, hover/dwell and drivers pressing a button on the steering wheel to execute the selection action. These were arrived at in an expert workshop study with twelve participants. The results of the subsequent evaluation study with twenty four participants demonstrate, using quantitative and qualitative measures, that immediate mid-air selection is a promising assistive scheme, where drivers need not touch a physical surface to select interface components, thus touch-free control.

This paper introduces a Bayesian framework for estimating the probability of a driver or passenger(s) returning to the vehicle, from the available partial (noisy) track of his/her location. The latter can be provided by a smartphone navigational service and/or other dedicated user to vehicle positioning solution, for instance RF-based. The proposed approach treats the addressed intent prediction problem, i.e. not tracking the object's state (e.g. the driver/passenger position, velocity, etc.) or predicting its next few values, within an object tracking formulation, leading to a Kalman-filter-based implementation of the inference routine. Hence, it is dubbed meta-tracker in lieu of a conventional “sensor-level” tracking algorithm and relies on utilising bridging distributions to encapsulate the long term dependencies in the trajectory followed by the driver or passenger as dictated by the intended endpoint, if any. Two example trajectories are shown to demonstrate the effectiveness of this flexible framework.

This paper presents formative research to inform the design of intelligent automotive user interfaces. It describes an evaluation of dialogue-based interfaces, mediating the driver to take back control from the autonomous mode of a car. Four concepts designed to increase driver Situation Awareness were evaluated in a driving simulator. They used dialogue-based interaction, where driving-related information was either asked from or repeated by the driver, with the alternative of a countdown-based interface with no additional information. An inclusive set of participants, with a wide age spectrum, tested the interfaces. The shorter and simpler interaction of the countdown timer was most accepted. The interface seeking answers to driving-related questions came next, and the interface requiring repetition of driving-related information, even when augmented by visual and tactile cues, was least accepted. Design guidelines on utilizing dialogue as a means of keeping the driver in the loop during a takeover were thus derived.

In several application areas, such as human computer interaction, surveillance and defence, determining the intent of a tracked object enables systems to aid the user/operator and facilitate effective, possibly automated, decision making. In this paper, we propose a probabilistic inference approach that permits the prediction, well in advance, of the intended destination of a tracked object and its future trajectory. Within the framework introduced here, the observed partial track of the object is modeled as being part of a Markov bridge terminating at its destination, since the target path, albeit random, must end at the intended endpoint. This captures the underlying long term dependencies in the trajectory, as dictated by the object intent. By determining the likelihood of the partial track being drawn from a particular constructed bridge, the probability of each of a number of possible destinations is evaluated. These bridges can also be employed to produce refined estimates of the latent system state (e.g., object position, velocity, etc.), predict its future values (up until reaching the designated endpoint) and estimate the time of arrival. This is shown to lead to a low complexity Kalman-filter-based implementation of the inference routine, where any linear Gaussian motion model, including the destination reverting ones, can be applied. Free hand pointing gestures data collected in an instrumented vehicle and synthetic trajectories of a vessel heading toward multiple possible harbors are utilized to demonstrate the effectiveness of the proposed approach.

This paper describes a study on modelling the Received Signal Strength Indicator (RSSI) measured by the smartphone of a vehicle user. The present transmissions are emitted by dedicated radio frequency sources, such as Bluetooth Low Energy (BLE) beacons, mounted to the vehicle to determine the driver/passenger(s) proximity or relative position(s). Based on empirical data, a model of the measurements noise, which utilises skewed distributions, is proposed to capture inconsistencies in reception and the impact of occlusions on the RSSI profile in an automotive setting, for example occlusions in car parks. Experimental data is used to demonstrate the suitability of the introduced model.

This paper proposes a probabilistic intent inference approach that is significantly more computationally efficient than other existing bridging-distributions-based predictors. It sequentially determines the probabilities of all possible destinations of a tracked object, whose motion is modelled by a Markov chain with the distribution of its terminal state equal to that of a nominal endpoint. This encapsulates the long term dependencies in the object trajectory as dictated by intent. Simulations using real data show that the notable reductions in computations achieved by the introduced bridging-based predictor does not impact the quality of the overall inference results.

In this paper, we first give an overview of the predictive display concept, which aims to minimise the demand associated with interacting with in-vehicle displays, such as touchscreens, via free hand pointing gestures. It determines the item the user intends to select, early in the pointing gesture, and accordingly simplifies-expedites the target acquisition. A study to evaluate the impact of using a predictive touchscreen in a car is then presented. The mid-air selection pointing facilitation scheme is applied, such that the user does not have to physically touch the interactive surface. Instead, the predictive display auto-selects the predicted interface icon on behalf of the user, once the required level of inference certainty is achieved. The study results, which are based on data collected from 20 participants under various driving-road conditions, demonstrate that a predictive display can significantly reduce the workload, effort and durations of completing on-screen selection tasks in vehicles.

In this paper a novel cloud-based Emergency Warning System (EWS) is described, using India as a primary case study. India is a country with nearly two dozen officially recognized languages and divergent communication technologies spread across a large geographical area. As of yet, this diversity has made the deployment of a single nationwide EWS near impossible. To address this deficiency, we have developed an EWS with adaptation as a central design tenet. This adaptation occurs on three levels: Dissemination adaptation addresses civilians' heterogeneous communication technologies; information adaptation morphs warnings to best reflect the capabilities of these communication technologies and users; while presentation adaptation is used to render information to the user in the most appropriate manner. We have developed a full cloud-based prototype, including the full EWS infrastructure and a civilian Android app.

Eye-gaze tracking is traditionally used to analyze ocular parameters for investigating visual psychology, marketing study, behavior analysis, and so on. Currently, eye-gaze trackers are also being used to control electronic interfaces in assistive technology, automobile control, and even consumer electronic products like smartphones and tablets. However, there are not many attempts to combine these two streams of research on active and passive uses of eye-gaze trackers. This article compares a few ocular parameters to estimate users’ cognitive load in eye-gaze-controlled interfaces. It was found that average velocity of a particular type of microsaccadic eye movement called Saccadic Intrusion is most indicative of users’ cognitive load compared to pupil dilation and eye-blink-based parameters. Results from the study can be used to develop new metrics of cognitive load measurement, as well as to design intelligent gaze-controlled interfaces that respond to users’ cognitive load.

Using interactive displays, such as a touchscreen, in vehicles typically requires dedicating a considerable amount of visual as well as cognitive capacity and undertaking a hand pointing gesture to select the intended item on the interface. This can act as a distractor from the primary task of driving and consequently can have serious safety implications. Due to road and driving conditions, the user input can also be highly perturbed resulting in erroneous selections compromising the system usability. In this paper, we propose intent-aware displays that utilize a pointing gesture tracker in conjunction with suitable Bayesian destination inference algorithms to determine the item the user intends to select, which can be achieved with high confidence remarkably early in the pointing gesture. This can drastically reduce the time and effort required to successfully complete an in-vehicle selection task. In the proposed probabilistic inference framework, the likelihood of all the nominal destinations is sequentially calculated by modeling the hand pointing gesture movements as a destination-reverting process. This leads to a Kalman filter-type implementation of the prediction routine that requires minimal parameter training and has low computational burden; it is also amenable to parallelization. The substantial gains obtained using an intent-aware display are demonstrated using data collected in an instrumented vehicle driven under various road conditions.

With the proliferation of the touchscreen technology, interactive displays are becoming an integrated part of the modern vehicle environment. However, due to road and driving conditions, the user input on such displays can be perturbed resulting in erroneous selections. This paper describes an evaluative study of the usability and input performance of in-vehicle touchscreens. The analysis is based on data collected in instrumented cars driven under various road/driving conditions. We assess the frequency of failed selection attempts, distances by which users miss the intended on-screen target and the durations of undertaken free hand pointing gestures to accomplish the selection tasks. It is shown that the road/driving conditions can notably undermine the usability of an interactive display when the user input is perturbed, e.g. due to the experienced vibrations and lateral accelerations in the vehicle. The distance between the location of an erroneous on-screen selection and the intended endpoint on the display, is closely related to the level of present in-vehicle noise. The conducted study can advise graphical user interfaces design for the vehicle environment where the user free hand pointing gestures can be subject to varying levels of perturbations.

We propose a novel probabilistic inference approach that permits predicting, well in advance, the intended destination of a pointing gesture aimed at selecting an icon on an in-vehicle interactive display. It models the partial 3D pointing track as a Markov bridge terminating at a nominal destination. The solution introduced leads to a low-complexity Kalman-filter-type implementation and is applicable in other areas in which early detection of the destination of a tracked object is beneficial. Data collected in an instrumented vehicle illustrate that the proposed technique can infer the intent notably early in the pointing gesture. This can drastically reduce the pointing task time and visual-cognitive-manual attention required.

This article presents a series of user studies to develop a new eye-gaze tracking–based pointing system. We developed a new target prediction model that works for different input modalities and combined the eye-gaze tracking–based pointing with a joystick controller that can reduce pointing and selection times. The system finds important applications in cockpit of combat aircraft and for computer novice users. User studies confirmed that users can perform significantly faster using this new eye-gaze tracking–based system for both military and everyday computing tasks compared to existing input devices. As part of the study it was also found that the amplitude of maximum power component obtained through Fourier Transform of pupil signal significantly correlates with selection times and perceived cognitive load of users in terms of Task Load Index scores.

Interactive displays are becoming an integrated part of the modern vehicle environment. Their use typically entails dedicating a considerable amount of attention and undertaking a pointing gesture to select an interface item/icon displayed on a touchscreen. This can have serious safety implications for the driver. The pointing gesture can also be highly perturbed due to the road and driving conditions, resulting in erroneous selections. In this paper, we propose a probabilistic intent prediction approach that facilitates establishing the targeted icon on the interface early in the pointing gesture. It employs a 3D vision sensory device to continuously track the pointing hand/finger in conjunction with suitable Bayesian prediction algorithms. The introduced technique can significantly reduce the pointing task completion time, the necessary associated visual, cognitive and movement efforts as well as enhance the selection accuracy. The substantial furnished gains and the pointing gesture characteristics are demonstrated using data collected in an instrumented vehicle.

This workshop aims to gap the bridge between mainstream research on intelligent systems and accessibility researchers by presenting papers and demonstrations on developing adaptable multimodal systems for elderly and disabled users. The workshop is organized in the context of EU GUIDE project and focus on Web and Digital TV applications. However the research and applications are relevant for different platforms like computers, tablet and ubiquitous devices. The workshop consists of a keynote speech on standardization of developing intelligent and accessible system followed by five paper and demonstration presentations. A set of papers from this workshop will later appear at the International Journal of Digital Television.

Recent research on interactive electronic systems like computer, digital TV, smartphones can improve the quality of life of many disabled and elderly people by helping them to engage more fully to the world. Previously, a simulator was developed that reflects the effect of impairment on interaction with electronic devices and thus helps designers in developing accessible systems. In this article, the scope of the simulator has been extended to multiple pointing devices. The way that hand strength affects pointing performance of people with and without mobility impairment in graphical user interfaces was investigated for four different input modalities, and a set of linear equations to predict pointing time and average number of submovements for different devices was developed. These models were used to develop an adaptation algorithm to facilitate pointing in electronic interfaces by users with motor impairment using different pointing devices. The algorithm attracts a pointer when it is near a target and thus helps to reduce random movement during homing and clicking. The algorithm was optimized using the simulator and then tested on a real-life application with multiple distractors involving three different pointing devices. The algorithm significantly reduces pointing time for different input modalities.

Elderly and disabled people can be hugely benefited through the advancement of modern electronic devices, as those can help them to engage more fully with the world. However, existing design practices often isolate elderly or disabled users by considering them as users with special needs. This article presents a simulator that can reflect problems faced by elderly and disabled users while they use computer, television, and similar electronic devices. The simulator embodies both the internal state of an application and the perceptual, cognitive, and motor processes of its user. It can help interface designers to understand, visualize, and measure the effect of impairment on interaction with an interface. Initially a brief survey of different user modeling techniques is presented, and then the existing models are classified into different categories. In the context of existing modeling approaches the work on user modeling is presented for people with a wide range of abilities. A few applications of the simulator, which shows the predictions are accurate enough to make design choices and point out the implication and limitations of the work, are also discussed.

We have investigated how hand strength affects pointing performance of people with and without mobility impair-ment in graphical user interfaces for four different input modalities. We have found that grip strength and active range of motion of wrist are most indicative of the point-ing performance. We have used the study to develop a set of linear equations to predict pointing time for different devices.

The development of TV applications nowadays excludes users with certain impairments from interacting with and accessing the same type of contents as other users do. Developers are also not interested in developing new or different versions of applications targeting different user characteristics. In this paper we describe a novel adaptive accessibility approach on how to develop accessible TV applications, without requiring too much additional effort from the developers. Integrating multimodal interaction, adaptation techniques and the use of simulators in the design process, we show how to adapt User Interfaces to the individual needs and limitations of elderly users. For this, we rely on the identification of the most relevant impairment configurations among users in practical user-trials, and we draw a relation with user specific characteristics. We provide guidelines for more accessible and centered TV application development.

Designers in inclusive design are challenged to create interactive products that cater for a wide range of prior experiences and cognitive abilities of their users. But suitable design guidance for this task is rare. This paper proposes the theory of primary metaphor and explores its validity as a source of design guidance. Primary metaphor theory describes how basic mental representations of physical sensorimotor experiences are extended to understand abstract domains. As primary metaphors are subconscious mental representations that are highly automated, they should be robustly available to people with differing levels of cognitive ability. Their proposed universality should make them accessible to people with differing levels of prior experience with technology. These predictions were tested for 12 primary metaphors that predict relations between spatial gestures and abstract interactive content. In an empirical study, 65 participants from two age groups (young and old) were asked to produce two-dimensional touch and three-dimensional free-form gestures in response to given abstract keywords and spatial dimensions of movements. The results show that across age groups in 92% of all cases users choose gestures that confirmed the predictions of the theory. Although the two age groups differed in their cognitive abilities and prior experience with technology, overall they did not differ in the amount of metaphor-congruent gestures they made. As predicted, only small or zero correlations of metaphor-congruent gestures with prior experience or cognitive ability could be found. The results provide a promising step toward inclusive design guidelines for gesture interaction with abstract content on mobile multitouch devices.

Journal Article Developing a model of cognitive interaction for analytical inclusive design evaluation Get access Patrick Langdon, Patrick Langdon * a Engineering Design Centre, Cambridge University Engineering Department, Trumpington Street, Cambridge CB2 1PZ, United Kingdom * Corresponding author. Tel.: +44 1223 766959, mobile: +44 7767 880815; fax: +44 1223 332662. E-mail addresses:[email protected], [email protected] (P. Langdon), [email protected] (U. Persad) Search for other works by this author on: Oxford Academic Google Scholar Umesh Persad, Umesh Persad b Centre for Production Systems, University of Trinidad and Tobago, Trinidad and Tobago Search for other works by this author on: Oxford Academic Google Scholar P. John Clarkson P. John Clarkson a Engineering Design Centre, Cambridge University Engineering Department, Trumpington Street, Cambridge CB2 1PZ, United Kingdom Search for other works by this author on: Oxford Academic Google Scholar Interacting with Computers, Volume 22, Issue 6, November 2010, Pages 510–529, https://doi.org/10.1016/j.intcom.2010.08.008 Published: 16 September 2010

The field of inclusive design relates the capabilities of the population to the design of products by better characterising the user–product relationship. Inclusion refers to the quantitative relationship between the demand made by design features and the capability ranges of users who may be excluded from use of the product because of those features. By 2020, almost half the adult population in the UK will be over 50, with the over 80s being the most rapidly growing sector. These “inclusive” populations contain a great variation in sensory, cognitive and physical user capabilities, particularly when non-age-related impairments are taken into account. While this sector of the user population is growing, the most productive research and design sector is inevitably shrinking in proportion to that growth. In user interface design we develop interactive products and we are used to designing “for the user,” but often the “user” is rarely precisely specified demographically, and rarely described in a way that represents the variance in abilities. At worst, we may think that the users are a bit like us, but maybe do not know so much; and the environment where designers work is very different from the environment where users live. Users are often under contextual pressures that designers cannot relate to except through proper usability studies. Consider entering a hard-to-remember numerical code into a noisy burglar alarm to silence it in the 10 s allowed, working in a dark cupboard with a small hard-to-read LCD display. Unfortunately, most usability studies in the literature come from university laboratories and use students as experimental participants, which is convenient but ignores the fact that almost all are under 30! The problem in interaction design mirrors the recent observation that painkiller research tends to be done on young men and almost never on pregnant women; thus painkillers are developed, evaluated and refined on population sectors that are not representative of all users, and hence for perfectly understandable reasons they are becoming less suitable for some sectors of the population. But one needs to quantify these issues carefully and find suitable ways to address the identified problems.

This paper describes a software tool that demonstrates the effects of common vision and hearing impairments on image and sound files. This helps designers to understand and empathize with the difficulties that less able users might experience when using their products and interfaces. A range of impairments and severity levels are simulated and guidance is provided on how to address some of the issues raised. The latest version of this tool will be available for demonstration at Assets.

Understanding human movement is key to improving input devices and interaction techniques. This paper presents a study of mouse movements of motion-impaired users, with an aim to gaining a better understanding of impaired movement. The cursor trajectories of six motion-impaired users and three able-bodied users are studied according to their submovement structure. Performance measures based on submovement structure are described, including the frequency and duration of pauses between submovements, verification times, the number of submovements, the peak speed of submovements and the accuracy of submovements in two dimensions. The measures are shown to be sensitive to differences between users with dissimilar physical capabilities. Results include findings that some motion-impaired users pause more often and for longer than able-bodied users, require up to five times more submovements to complete the same task, and exhibit a greater decline in accuracy with increasing speed than able-bodied users.

Understanding human movement is key to improving input devices and interaction techniques. This paper presents a study of mouse movements of motion-impaired users, with an aim to gaining a better understanding of impaired movement. The cursor trajectories of six motion-impaired users and three able-bodied users are studied according to their submovement structure. Several aspects of the movement are studied, including the frequency and duration of pauses between submovements, verification times, the number of submovements, the peak speed of submovements and the accuracy of submovements in two-dimensions. Results include findings that some motion-impaired users pause more often and for longer than able-bodied users, require up to five times more submovements to complete the same task, and exhibit a correlation between error and peak submovement speed that does not exist for able-bodied users.

Although a number of studies have reported that force feedback gravity wells can improve performance in "point-and-click" tasks, there have been few studies addressing issues surrounding the use of gravity wells for multiple on-screen targets. This paper investigates the performance of users, both with and without motion-impairments, in a "point-and-click" task when an undesired haptic distractor is present. The importance of distractor location is studied explicitly. Results showed that gravity wells can still improve times and error rates, even on occasions when the cursor is pulled into a distractor. The greatest improvement is seen for the most impaired users. In addition to traditional measures such as time and errors, performance is studied in terms of measures of cursor movement along a path. Two cursor measures, angular distribution and temporal components, are proposed and their ability to explain performance differences is explored.

"Point and click" interactions remain one of the key features of graphical user interfaces (GUIs). People with motion-impairments, however, can often have difficulty with accurate control of standard pointing devices. This paper discusses work that aims to reveal the nature of these difficulties through analyses that consider the cursor's path of movement. A range of potential cursor measures was applied, and a number of them were found to be significant in capturing the differences between able-bodied users and motion-impaired users, as well as the differences between a haptic force feedback condition and a control condition. cursor measures found in the literature, however, do not make up a comprehensive list, but provide a starting point for analysing cursor movements more completely. Six new cursor characteristics for motion-impaired users are introduced to capture aspects of cursor movement different from those already proposed.