We propose a zero-shot unsupervised image segmentation method by utilising self-attention activations extracted from Stable Diffusion. We demonstrate that self-attention can directly be interpreted as transition probabilities in a Markov random walk between image patches. This property enables us to modulate multi-hop relationships through matrix exponentiation, which captures k-step transitions between patches. We then construct a graph representation based on self-attention feature similarity and apply Normalised Cuts to cluster them. We quantitatively analyse the effects of incorporating multi-node paths when constructing the NCuts adjacency matrix, showing that higher-order transitions enhance hierarchical relationships in the proposed segmentations. Finally, we describe an approach to automatically determine the NCut threshold criterion, avoiding the need to manually tune it. Our approach surpasses all existing methods for zero-shot unsupervised segmentation based on pretrained diffusion models features, achieving state-of-the-art results on COCO-Stuff-27, Cityscapes and ADE20K.

Immersive realities enable social interactions that are radically different from traditional communication technologies, but how we experience immersion together is not yet indistinguishable from face-to-face interactions. Some social signals are not stable across realities, may change in semantics, or are missing all together. Understanding how social signals impact behaviours and experiences of social connection in immersive environments is key to creating experiences that are meaningful, satisfying, and productive. We completed a lab study where 6 groups of 6 participants (N=36) completed co-located social tasks in an instrumented face-to-face environment and its digital twin, creating a rich open dataset of 1.8 million rows across 45 columns. Our quantitative results demonstrate the stability of position as a social signal, measure lower social synchronisation in XR compared to face-to-face, and propose a method for bench marking XR against face-to-face interactions. This enables direct quantitative comparisons between experiences of co-located physical and virtual interactions for the first time.

Accurately detecting and classifying damage in analogue media such as paintings, photographs, textiles, mosaics, and frescoes is essential for cultural heritage preservation. While machine learning models excel in correcting degradation if the damage operator is known a priori, we show that they fail to robustly predict where the damage is even after supervised training; thus, reliable damage detection remains a challenge. Motivated by this, we introduce ARTeFACT, a dataset for damage detection in diverse types analogue media, with over 11,000 annotations covering 15 kinds of damage across various subjects, media, and historical provenance. Furthermore, we contribute human-verified text prompts describing the semantic contents of the images, and derive additional textual descriptions of the annotated damage. We evaluate CNN, transformer, diffusion-based segmentation models, and foundation vision models in zero-shot, supervised, unsupervised and text-guided settings, revealing their limitations in generalising across media types. Our dataset is available at https://daniela997.github.ioIARTeFACTI as the first-of-its-kind benchmark for analogue media damage detection and restoration.

Camera-equipped smartphones pose security risks to organisations by allowing intentional or accidental leaks of confidential on-screen information. We introduce SafeScreen, an Android camera app that detects and obfuscates screen content in real-time using deep-learning recognition for distant screens and Moiré pattern detection for close-up screen captures. Our mixed-methods, ecologically focused study compared “benign” (ordinary photography) and “malign” (circumventing detection) uses. Results show SafeScreen effectively prevents accidental leaks, but that the majority of users were able to exploit it by discovering workarounds such as partial screen occlusion. Our work contributes (1) a novel screen-blocking camera system, and (2) insights from real-world, unguided interactions. We show how evaluating security systems in authentic settings uncovers user-driven vulnerabilities and frustrations that inform future researchers and organisations. We close by discussing future technical features which could offer usability or security improvements, as well as emphasising the benefits of unscripted and adversarial user evaluations.

Active Inference is a closed-loop computational theoretical basis for understanding behaviour, based on agents with internal probabilistic generative models that encode their beliefs about how hidden states in their environment cause their sensations. We review Active Inference and how it could be applied to model the human–computer interaction loop. Active Inference provides a coherent framework for managing generative models of humans, their environments, sensors and interface components. It informs off-line design and supports real-time, online adaptation. It provides model-based explanations for behaviours observed in HCI, and new conceptual tools with the potential to measure important concepts such as agency and engagement in interaction. We discuss how Active Inference offers a new basis for a theory of interaction in HCI, tools for design of modern, complex sensor-based systems, and integration of artificial intelligence technologies, enabling it to cope with diversity in human users and contexts. We discuss the practical challenges in implementing such Active Inference-based systems.

Despite growing interest in probabilistic modeling approaches and availability of learning tools, people are hesitant to use them. There is a need for tools to communicate probabilistic models more intuitively and help users build, validate, use effectively or trust probabilistic models. We focus on visual representations of probabilistic models and introduce the Interactive Pair Plot (IPP) for visualization of a model's uncertainty, a scatter plot matrix of a probabilistic model allowing interactive conditioning on the model's variables. We investigate whether the use of interactive conditioning in a scatter plot matrix of a model helps users better understand variables' relations. We conducted a user study and the findings suggest that improvements in the understanding of the interaction group are the most pronounced for more exotic structures, such as hierarchical models or unfamiliar parameterizations, in comparison to the understanding of the static group. As the detail of the inferred information increases, interactive conditioning does not lead to considerably longer response times. Finally, interactive conditioning improves participants' confidence about their responses.

Abstract Digital scans of analogue photographic film typically contain artefacts such as dust and scratches. Automated removal of these is an important part of preservation and dissemination of photographs of historical and cultural importance. While state‐of‐the‐art deep learning models have shown impressive results in general image inpainting and denoising, film artefact removal is an understudied problem. It has particularly challenging requirements, due to the complex nature of analogue damage, the high resolution of film scans, and potential ambiguities in the restoration. There are no publicly available high‐quality datasets of real‐world analogue film damage for training and evaluation, making quantitative studies impossible. We address the lack of ground‐truth data for evaluation by collecting a dataset of 4K damaged analogue film scans paired with manually‐restored versions produced by a human expert, allowing quantitative evaluation of restoration performance. We have made the dataset available at https://doi.org/10.6084/m9.figshare.21803304. We construct a larger synthetic dataset of damaged images with paired clean versions using a statistical model of artefact shape and occurrence learnt from real, heavily‐damaged images. We carefully validate the realism of the simulated damage via a human perceptual study, showing that even expert users find our synthetic damage indistinguishable from real. In addition, we demonstrate that training with our synthetically damaged dataset leads to improved artefact segmentation performance when compared to previously proposed synthetic analogue damage overlays. The synthetically damaged dataset can be found at https://doi.org/10.6084/m9.figshare.21815844, and the annotated authentic artefacts along with the resulting statistical damage model at https://github.com/daniela997/FilmDamageSimulator. Finally, we use these datasets to train and analyse the performance of eight state‐of‐the‐art image restoration methods on high‐resolution scans. We compare both methods which directly perform the restoration task on scans with artefacts, and methods which require a damage mask to be provided for the inpainting of artefacts. We modify the methods to process the inputs in a patch‐wise fashion to operate on original high resolution film scans.

Thermal cameras can reveal heat traces on user interfaces, such as keyboards. This can be exploited maliciously to infer sensitive input, such as passwords. While previous work considered thermal attacks that rely on visual inspection of simple image processing techniques, we show that attackers can perform more effective artificial intelligence (AI)–driven attacks. We demonstrate this by presenting the development of ThermoSecure and its evaluation in two user studies (N = 21, N = 16), which reveal novel insights about thermal attacks. We detail the implementation of ThermoSecure and make a dataset of 1,500 thermal images of keyboards with heat traces resulting from input publicly available. Our first study shows that ThermoSecure successfully attacks 6-symbol, 8-symbol, 12-symbol, and 16-symbol passwords with an average accuracy of 92%, 80%, 71%, and 55% respectively, and even higher accuracy when thermal images are taken within 30 seconds. We found that typing behavior significantly impacts vulnerability to thermal attacks: hunt-and-peck typists are more vulnerable than fast typists (92% vs. 83% thermal attack success. respectively, if performed within 30 seconds). The second study showed that keycap material has a statistically significant effect on the effectiveness of thermal attacks: ABS keycaps retain the thermal trace of user presses for a longer period of time, making them more vulnerable to thermal attacks, with a 52% average attack accuracy compared with 14% for keyboards with PBT keycaps. Finally, we discuss how systems can leverage our results to protect from thermal attacks and present 7 mitigation approaches that are based on our results and previous work.

Electrotactile stimulation is a novel form of haptic feedback. There is little work investigating its basic design parameters and how they create effective tactile cues. This paper describes two experiments that extend our knowledge of two key parameters. The first investigated the combination of pulse width and amplitude (Intensity) on sensations of urgency, annoyance, valence and arousal. Results showed significant effects: increasing Intensity caused higher ratings of urgency, annoyance and arousal but reduced valence. We established clear levels for differentiating each sensation. A second study then investigated Intensity and Pulse Frequency to find out how many distinguishable levels could be perceived. Results showed that both Intensity and Pulse Frequency significantly affected perception, with four distinguishable levels of Intensity and two of Pulse Frequency. These results add significant new knowledge about the parameter space of electrotactile cue design and help designers select suitable properties to use when creating electrotactile cues.

Behaviour in virtual environments might be informed by our experiences in physical environments, but virtual environments are not constrained by the same physical, perceptual, or social cues. Instead of replicating the properties of physical spaces, one can create virtual experiences that diverge from reality by dynamically manipulating environmental, aural, and social properties. This paper explores digital proxemics, which describe how we use space in virtual environments and how the presence of others influences our behaviours, interactions, and movements. First, we frame the open challenges of digital proxemics in terms of activity, social signals, audio design, and environment. We explore a subset of these challenges through an evaluation that compares two audio designs and two displays with different social signal affordances: head-mounted display (HMD) versus desktop PC. We use quantitative methods using instrumented tracking to analyse behaviour, demonstrating how personal space, proximity, and attention compare between desktop PC and HMDs.

MOTIVATION: Identification of small molecules in a biological sample remains a major bottleneck in molecular biology, despite a decade of rapid development of computational approaches for predicting molecular structures using mass spectrometry (MS) data. Recently, there has been increasing interest in utilizing other information sources, such as liquid chromatography (LC) retention time (RT), to improve identifications solely based on MS information, such as precursor mass-per-charge and tandem mass spectrometry (MS2). RESULTS: We put forward a probabilistic modelling framework to integrate MS and RT data of multiple features in an LC-MS experiment. We model the MS measurements and all pairwise retention order information as a Markov random field and use efficient approximate inference for scoring and ranking potential molecular structures. Our experiments show improved identification accuracy by combining MS2 data and retention orders using our approach, thereby outperforming state-of-the-art methods. Furthermore, we demonstrate the benefit of our model when only a subset of LC-MS features has MS2 measurements available besides MS1. AVAILABILITY AND IMPLEMENTATION: Software and data are freely available at https://github.com/aalto-ics-kepaco/msms_rt_score_integration. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

There is a growing need for usable and secure authentication in immersive virtual reality (VR). Established concepts (e.g., 2D authentication schemes) are vulnerable to observation attacks, and most alternatives are relatively slow. We present RubikAuth, an authentication scheme for VR where users authenticate quickly and secure by selecting digits from a virtual 3D cube that leverages coordinated 3D manipulation and pointing. We report on results from three studies comparing how pointing using eye gaze, head pose, and controller tapping impact RubikAuth’s usability, memorability, and observation resistance under three realistic threat models. We found that entering a four-symbol RubikAuth password is fast: 1.69–3.5 s using controller tapping, 2.35–4.68 s using head pose and 2.39 –4.92 s using eye gaze, and highly resilient to observations: 96–99.55% of observation attacks were unsuccessful. RubikAuth also has a large theoretical password space: 45 n for an n -symbols password. Our work underlines the importance of considering novel but realistic threat models beyond standard one-time attacks to fully assess the observation-resistance of authentication schemes. We conclude with an in-depth discussion of authentication systems for VR and outline five learned lessons for designing and evaluating authentication schemes.

The use of ultrasound haptic feedback for mid-air gestures in cars has been proposed to provide a sense of control over the user's intended actions and to add touch to a touchless interaction. However, the impact of ultrasound feedback to the gesturing hand regarding lane deviation, eyes-off-the-road time (EORT) and perceived mental demand has not yet been measured. This paper investigates the impact of uni- and multimodal presentation of ultrasound feedback on the primary driving task and the secondary gesturing task in a simulated driving environment. The multimodal combinations of ultrasound included visual, auditory, and peripheral lights. We found that ultrasound feedback presented unimodally and bimodally resulted in significantly less EORT compared to visual feedback. Our results suggest that multimodal ultrasound feedback for mid-air interaction decreases EORT whilst not compromising driving performance nor mental demand and thus can increase safety while driving.

This paper presents an investigation into the effects of different feedback modalities on mid-air gesture interaction for infotainment systems in cars. Car crashes and near-crash events are most commonly caused by driver distraction. Mid-air interaction is a way of reducing driver distraction by reducing visual demand from infotainment. Despite a range of available modalities, feedback in mid-air gesture systems is generally provided through visual displays. We conducted a simulated driving study to investigate how different types of multimodal feedback can support in-air gestures. The effects of different feedback modalities on eye gaze behaviour, and the driving and gesturing tasks are considered. We found that feedback modality influenced gesturing behaviour. However, drivers corrected falsely executed gestures more often in non-visual conditions. Our findings show that non-visual feedback can reduce visual distraction significantly

Public evaluations are popular because some research questions can only be answered by turning "to the wild." Different approaches place experimenters in different roles during deployment, which has implications for the kinds of data that can be collected and the potential bias introduced by the experimenter. This paper expands our understanding of how experimenter roles impact public evaluations and provides an empirical basis to consider different evaluation approaches. We completed an evaluation of a playful gesture-controlled display not to understand interaction at the display but to compare different evaluation approaches. The conditions placed the experimenter in three roles, steward observer, overt observer, and covert observer, to measure the effect of experimenter presence and analyse the strengths and weaknesses of each approach.

This demonstration showcases novel multimodal feedback designs for in-car mid-air gesture interaction. It explores the potential of multimodal feedback types for mid-air gestures in cars and how these can reduce eyes-off-the-road time thus make driving safer. We will show four different bimodal feedback combinations to provide effective information about interaction with systems in a car. These feedback techniques are visual-auditory, auditory-ambient (peripheral vision), ambient-tactile, and tactile-auditory. Users can interact with the system after a short introduction, creating an exciting opportunity to deploy these displays in cars in the future.

The association of tabletop interaction with gesture typing presents interaction potential for situationally or physically impaired users. In this work, we use depth cameras to create touch surfaces on regular tabletops. We describe our prototype system and report on a supervised learning approach to fingertips touch classification. We follow with a gesture typing study that compares our system with a control tablet scenario and explore the influence of input size and aspect ratio of the virtual surface on the text input performance. We show that novice users perform with the same error rate at half the input rate with our system as compared to the control condition, that an input size between A5 and A4 present the best tradeoff between performance and user preference and that users' indirect tracking ability seems to be the overall performance limiting factor.

Infotainment Systems can increase mental workload and divert visual attention away from looking ahead on the roads. When these systems give information to the driver, providing it through the tactile channel on the steering wheel might improve driving behaviour and safety. This paper describes an investigation into the perceivability of haptic feedback patterns using an actuated surface on a steering wheel. Six solenoids were embedded along the rim creating three bumps under each palm. A simulated driving study was conducted to test for recognition accuracy of the haptic patterns (81.3%). There was no significant increase in lane deviation or steering angle during haptic pattern presentation. These results suggest that drivers can reliably distinguish between cutaneous patterns presented on the steering wheel. Our findings can assist in delivering non-critical messages to the driver (e.g. driving performance, incoming text messages, etc.) without decreasing driving performance or increasing perceived mental workload.

We show that when users make errors on mobile devices they make immediate and distinct physical responses that can be observed with standard sensors. We used three standard cognitive tasks (Flanker, Stroop and SART) to induce errors from 20 participants. Using simple low-resolution capacitive touch sensors placed around a standard mobile device and the built-in accelerometer, we demonstrate that errors can be predicted at low error rates from micro-adjustments to hand grip and movement in the period shortly after swiping the touchscreen. Specifically, when combining features derived from hand grip and movement we obtain a mean AUC of 0.96 (with false accept and reject rates both below 10%). Our results demonstrate that hand grip and movement provide strong and low latency evidence for mistakes. The ability to detect user errors in this way could be a valuable component in future interaction systems, allowing interfaces to make it easier for users to correct erroneous inputs.

When using non-contact finger tracking, fingers can be classified as to which hand they belong to by analysing the phase relation of physiological tremor. In this paper, we show how 3D capacitive sensors can pick up muscle tremor in fingers above a device. We develop a signal processing pipeline based on nonlinear phase synchronisation that can reliably group fingers to hands and experimentally validate our technique. This allows significant new gestural capabilities for 3D finger sensing without additional hardware

In this paper, we investigate the effects of mobility and encumbrance (holding objects such as shopping bags) on standard gestures commonly performed on touchscreens: tapping, dragging, spreading & pinching and rotating clockwise & anticlockwise when completed using a two-handed input posture. These one- and two- finger on-screen gesture inputs have become common but previous research has only examined tapping performance in everyday walking and encumbered situations. Therefore, a series of Fitts' Law style targeting tasks was designed to measure the performance of each gesture with users walking only and walking while carrying bags. The results showed that encumbrance and walking had a negative impact on each gesture in terms of accuracy except for rotational actions, which were performed well. Tapping and dragging both performed poorly which shows the input difficulties of single finger interactions when encumbered and on the move. Our findings will help designers choose the appropriate input techniques for future mobile user interfaces and apps in physically demanding contexts.

In this paper, we investigate the effects of encumbrance (carrying typical objects such as shopping bags during interaction) and walking on target acquisition on a touchscreen mobile phone. Users often hold objects and use mobile devices at the same time and we examined the impact encumbrance has on one- and two- handed interactions. Three common input postures were evaluated: two-handed index finger, one-handed preferred thumb and two-handed both thumbs, to assess the effects on performance of carrying a bag in each hand while walking. The results showed a significant decrease in targeting performance when users were encumbered. For example, input accuracy dropped to 48.1% for targeting with the index finger when encumbered, while targeting error using the preferred thumb to input was 4.2mm, an increase of 40% compared to unencumbered input. We also introduce a new method to evaluate the user's preferred walking speed when interacting - PWS&I, and suggest future studies should use this to get a more accurate measure of the user's input performance.

We demonstrate that front-of-screen targeting on mobile phones can be predicted from back-of-device grip manipulations. Using simple, low-resolution capacitive touch sensors placed around a standard phone, we outline a machine learning approach to modelling the grip modulation and inferring front-of-screen touch targets. We experimentally demonstrate that grip is a remarkably good predictor of touch, and we can predict touch position 200ms before contact with an accuracy of 18mm.

In this paper, two walking evaluation methods were compared to evaluate the effects of encumbrance while the preferred walking speed (PWS) is controlled. Users frequently carry cumbersome objects (e.g. shopping bags) and use mobile devices at the same time which can cause interaction difficulties and erroneous input. The two methods used to control the PWS were: walking on a treadmill and walking around a predefined route on the ground while following a pacesetter. The results from our target acquisition experiment showed that for ground walking at 100% of PWS, accuracy dropped to 36% when carrying a bag in the dominant hand while accuracy reduced to 34% for holding a box under the dominant arm. We also discuss the advantages and limitations of each evaluation method when examining encumbrance and suggest treadmill walking is not the most suitable approach to use if walking speed is an important factor in future mobile studies.

Rotations performed with the index finger and thumb involve some of the most complex motor action among common multi-touch gestures, yet little is known about the factors affecting performance and ergonomics. This note presents results from a study where the angle, direction, diameter, and position of rotations were systematically manipulated. Subjects were asked to perform the rotations as quickly as possible without losing contact with the display, and were allowed to skip rotations that were too uncomfortable. The data show surprising interaction effects among the variables, and help us identify whole categories of rotations that are slow and cumbersome for users.

OBJECTIVE: The array of available brain-computer interface (BCI) paradigms has continued to grow, and so has the corresponding set of machine learning methods which are at the core of BCI systems. The latter have evolved to provide more robust data analysis solutions, and as a consequence the proportion of healthy BCI users who can use a BCI successfully is growing. With this development the chances have increased that the needs and abilities of specific patients, the end-users, can be covered by an existing BCI approach. However, most end-users who have experienced the use of a BCI system at all have encountered a single paradigm only. This paradigm is typically the one that is being tested in the study that the end-user happens to be enrolled in, along with other end-users. Though this corresponds to the preferred study arrangement for basic research, it does not ensure that the end-user experiences a working BCI. In this study, a different approach was taken; that of a user-centered design. It is the prevailing process in traditional assistive technology. Given an individual user with a particular clinical profile, several available BCI approaches are tested and - if necessary - adapted to him/her until a suitable BCI system is found. METHODS: Described is the case of a 48-year-old woman who suffered from an ischemic brain stem stroke, leading to a severe motor- and communication deficit. She was enrolled in studies with two different BCI systems before a suitable system was found. The first was an auditory event-related potential (ERP) paradigm and the second a visual ERP paradigm, both of which are established in literature. RESULTS: The auditory paradigm did not work successfully, despite favorable preconditions. The visual paradigm worked flawlessly, as found over several sessions. This discrepancy in performance can possibly be explained by the user's clinical deficit in several key neuropsychological indicators, such as attention and working memory. While the auditory paradigm relies on both categories, the visual paradigm could be used with lower cognitive workload. Besides attention and working memory, several other neurophysiological and -psychological indicators - and the role they play in the BCIs at hand - are discussed. CONCLUSION: The user's performance on the first BCI paradigm would typically have excluded her from further ERP-based BCI studies. However, this study clearly shows that, with the numerous paradigms now at our disposal, the pursuit for a functioning BCI system should not be stopped after an initial failed attempt.

This paper presents a systematic and general technique for establishing a set of motions suitable for use with sensor systems, by drawing performable and measurable motions directly from users. It uses reinforcement which rewards originality to induce users to explore the space of motions they can perform. A decomposition of movements into motion primitives is constructed, among which a meaningful originality metric can be defined. Because the originality measure is defined in terms of the sensed input, the resulting space contains only movements which can both be performed and sensed. We show how this can be used to evaluate the relative performance of different joint user-sensor systems, providing objective analyses of gesture lexicons with regard to the technical limitations of sensors and humans. In particular, we show how the space of motions varies across the arm for a body-mounted inertial sensor.

A tablet interface for manipulating microscopic particles is augmented with vibrotactile and audio feedback. The feedback is generated using a novel real-time synthesis library based on approximations to physical processes, and is efficient enough to run on mobile devices, despite their limited computational power. The feedback design and usability testing was done with a realistic simulator on appropriate tasks, allowing users to control objects more rapidly, with fewer errors and applying more consistent forces. The feedback makes the interaction more tangible, giving the user more awareness of changes in the characteristics of the optical tweezers as the number of optical traps changes.

We describe an approach to improving the design and development of Brain-Computer Interface (BCI) applications by simulating the error-prone characteristics and subjective feel of electroencephalogram (EEG), motor-imagery based BCIs. BCIs have the potential to enhance the quality of life of people who are severely disabled, but it is often time-consuming to test and develop the systems. Simulation of BCI characteristics allows developers to rapidly test design options, and gain both subjective and quantitative insight into expected behaviour without using an EEG cap. A further motivation for the use of simulation is that 'impairing' a person without motor disabilities in a game with a disabled BCI user can create a level playing field and help carers empathise with BCI users. We demonstrate a use of the simulator in controlling a game of Brain Pong.

We present a finger-tracking system for touch-based interaction which can track 3D finger angle in addition to position, using low-resolution conventional capacitive sensors, therefore compensating for the inaccuracy due to pose variation in conventional touch systems. Probabilistic inference about the pose of the finger is carried out in real-time using a particle filter; this results in an efficient and robust pose estimator which also gives appropriate uncertainty estimates. We show empirically that tracking the full pose of the finger results in greater accuracy in pointing tasks with small targets than competitive techniques. Our model can detect and cope with different finger sizes and the use of either fingers or thumbs, bringing a significant potential for improvement in one-handed interaction with touch devices. In addition to the gain in accuracy we also give examples of how this technique could open up the space of novel interactions.

We describe a particle filtering approach to inferring finger movements on capacitive sensing arrays. This technique allows the efficient combination of human movement models with accurate sensing models, and gives high-fidelity results with low-resolution sensor grids and tracks finger height. Our model provides uncertainty estimates, which can be linked to the interaction to provide appropriately smoothed responses as sensing perfomance degrades; system autonomy is increased as estimates of user behaviour become less certain. We demonstrate the particle filter approach with a map browser running with a very small sensor board, where finger position uncertainty is linked to autonomy handover.

We describe a virtual "tether" for mobile devices that allows groups to have quick, simple and privacy-preserving meetups. Our design provides cues which allow dynamic coordination of rendezvous without revealing users' positions. Using accelerometers and magnetometers, combined with GPS positioning and non-visual feedback, users can probe and sense a dynamic virtual object representing the nearest meeting point. The Social Gravity system makes social bonds tangible in a virtual world which is geographically grounded, using haptic feedback to help users rendezvous. We show dynamic navigation using this physical model-based system to be efficient and robust in significant field trials, even in the presence of low-quality positioning. The use of simulators to build models of mobile geolocated systems for pre-validation purposes is discussed, and results compared with those from our trials. Our results show interesting behaviours in the social coordination task, which lead to guidelines for geosocial interaction design. The Social Gravity system proved to be very successful in allowing groups to rendezvous efficiently and simply and can be implemented using only commercially available hardware.

Large collections of photographs are commonplace, and many interfaces for viewing, sorting and organizing them have been proposed. This work describes the design and implementation of a "living photo frame" - designed not to navigate or browse collections but to create an enjoyable activity from a collection of images. Tangible interactions with a tablet-style PC are used to bind the user closely to the system. Every interaction is logged and used to gradually evolve the structure of photo collections.

Stane is a hand-held interaction device controlled by tactile input: scratching or rubbing textured surfaces and tapping. The system has a range of sensors, including contact microphones, capacitive sensing and inertial sensing, and provides audio and vibrotactile feedback. The surface textures vary around the device, providing perceivably different textures to the user. We demonstrate that the vibration signals generated by stroking and scratching these surfaces can be reliably classified, and can be used as a very cheaply manufacturable way to control different aspects of interaction. The system is demonstrated as a control for a music player.

In this paper, we investigate wrist rotation as a hands-free method of interaction with a mobile device. To evaluate this technique, a Fitts' Law targeting study is described in four different postures: resting, seated, standing and walking. Results show correlations in movement time and the Index of Difficulty of the task and similarities in the targeting performance for the first three conditions, but show walking and targeting using this method was significantly more difficult.

We demonstrate the use of uncertain prediction in asystem for pedestrian navigation via audio with a combination ofGlobal Positioning System data, a music player, inertial sensing,magnetic bearing data and Monte Carlo sampling for a densityfollowing task, where a listener's music is modulated according tothe changing predictions of user position with respect to a targetdensity, in this case a trajectory or path. We show that this system enables eyes-free navigation around set trajectories or paths unfamiliar to the user and demonstrate that the system may be used effectively for varying trajectory width and context.

Shoogle is a novel, intuitive interface for sensing data withina mobile device, such as presence and properties of textmessages or remaining resources. It is based around activeexploration: devices are shaken, revealing the contents rattlingaround "inside". Vibrotactile display and realistic impactsonification create a compelling system. Inertial sensingis used for completely eyes-free, single-handed interactionthat is entirely natural. Prototypes are described runningboth on a PDA and on a mobile phone with a wireless sensorpack. Scenarios of use are explored where active sensing ismore appropriate than the dominant alert paradigm.

We present a mobile, GPS-based multimodal navigation system, equipped with inertial control that allows users to explore and navigate through an augmented physical space, incorporating and displaying the uncertainty resulting from inaccurate sensing and unknown user intentions. The system propagates uncertainty appropriately via Monte Carlo sampling and predicts at a user-controllable time horizon. Control of the Monte Carlo exploration is entirely tilt-based. The system output is displayed both visually and in audio. Audio is rendered via granular synthesis to accurately display the probability of the user reaching targets in the space. We also demonstrate the use of uncertain prediction in a trajectory following task, where a section of music is modulated according to the changing predictions of user position with respect to the target trajectory. We show that appropriate display of the full distribution of potential future users positions with respect to sites-of-interest can improve the quality of interaction over a simplistic interpretation of the sensed data.

The paper presents a robot cell for multimodal standing-up motion augmentation. The robot cell is aimed at augmenting the standing-up capabilities of impaired or paraplegic subjects. The setup incorporates the rehabilitation robot device, functional electrical stimulation system, measurement instrumentation and cognitive feedback system. For controlling the standing-up process a novel approach was developed integrating the voluntary activity of a person in the control scheme of the rehabilitation robot. The simulation results demonstrate the possibility of “patient-driven” robot-assisted standing-up training. Moreover, to extend the system capabilities, the audio cognitive feedback is aimed to guide the subject throughout rising. For the feedback generation a granular synthesis method is utilized displaying high-dimensional, dynamic data. The principle of operation and example sonification in standing-up are presented. In this manner, by integrating the cognitive feedback and “patient-driven” actuation systems, an effective motion augmentation system is proposed in which the motion coordination is under the voluntary control of the user.

This work introduces the idea of haptic rendering using granular synthesis - an established technique for synthesising audio. It describes the technique along with potential application areas, and initial results from an implementation on a PHANToM force feedback device. Three main applications are considered. Firstly, rendering of probabilistic vector fields for presenting ambiguity and context information to the user. Secondly, the possibility of producing textured virtual objects using granular synthesis is discussed. Thirdly, we use the approach to display scatterplot data on haptic devices.