Fatigue, sleep, or disturbances in daily activities are common symptoms among patients with neurodegenerative disorders (NDD) and immune-mediated inflammatory diseases (IMID). The current assessment of such symptoms is usually conducted using patient reported outcomes (PROs) based on standardized questionnaires that patients usually complete every few months. This assessment protocol has raised some concerns, due to its propensity to exhibit biases derived from its subjectivity nature, or the low sensibility to changes, which may lead to a failure when trying to capture variability over time. In this work, we explore the use of smartphone data, which can serve as a proxy for how patients interact with their devices, to provide an effective, reliable, and objective assessment of the symptoms mentioned above. Our study comprises data from 137 participants belonging to 6 different disease groups, plus a healthy control group. We conducted statistical analysis based on repeated measures correlation, in which we analyze the correlation between screen-time and app-usage features with scores obtained from the PROs collected from the participants using a smartphone application.

Device deformation allows new types of gestures to be used in interaction. We identify that the gesture/use-case pairings proposed by interaction designers are often driven by factors relating improved tangibility, spatial directionality and strong metaphorical bonds. With this starting point, we argue that some of the designs may not make use of the full potential of deformation gestures as continuous, bipolar input techniques. In two user studies, we revisited the basics of deformation input by taking a new systematic look at the question of matching gestures with use cases. We observed comparable levels of UX when using bend input in different continuous bipolar interactions, irrespective of the choice of tangibility, directionality and metaphor. We concluded that device bend gestures use their full potential when used to control continuous bipolar parameters, and when quick reactions are needed. From our studies, we also identify relative strengths of absolute and relative mappings, and report a Fitts' law study for device bending input.

Shape-retaining freely-deformable interfaces can take innumerable distinct shapes, and creating specific target configurations can be a challenge. In this paper, we investigate how audio can guide a user in this process, through the use of either musical or metaphoric sounds. In a formative user study, we found that sound encouraged action possibilities and made the affordances of the interface perceivable. We also found that adding audio as a modality along with vision and touch, made a positive contribution to guiding users' interactions with the interface.

This study investigates movement time needed for exploring and selecting a target that is not seen in advance. An experiment where targets were presented with haptic, audio or visual feedback was conducted. The task of the participant was to search the targets with a hand held sensor-actuator device by horizontal scanning movements. The feedback appeared when the pointing was within the target boundaries. Range of distances to the target was varied between the experiment blocks. The results show that the modality did not have a significant effect on the total movement time but visual feedback yielded the shortest and haptic feedback the longest dwell time on target area. This was most probably caused by a visual priming effect and the slow haptic actuator rise time. The wider range of distances yielded longer movement times and within the widest range of distances the closest targets were explored longer than the targets in the middle. This was shown to be caused by the increased number of secondary submovements. The finding suggests that an alternative model to Fitts' law or linear prediction of target acquisition time should be developed taking into account the user's prior knowledge about target location.

This paper investigates how the perceived physicality of the action of applying force with a finger on a rigid surface (such as on a force-sensing touch screen) can be enhanced using real-time synthesized audio feedback. A selection of rich and evocative audio designs was used. Additionally, audio-tactile cross-modal integration was encouraged, by observing that the main rules of multisensory integration were supported. The study conducted showed that richness of perceived physicality increased considerably, mostly in its auditory expression (what pressing sounded like). In addition, in many instances it was observed that the haptic expression of physicality also increased (what pressing felt like), including some perception of compliance. This last result was particularly interesting as it showed that audio-tactile cross-modal integration might be present.

The present aim was to study if thermal stimuli presented to the palm can affect emotional responses when measured with emotion related subjective rating scales and changes in skin conductance response (SCR). Two target temperatures, cold and warm, were created by either decreasing or increasing the temperature of the stimulus 4 °C in respect to the participants current hand temperature. Both cold and warm stimuli were presented by using two presentation methods, i.e., dynamic and pre-adjusted. The results showed that both the dynamic and pre-adjusted warm stimuli elevated the ratings of arousal and dominance. In addition, the pre-adjusted warm and cold stimuli elevated the SCR. The results suggest that especially pre-adjusted warm stimuli can be seen as effective in activating the autonomic nervous system and arousal and dominance dimensions of the affective rating space.

This study is based on a user scenario where augmented reality targets could be found by scanning the environment with a mobile device and getting a tactile feedback exactly in the direction of the target. In order to understand how accurately and quickly the targets can be found, we prepared an experiment setup where a sensor-actuator device consisting of orientation tracking hardware and a tactile actuator were used. The targets with widths 5°, 10°, 15°, 20°, and 25° and various distances between each other were rendered in a 90° -wide space successively, and the task of the test participants was to find them as quickly as possible. The experiment consisted of two conditions: the first one provided tactile feedback only when pointing was on the target and the second one included also another cue indicating the proximity of the target. The average target finding time was 1.8 seconds. The closest targets appeared to be not the easiest to find, which was attributed to the adapted scanning velocity causing the missing the closest targets. We also found that our data did not correlate well with Fitts' model, which may have been caused by the non-normal data distribution. After filtering out 30% of the least representative data items, the correlation reached up to 0.71. Overall, the performance between conditions did not differ from each other significantly. The only significant improvement in the performance offered by the close-to-target cue occurred in the tasks where the targets where the furthest from each other.

In this paper we present results of a study where perception of dynamic audiotactile feedback to gesture input was examined. Our main motivation was to investigate how users' active input and different modality conditions effect the perception of the feedback. The experimental prototype in the study was a handheld sensor-actuator device that responds dynamically to user's hand movements creating an impression of a virtual texture. The feedback was designed so that the amplitude and frequency of texture were proportional to the overall angular velocity of the device. We used four different textures with different velocity responses. The feedback was presented to the user by the tactile actuator in the device, by audio through headphones, or by both. During the experiments, textures were switched in random intervals and the task of the user was to detect the changes while moving the device freely. The performances of the users with audio or audiotactile feedback were quite equal while tactile feedback alone yielded poorer performance. The texture design didn't influence the movement velocity or periodicity but tactile feedback induced most and audio feedback the least energetic motion. In addition, significantly better performance was achieved with slower motion. We also found that significant learning happened over time; detection accuracy increased significantly during and between the experiments. The masking noise used in tactile modality condition did not significantly influence the detection accuracy when compared to acoustic blocking but it increased the average detection time.

Proper feedback is one of the challenges in gesture interaction. Providing continuous feedback during the execution of the gesture increases the feeling of control and it can help user to perform the task more efficiently. In this paper we introduce an experimental handheld sensor-actuator device that responds dynamically to user's motion. With the device we explored the potential of continuous audiotactile feedback in closed-loop gesture interaction, designed simple synthesis methods for feedback, and tested the user perception. We designed four simple textures that respond to overall angular velocity of the device, all with different velocity responses. The system enabled us to examine how well subjects can distinguish the textures on the fly. Our preliminary findings show that audio modality dominates the perception. Tactile feedback worked quite well alone but the modalities together didn't lead to any better performance than audio alone.