Computation-based aesthetics metrics have been developed to help designers predict visual aesthetics scores for GUI design. However, designers find these evaluative scores difficult to understand. This paper proposed an interpretable aesthetics metric for GUI design that integrates visual aesthetics (visual similarity and spatial proximity) and GUI structure (semantic similarity and white space) to model visual grouping distribution. Two experiments were conducted to validate the metric’s ability to predict aesthetics and interpret outputs. Experiment 1 showed that our metric had a stronger correlation with users’ impressions of GUI visual aesthetics than past metrics. Experiment 2 suggested that our metric was easier to interpret and appeared more useful to Visual/Graphic/GUI designers than a conventional score-based alternative, by visualising the metric outputs as an experimental tool. Furthermore, this paper provided five potential insights to further advance computational aesthetics research.

Although virtual reality (VR) wayfinding systems are available for people with visual impairment, there is a lack of studies providing the choice of VR environments can affect their spatial information acquisition. To address these issues, we designed two VR-based wayfinding systems for people with visual impairment. We compared the efficacy of walk-in-place, where the user walks in a virtual environment by walking on a treadmill in the real world, and overground walking, where the user walks in a virtual environment by walking in a constrained analogous real walking space without walls and obstacles. By conducting user experiments on these two VR-based wayfinding systems, we evaluated how people with visual impairments acquired spatial information regarding paths and obstacles. Our results showed that people with visual impairment memorized the paths more accurately after the wayfinding with walk-in-place than with the overground walking. Conversely, they memorized the obstacles on the path more accurately after the wayfinding task with the overground walking than the walk-in-place. Lastly, we discussed the rationale for these results of task on the walk-in-place and the overground walking.

Simultaneous declines in dynamic visual, cognitive and physical modalities are common symptoms of aging which impact synergistically on the daily lives of older adults. VR game-based interventions offer promising alternatives for combining multimodalities (i.e., dynamic visual acuity, cognitive, and physical). This paper presents the first review of two aspects in immersive VR games for older adults using the PRISMA method: (1) psychology-based design mechanisms of visual, cognitive, and physical modalities; (2) experimental designs and intervention effects on cognitive and physical functions. Thirteen studies were identified—six involving healthy older adults and seven with mild cognitive impairment. Our findings revealed diverse game designs, reflecting a broad spectrum of features across visual, cognitive, and physical modalities, along with varying effects observed in cognitive and physical improvement. This study provides a comprehensive understanding of multimodal game design characteristics and intervention effects of VR games for older adults in the fields of psychology and human-computer interaction.

Rhythm can create a sense of structure or order, making it easier for users to understand and navigate interactive systems, thereby making user engagement more enjoyable and motivating. Recently, designers are increasingly interested in integrating rhythm into the design of interactive systems. However, the existing literature on this topic is fragmented, making it difficult for designers to find comprehensive guidance. This paper presents the first literature review of rhythm research for designing an interactive system based on the PRISMA method. Through an analysis of 59 papers based on the critical review method (e.g., research through design), we identify six research genres and provide a map to help future researchers locate their research position. Because there is currently no clear classification framework to guide designers in integrating rhythm into interactive systems, we propose a classification framework that includes six design qualities: three human rhythm-driven qualities and three computer rhythm-driven qualities based on the literature review. This classification framework provides a foundation for designers to regulate rhythm in interactive systems. In addition, we discuss emerging issues related to rhythm genres and rhythm classification framework and design qualities. We reflect on the effectiveness of our proposed framework by revisiting previous works on designing interactive systems, and provide potential directions for future research in this area. Overall, this paper aims to provide comprehensive guidance for designers who wish to integrate rhythm into the design of interactive systems, and to advance the understanding of the role of rhythm in interactive system design.

Existing research has explored how walking can stimulate creativity through the rhythm of footsteps. Researchers have also studied the positive effects of various sounds on creativity and creative thinking. However, the effects and potential benefits on creativity of the combination of footsteps and sounds remain unexplored. To fill this gap, we designed a FSI framework which we call Footstep and Sound Interactions. FSI encompasses three modes - Footstep-driven Sound Stimulation (FSS), Sound-driven Footstep Regulation (SFR), and Random Footstep and Sound Interactions (RFSI). We then conducted a user study in indoor and outdoor environments which compares the effects of three FSI modes with a walking only condition. Our results indicate that: (1) The FSS mode demonstrated higher performance in body awareness and attention regulation in both indoor and outdoor environments; (2) the creativity scores of indoor participants were higher than those of outdoor participants; (3) The indoor FSS mode produced greater creativity compared to the indoor RFSI mode; (4) Participants in the indoor FSS mode showed a significant correlation between heart rate and cadence with creativity compared to other groups; etc. Based on the findings, we discuss the FSI framework, the effects of FSI related to attention and creativity in the context of the walking experience, and we also discuss design implications and future work for the investigation and development of interactive (Footstep and Sound) technologies aimed at enhancing creativity while walking.

Design issues related to game difficulty in video games have attracted much attention. Although existing studies suggest a discrepancy between subjective game difficulty (SGD) and objective game difficulty (OGD), research exploring this relationship and its impacts on players remains limited. This study investigates whether there is a mismatch between OGD and SGD and their impacts on player enjoyment, engagement, and self-efficacy. A research framework is proposed, accompanied by seven hypotheses for testing. We first determined six dimensions of SGD as a new measuring structure to develop the questionnaire. While OGD is measured by the failure rate, which is a universal method to quantify it. We experimented to validate whether SGDs match OGDs in different OGD conditions in the designed Match-3 game. The three factors of player enjoyment, engagement, and self-efficacy were also measured to assess how the two difficulties affected them. A semi-structured interview was adopted to provide further insight into the experimental results. We found that OGD and SGD only partially match each other and that SGD acts as a mediator between the effects of OGD on player enjoyment, engagement, and self-efficacy. Although preliminary, our findings contribute to a deeper understanding of game difficulty and offer insights for future game difficulty design and human–computer interaction (HCI) research.

Running has emerged as an alternative to traditional meditation practices that involve sitting with closed eyes. The rhythmic nature of running establishes a connection between mind and body to promote mindfulness and improve the overall experience and benefits of running. The use of technology-supported meditation is gaining attention in the fields of mental and physical wellness. However, existing meditation technologies primarily focus on one-way stimuli for sitting or walking meditation, with little emphasis on adaptive design and development for running meditation. To address this issue, we present the design framework and system development of RunMe, an adaptive sound system specifically designed for running meditation. RunMe integrates stimulation and regulation mechanisms to enhance adaptive data interactions between the sounds and runners’ biodata. We compared the significance of the RunMe group with three other groups: use of non-adaptive sound, use of favorite music, and use of no music/sound. The results show that the RunMe group outperforms the other groups in attention regulation, body awareness, exercise motivation, and mindfulness. Importantly, RunMe has the potential to allow users to engage in running meditation without specialized equipment, making it accessible for daily practice. We also discuss the design framework and practical distinctions of RunMe, as well as the design implications and future directions for advancing running meditation.

Simultaneous declines in visual function (e.g., dynamic visual acuity), cognitive ability (e.g., cognitive control/multitasking), and physical function (e.g., balance) are major symptoms of aging. Integrating stimulation for those sensory channels into a game could be a suitable way for older adults to engage in long-term health interventions. However, existing game design has not considered the relationship and synergistic impact of multisensory channels of dynamic visual acuity, cognitive ability, and physical function for older adults. We therefore developed the first multisensory VR game system prototype based on cognitive psychology paradigms (e.g., multitasking and Go/No-Go tasks), full-body movement (limb movement), and dynamic visual acuity exercises (horizontal, vertical and forward-backward eye movements) in the VR system environment. We then conducted an experiment to measure the acceptability (in terms of e.g., cybersickness, mental workload, etc.) of our VR game for older adults. The young adults and a PC task were included for comparisons. Qualitative and quantitative results showed that older adults did not experience cybersickness in either sitting or standing postures during the VR gameplay; they well-accepted the workload of the VR game compared to the PC task. Our findings revealed that the design combination of three sensory channels shows synergistic benefits for older adults. Our game encourages older adults to engage in extensive body movement in sitting and standing postures, this is particularly important to people with disabilities who cannot stand. Design implications are provided for the future development and implementation of VR game design for older adults. Our work provides empirical support for the acceptability of multisensory VR systems in older adults, and contributes to the future design of VR games for older adults.

Abstract The paper presents a qualitative study to explore the use of fitness trackers and their social functions in intergenerational settings. The study covered three phases of semi-structured interviews with older and younger adults during individual and intergenerational use of the fitness trackers. The study revealed comparability as common fitness practice for older adults. The findings show that intergenerational fitness tracking practices can increase in-person meetings and daily discourses and thus enhance family social bonds. An unexpected benefit of this practice is its ability to help older adults overcome technology barriers related to the use of fitness trackers. Overall speaking, families whose intergenerational members already enjoy a strong relationship are likely to gain the most from such practices. Many challenges remain especially concerning the motivation and involvement of younger partners and the user experience design aspect of such digital programs. For this purpose, we have developed some recommendations for the future development and deployment of intergenerational fitness tracking systems to stimulate interactions between younger and older family members and thus to promote their physical and emotional well-being.

Current VR systems provide various text input methods that enable users to enter text efficiently with virtual keyboards. However, little attention has been paid to facilitate text revision during the VR text input process. We first summarized existing text revision solutions in current VR text input research and found that backspace is the only tool available for text revision with virtual keyboards with few mentioning designs for caret control. To systematically explore VR text revision designs, we presented a design space for VR text revision based on backspace and caret. With the proposed design space, we further analyzed the feasibility of the combined usage of backspace and caret by proposing and evaluating four VR text revision techniques. Outcomes of this research can provide a fundamental understanding of VR text revision solutions (with backspace and caret) and a comparable basis for evaluating future VR text revision techniques.

Steering law reveals a linear relationship between the movement time (MT) and the index of difficulty (ID) in trajectory-based steering tasks. However, it does not relate the variance or distribution of MT to ID. In this paper, we propose and evaluate models that predict the variance and distribution of MT based on ID for steering tasks. We first propose a quadratic variance model which reveals that the variance of MT is quadratically related to ID with the linear coefficient being 0. Empirical evaluation on a new and a previously collected dataset show that the quadratic variance model accounts for between 78% and 97% of variance of observed MT variances; it outperforms other model candidates such as linear and constant models; adding the linear coefficient leads to no improvement on the model fitness. The variance model enables predicting the distribution of MT given ID: we can use the variance model to predict the variance (or scale) parameter and Steering law to predict the mean (or location) parameter of a distribution. We have evaluated six types of distributions for predicting the distribution of MT. Our investigation also shows that positively skewed distribution such as Gamma, Lognormal, Exponentially Modified Gaussian (ExGaussian), and Extreme value distributions outperformed the symmetric distribution such as Gaussian and truncated Gaussian distribution in predicting the MT distribution, and Gamma distribution performed slightly better than other positively skewed distributions. Overall, our research advances the MT prediction of steering tasks from a point estimate to variance and distribution estimates, which provides a more complete understanding of steering behavior and quantifies the uncertainty of MT prediction.

In this paper, we present ArmMenu, a command input approach for distant displays. ArmMenu has a circular interface like pie menus and menu selection is performed by proprioception-based lateral arm movements. We implemented ArmMenu with an off-the-shelf body tracking device (Kinect) and conducted two experiments to validate its efficacy. In the first experiment, we explored the design space of ArmMenu by varying the number of menu items, with exposed or hidden menu modes. Users can operate up to 8-item menus with high selection accuracy (>98%). ArmMenu was fast and accurate even with the hidden menu mode. The second experiment compared the performance of ArmMenu and touchless marking menus. While having similar selection accuracy, ArmMenu was faster and more preferable by users. Our studies consequently demonstrate ArmMenu's effectiveness for command input on distant displays.

Relatively little is known about eye and finger movement in typing with mobile devices. Most prior studies of mobile typing rely on log data, while data on finger and eye movements in typing come from studies with physical keyboards. This paper presents new findings from a transcription task with mobile touchscreen devices. Movement strategies were found to emerge in response to sharing of visual attention: attention is needed for guiding finger movements and detecting typing errors. In contrast to typing on physical keyboards, visual attention is kept mostly on the virtual keyboard, and glances at the text display are associated with performance. When typing with two fingers, although users make more errors, they manage to detect and correct them more quickly. This explains part of the known superiority of two-thumb typing over one-finger typing. We release the extensive dataset on everyday typing on smartphones.

Text revision is an important task to ensure the accuracy of text content. Revising text on mobile devices is cumbersome and time-consuming due to the imprecise caret control and the repetitive use of the backspace. We present Swap, a novel replacement-based technique to facilitate text revision on mobile devices. We conducted two user studies to validate the feasibility and the effectiveness of Swap compared to traditional text revision techniques. Results showed that Swap reduced efforts in caret control and repetitive backspace pressing during the text revision process. Most participants preferred to use the replacement-based technique rather than backspace and caret. They also commented that the new technique is easy to learn, and it makes text revision rapid and intuitive.

We present a computational model of visual search on graphical layouts. It assumes that the visual system is maximising expected utility when choosing where to fixate next. Three utility estimates are available for each visual search target: one by unguided perception only, and two, where perception is guided by long-term memory (location or visual feature). The system is adaptive, starting to rely more upon long-term memory when its estimates improve with experience. However, it needs to relapse back to perception-guided search if the layout changes. The model provides a tool for practitioners to evaluate how easy it is to find an item for a novice or an expert, and what happens if a layout is changed. The model suggests, for example, that (1) layouts that are visually homogeneous are harder to learn and more vulnerable to changes, (2) elements that are visually salient are easier to search and more robust to changes, and (3) moving a non-salient element far away from original location is particularly damaging. The model provided a good match with human data in a study with realistic graphical layouts.

Hierarchy structures such as file systems are widespread interfaces for item retrieval and selection tasks. Some hierarchies can be modified by end-users, such as application launchers on smartphones or pictures in a file folder. These modifiable hierarchies cannot benefit from an optimization made beforehand as their content, unknown during the design process, is constantly evolving. We hence propose an analytic model which designers can integrate in their system to recommend a range of local structure modifications (e.g., creating new folders) to end-users. Proposing a range of modifications gives flexibility to end-users regarding their own meaningful grouping and labeling choices to follow a recommendation. A first experiment confirms that the recommendations built on our model can lead to modified hierarchies resulting in faster theoretical selection times. A second experiment confirms that the theoretical selection times fit empirical selection times in different hierarchy visual layouts: linear, radial, and grid.

This paper presents the first investigation into using the goal-crossing paradigm for object selection with virtual reality (VR) head-mounted displays. Two experiments were carried out to evaluate ray-casting crossing tasks with target discs in 3D space and goal lines on 2D plane respectively in comparison to ray-casting pointing tasks. Five factors, i.e. task difficulty, the direction of movement constraint (collinear vs. orthogonal), the nature of the task (discrete vs. continuous), field of view of VR devices and target depth, were considered in both experiments. Our findings are: (1) crossing generally had shorter or no longer time, and higher or similar accuracy than pointing, indicating crossing can complement or substitute pointing; (2) crossing tasks can be well modelled with Fitts' Law; (3) crossing performance depended on target depth; (4) crossing target discs in 3D space differed from crossing goal lines on 2D plane in many aspects such as time and error performance, the effects of target depth and the parameters of Fitts' models. Based on these findings, we formulate a number of design recommendations for crossing-based interaction in VR.

Autostereograms are 2D images that can reveal 3D content when viewed with a specific eye convergence, without using extra-apparatus. We contribute to autostereogram studies from an HCI perspective. We explore touch inputs and output design options when interacting with autostereograms on smartphones. We found that an interactive help (i.e. to control the autostereogram stereo-separation), a color-based feedback (i.e. highlight of the screen), and a direct touch input can provide support for faster and more accurate interaction than a static help (i.e. static dots indicating the stereo-separation), an animated feedback (i.e., a 'pressed' effect), and an indirect input. In addition, results reveal that participants learn to perceive smaller and smaller autostereogram content faster with practice. This learning effect transfers across display devices (smartphone to desktop screen).

Current gesture interaction paradigm mainly involves a one-to-one gesture-command mapping. This leads to memorability issues regarding (1) the mapping - as each new command requires a new gesture, and (2) the gestures specifics (e.g., motion paths) - that can be complex to leverage the recognition of several gestures. We explore the concept of combining 3D gestures when interacting in smart environments. We first propose a design space to characterize the temporal and spatial combination aspects, and the gesture types used by the combination. We then report results from three user studies in the context of smart TV interaction. The first study reveals that end-users can create gesture sets with combinations fully optimized to reuse gestures. The second study shows that combining gestures can lead to improved memorability compared to single gestures. The third study reveals that preferences for gestures combination appear when single gestures have an abstract gesture-command mapping.

Mindfulness practices are well-known for their benefits to mental and physical well-being. Given the prevalence of smartphones, mindfulness applications have attracted growing global interest. However, the majority of existing applications use guided meditation that is not adaptable to each user's unique needs or pace. This article proposes a novel framework called Attention Regulation Framework (ARF) , which studies how more flexible and adaptable mindfulness applications could be designed, beyond guided meditation and toward self-regulated meditation. ARF proposes mindfulness interaction design guidelines and interfaces whereby practitioners naturally and constantly bring their attention back to the present moment and develop non-judgmental awareness. This is achieved by the performance of subtle movements, which are supported by non-intrusive detection-feedback mechanisms. We used two design cases to demonstrate ARF in static and kinetic meditation conditions. We conducted four user evaluation studies in unique situations where ARF is particularly effective, vis-à-vis mindfulness practice in busy environments and mindfulness interfaces that adapt to the pace of the user. The studies show that the design cases, compared with guided meditation applications, are more effective in improving attention, mindfulness, mood, well-being, and physical balance. Our work contributes to the development of self-regulated mindfulness technologies.

Although the HCI community inevitably contributes to engagement via beauty according to the attention paid to known and yet to be discovered principles of aesthetics for digital interface design, it is lacking an epistemological corpus which should include the notion, human factors and the quantification of aesthetic aspects. The aim of the proposed workshop is to discuss these issues in order to strengthen aesthetic studies specifically for HCI and related fields. We want to create a forum for discussing, drafting and promoting the foundations for disciplined aesthetics design within the HCI community. We thus welcome contributions such as theories, methodologies, evaluation methods, and potential applications regarding effective aesthetics for HCI and related fields. Concretely, we aim to (i) map the present state-of-art of aesthetic research in HCI, (ii) build a multidisciplinary community of experts, and (iii) raise the profile of this aesthetics research area within HCI community.

Pointing and gesturing with the dominant thumb (DT) and the non-dominant thumb (NT) are two common tasks for bimanual tablet gripping interaction. Understanding the differences between DT and NT is important to pointing and gesturing based interface design on tablets, but is overlooked by previous studies. We therefore conducted two experiments. In the first experiment, participants carried out pointing tasks with DT and NT, respectively, on a tablet. We found DT and NT input differed in pointing time (DT had average shorter time than NT) and required target sizes for fast and accurate pointing (7.7 mm and 9.4 mm in diameter for DT and NT). On the other hand, DT and NT were alike in pointing accuracy. They performed about the same for target size larger than 9.6 mm and distance shorter than 21 mm. Both DT and NT pointing can be modeled by Fitts’ law. In the second experiment, participants performed gesturing tasks with DT and NT respectively on a tablet. The collected data were analyzed using a set of gesture features. Results showed DT and NT were different in features such as articulation time, size ratio and indicative angle difference, but similar in features like aperture, axial symmetry and shape distance. The differences of time and accuracy between DT and NT depended on gesture complexity but not gesture sizes. We discuss these findings with implications for future bimanual thumb interaction design and research.

In our fast-paced society, stress and anxiety have become increasingly common. Meditation for relaxation has received much attention. Meditation apps exploit various senses, e.g., touch, audio and vision, but the relationship between human senses and interactive meditation is not well understood. This paper empirically evaluates the effects of single and combined human senses on interactive meditation. We found that the effectiveness of human senses can be defined by their respective roles in maintaining the balance between relaxation and focus. This work is the first to attempt to understand these relationships. The findings have broad implications for the field of multi-modal interaction and interactive meditation applications.

Predicting how users learn new or changed interfaces is a long-standing objective in HCI research. This paper contributes to understanding of visual search and learning in text entry. With a goal of explaining variance in novices' typing performance that is attributable to visual search, a model was designed to predict how users learn to locate keys on a keyboard: initially relying on visual short-term memory but then transitioning to recall-based search. This allows predicting search times and visual search patterns for completely and partially new layouts. The model complements models of motor performance and learning in text entry by predicting change in visual search patterns over time. Practitioners can use it for estimating how long it takes to reach the desired level of performance with a given layout.

We are often overwhelmed by everyday stressors. Mindfulness meditation can help slow things down and bring one's attention into the present moment. Given the prevalence of smartphones, mindfulness-based mobile applications (MBMAs) have received much attention. Current MBMAs mainly use the guided meditation method which may not be always effective, e.g., users may not be able to follow the pace of instructions and they need a private environment. This paper presents a framework for interactive MBMAs which allows users to self-regulate their attention according to their abilities and conditions. The framework is described by an Attention-Regulation Process and has two components: (1) Relaxation Response and (2) Attention Restoration Theory. The framework is validated by our experiment. It also informs future development for interactive meditation and has broad implications for designing mindfulness and well-being.

This paper presents the EV2-Pen which leverages electrovibration technology and vibration technology in pen interaction. Electrovibration technology can produce multisensory feedback when the pen is in motion (sliding/moving on the screen), and vibration technology can provide vibrative feedback when the pen is stationary (pointing/resting on the screen). We conducted an experiment to investigate user performance with the EV2-Pen. The results indicated that the EV2-Pen outperformed the EV-Pen [18, 19] in pointing-steering tasks. Finally, we discuss the characteristics of the EV2-Pen, and explore some possible applications and scenarios.

Although there have been recent advances in smartphone accessibility for blind people, they rely on screen readers and voice commands which are not ideal for users with visual impairment in mobile situations. By contrast, recent research has shown that marking menus would be beneficial to users' eyes-free interactions. However, the literature lacks accessibility implications and adaptation to the needs of blind people. This paper investigates blind people's capabilities to perform marking menu selections using the 3D motion of a smartphone in order to invoke smartphone functions. We present the bounds and range of marking gestures that a blind person can perform at each level, and the number of levels that a blind person can successfully cope with. Based on the experiment results, we also presented design guidelines.

Body-based motion gestures have been gaining popularity in designing interactive systems. However, theories and design guidelines on the use of body movement in design have not been fully evaluated. This article investigates human ability to perform discrete target selection tasks using stretching action through two controlled experiments. The experimental results indicate that: (1) the range of the discrete levels of depth that users can easily discriminate with arm stretching is up to 16 with full visual feedback, but is down to 4 without the feedback; (2) dwelling, a gesture with keeping a hand motionless and the cursor within a target area for a certain amount of time, may be the best gesture for confirmation command; (3) full visual feedback can improve the user performance; and (4) arm stretching action can be modeled using Fitts’ law. We also discuss the design potentials for Stretch Widgets based on these results.

Mid-air gestures enable intuitive and natural interactions. However, few studies have investigated the use of mid-air gestures for blind people. TV interactions are one promising use of mid-air gestures for blind people, as "listening"' to TV is one of their most common activities. Thus, we investigated mid-air TV gestures for blind people through two studies. Study 1 used a user-elicitation approach where blind people were asked to define gestures given a set of commands. Then, we present a classification of gesture types and the frequency of body parts usage. Nevertheless, our participants had difficulty imagining gestures for some commands. Thus, we conducted Study 2 that used a choice-based elicitation approach where the participants selected their favorite gesture from a predefined list of choices. We found that providing choices help guide users to discover suitable gestures for unfamiliar commands. We discuss concrete design guidelines for mid-air TV gestures for blind people.

Engagement is a key reason for introducing gamification to learning and thus serves as an important measurement of its effectiveness. Based on a literature review and meta-synthesis, this paper proposes a comprehensive framework of engagement in gamification for learning. The framework sketches out the connections among gamification strategies, dimensions of engagement, and the ultimate learning outcome. It also elicits other task - and user - related factors that may potentially impact the effect of gamification on learner engagement. To verify and further strengthen the framework, we conducted a user study to demonstrate that: 1) different gamification strategies can trigger different facets of engagement; 2) the three dimensions of engagement have varying effects on skill acquisition and transfer; and 3) task nature and learner characteristics that were overlooked in previous studies can influence the engagement process. Our framework provides an in-depth understanding of the mechanism of gamification for learning, and can serve as a theoretical foundation for future research and design.

This paper presents an Electrovibration Pen (EV-Pen) which incorporates electrovibration technology into pen interactions. The EV-Pen has two unique characteristics: precise interaction and pen-on-paper feeling. We conducted four experiments for evaluating the EV-Pen. Experiment 1 was to determine preliminary characteristics of the EV-Pen. Experiments 2 and 3 were to evaluate precise interaction task performance through a steering task and a tracing task. We compared user performance between the EV-Pen, a mechanical-vibration pen, a normal pen without feedback and electrovibration-based finger interaction. Experimental results indicated that the EV-Pen outperformed the other devices in precise task interactions. Experiment 4 tested pen-on-paper feeling in drawing and handwriting tasks, and it was observed that the EV-Pen significantly enhances user experience. Based on the experimental results, we discuss implications and potential benefits for the design of the EV-Pen.

To simplify the selection of remote targets on large displays, a novel technique called ShifTable is presented in this paper. With ShifTable, users move a finger or a pen towards a desired target, and the whole screen carrying the target shifts faster in the opposite direction, towards the finger or pen. We conducted two experiments to compare ShifTable with unaided direct selection (baseline), Gesture Select and Pan. The results show that ShifTable was significantly faster than unaided direct selection and Gesture Select in selection time, and outperformed Gesture Select in error rate. In addition, elderly participants were able to copy-and-paste targets significantly faster with ShifTable than with Pan. Although ShifTable's simple operation allows users to easily and naturally select remote targets on a large display, we also found usability issues that need to be addressed for this intuitive technique to be more widely accepted by the elderly users.

This study investigated differences and similarities between finger and pen gestures on stationary devices (sitting posture) and mobile devices (sitting and walking postures). The recorded gestures were analyzed according to multiple gesture features. We found (1) pen and index finger gestures were different in features like size ratio but similar in features like angle difference ; (2) implement (pen vs. index finger vs. thumb) interacted with gesture complexity and size in features like articulation time ; (3) features like time and shape distance , were different between the pen and index finger on mobile devices (walking) but similar on stationary devices; (4) one-handed thumb gestures had worse performances than index finger gestures by time and accuracy in sitting but similar performances in walking; and (5) for the three implements, gesture drawing time and accuracy on mobile devices reduced from sitting to walking condition. We discuss these findings with implications for future gesture design and research.

Motion gestures enable natural and intuitive input in video games. However, game gestures designed by developers may not always be the optimal gestures for players. A key challenge in designing appropriate game gestures lies in the interaction-intensive nature of video games, i.e., several actions/commands may need to be executed concurrently using different body parts. This study analyzes user preferences in game gestures, with the aim of accommodating high interactivity during gameplay. Two user-elicitation studies were conducted: first, to determine user preferences, participants were asked to define gestures for common game actions/commands; second, to develop effective combined-gestures, participants were asked to define possible game gestures using each body part (one and two hands, one and two legs, head, eyes, and torso). Our study presents a set of suitable and alternative body parts for common game actions/commands. We also present some simultaneously applied game gestures that assist interaction in highly interactive game situations (e.g., selecting a weapon with the feet while shooting with the hand). Interesting design implications are further discussed, e.g., transferability between hand and leg gestures.

It is our great pleasure to present this special issue on human–computer interaction (HCI) in the Asia-Pacific Region in the International Journal of Human–Computer Interaction (IJHCI). This is a c...

This article experimentally investigates user performances with various surface textures in steering and gesture input tasks. Results reveal that (a) low friction material makes users spend more time on each task, and (b) although low friction material benefits the smoothness of trajectory, it causes more trajectory errors, and (c) users apply less force or pressure with slippery materials during the tasks. These findings are the more significant because they demonstrate that the common glass surface of most tablet surfaces is not the best kind of surface for optimum accuracy or for user satisfaction. The results suggest that users should be free to change the surface texture of the device in order to get natural and realistic haptic feedback according to different tasks and personal preferences.

This study examined the performance of two scrolling techniques (flick and ring) for document navigation in touch-based mobile phones using three input methods (index finger, pen, and thumb), with specific consideration given to two postures: sitting and walking. The findings are as follows: (a) in both sitting and walking postures, for the three input methods, flick resulted in shorter movement time and fewer crossings than ring, suggesting flick is superior to ring for document navigation; (b) for sitting posture, regarding pen and thumb input, ring led to shorter movement time than flick for large target distances, indicating ring has a potential interaction advantage; (c) regarding sitting and walking postures, both flick and ring document scrolling in touch-based mobile phones can be modeled by the Anderson model (Andersen, 2005). Designers of future scrolling techniques should consider these differences, as well as exploit the advantages and avoid the disadvantages of ring and flick scrolling.

This study quantitatively investigated optimal finger-based entry size in touch-based mobile phones for two commonly used Chinese handwriting input styles: two-handed entry with the nondominant hand holding the device and the index finger of the dominant hand entering characters, and one-handed entry with the dominant hand holding the device and the thumb of the dominant hand being used for character entry. Results were assessed in terms of the number and length of protruding strokes, writing time, stroke writing speed, size ratio, number of writing attempts, and subjective preference. For both one-handed entry and two-handed entry, the optimal entry box size was found to be 2.5 × 2.5 cm. This size entry box is large enough for fast and accurate handwriting with high-entry-area utilization rate and few, short protruding strokes. The experimental results and methodology of this study can be employed in user interface design for handwriting in touch-based mobile phones.

This paper reports an empirical investigation in which participants produced a set of stroke gestures with varying degrees of complexity and in different target sizes using both the finger and the pen. The recorded gestures were then analyzed according to multiple measures characterizing many aspects of stroke gestures. Our findings were as follows: (1) Finger drawn gestures were quite different to pen drawn gestures in basic measures including size ratio and average speed. Finger drawn gestures tended to be larger and faster than pen drawn gestures. They also differed in shape geometry as measured by, for example, aperture of closed gestures, corner shape distance and intersecting points deviation; (2) Pen drawn gestures and finger drawn gestures were similar in several measures including articulation time, indicative angle difference, axial symmetry and proportional shape distance; (3) There were interaction effects between gesture implement (finger vs. pen) and target gesture size and gesture complexity. Our findings show that half of the features we tested were performed well enough by the finger. This finding suggests that "finger friendly" systems should exploit these features when designing finger interfaces and avoid using the other features in which the finger does not perform as well as the pen.

Inherent pen input modalities such as tip pressure, tilt and azimuth (PTA) have been extensively used as additional input channels in pen-based interactions. We conducted a study to investigate the natural use profiles of PTA, which describes the features of PTA in the course of normal pen use such as writing and drawing. First, the study reveals the ranges of PTA in normal pen use, which can distinguish pen events accidently occurring in normal drawing and writing from those used for mode switch. The natural use profiles also show that azimuth is least likely to cause false pen mode switching while tip pressure is most likely to cause false pen mode switching. Second, the study reveals correlations among various modalities, indicating that pressure plus azimuth is superior to other pairs for dual-modality control.

Research literature has shown that pen tilt is a promising input modality in pen-based interaction. However, the human capability to control pen tilt has not been fully evaluated. This paper systematically investigates the human ability to perform discrete target selection tasks by varying the stylus' tilt angle through two controlled experiments: pen tilt target acquiring (Experiment 1) and tilt pointing (Experiment 2). Results revealed a decreasing power relationship between angular width and selection time in Experiment 1. The results of Experiment 2 confirmed that pen tilt pointing can be modeled by Fitts' law. Based on our quantitative analysis, we discuss the human ability to control pen tilt and the implications of pen tilt use. We also propose a taxonomy of pen tilt based interaction techniques and showcase a series of possible pen tilt technique designs.

Many handheld devices with stylus pens are available in the market; however, there have been few studies which examine the effects of the size of the stylus pen on user performance and subjective preferences for handheld device interfaces for various age groups. Two experiments (pen-length experiment and pen-tip width/pen-width experiment) were conducted to determine the most suitable dimensions (pen-length, pen-tip width and pen-width) for a stylus pen among young adult, child and older users. In pen-length experiment, five pen-lengths (7, 9, 11, 13 and 15 cm) were evaluated for young adult, child and older users, respectively. In pen-tip width/pen-width experiment, six combinations of three pen-tip widths (0.5, 1.0 and 1.5 mm) and two pen-widths (4 and 7 mm) were compared for young adult, child and older users, respectively. In both experiments, subjects conducted pointing, steering and writing tasks on a PDA. The results were assessed in terms of user performance and subjective evaluations for all three pointing, steering and writing tasks. We determined that the most suitable dimensions of the pen are as follows: pen-length 11–15 cm, pen-tip width 1.0–1.5 mm and pen-width 7 mm for older users; for child users, pen-length 7–13 cm, pen-tip width 1.0–1.5 mm and pen-width 4 mm; and for young adult users, pen-length 11 cm, pen-tip width 0.5 mm, and pen-width 7 mm.

A tracking state increases the bandwidth of pen-based interfaces. However, this state is difficult to detect with default visual feedback. This paper reports on two experiments that are designed to evaluate multimodal feedback for pointing tasks (both 1D and 2D) in tracking state. In 1D pointing experiments, results show that there is a significant effect for input types on movement time while feedback type and the use of different hands for receiving feedback (i.e. the dominant or non-preferred hand) do not affect movement time significantly. We also report that there is a significant effect for feedback types and input device types on error rate while the choice of hand (used for detecting feedback vibrations) does not affect the error rate significantly. In the 2D pointing experiment, results show that there are no significant effects for either input type or the use of different hands on movement time while feedback type affects movement time significantly. Results for both the 1D and 2D pointing tasks show that tactile plus visual feedback can improve accuracy and audio is not efficient to give user feedback in tracking state. This paper proposes several guidelines for feedback design in tracking state. We believe these results can aid designers of pen-based interfaces.

We propose a quantitative model for dwell-based eye pointing tasks. Using the concepts of information theory to analogize eye pointing, we define an index of difficulty (IDeye) for the corresponding tasks in a similar manner to the definition that Fitts made for hand pointing. According to our validations in different situations, IDeye, which takes account of the distinct characteristics of rapid saccades and involuntary eye jitters, can accurately and meaningfully describe eye pointing tasks. To the best of our knowledge, this work is the first successful attempt to model eye gaze interactions.

This study presents a series of three experiments that evaluate human capabilities and limitations in using pen-tip pressure as an additional channel of control information in carrying out trajectory tasks such as drawing, writing and gesturing on computer screen. The first experiment measured the natural range of force used in regular drawing and writing tasks. The second experiment tested human performance of maintaining pen-tip pressure at different levels with and without a visual display of the pen pressure. The third experiment, using the steering law paradigm, studied path steering performance as a function of the steering law index of difficulty, steering path type (linear and circular) and pressure precision tolerance interval. The main conclusions of our investigation are as follows. The natural range of pressure used in drawing and writing is concentrated in the 0.82 N (SI force unit Newton (N) is used in this article) to 3.16 N region. The resting force of the pen tip on the screen is between 0.78 N and 1.58 N. Pressure near or below the resting force is markedly more difficult to control. Visual feedback improves pressure-modulated trajectory tasks. Up to six layers of pressure can be controlled in steering tasks, but the error rate changed from 4.9% for one layer of pressure to 35% for six layers. The steering law holds for pressure steering tasks, which enables systematic prediction of successful steering time for a given path's length, width and pressure precision criterion. The steering time can also be modelled as a logarithmic function of pressure control precision ratio σ. Taken together, the current work provides a systematic body of empirical knowledge as basis for future research and design of digital pen applications.

The steering law is an excellent performance model for trajectory-based tasks, such as drawing and writing in GUIs. Current studies on steering tasks focus on the effect of system factors (i.e. path width and amplitude) on the movement time and steering law's related applications. We conducted a series of experiments to further explore the effect of different operational biases (bias speed or accuracy) on steering completion time and standard deviation for two steering trajectory shapes, i.e. a straight steering task and a circular steering task, and then establish a new model accommodating system and subjective factor in steering tasks. Empirical results showed that the new model is more predictive and robust than the traditional steering law.

Current multi-touch interaction techniques typically only use the x-y coordinates of the human finger's contact with the screen. However, when fingers contact a touch-sensitive surface, they usually approach at an angle and cover a relatively large 2D area instead of a precise single point. In this paper, a Frustrated Total Internal Reflection (FTIR) based multi-touch device is used to collect the finger imprint data. We designed a series of experiments to explore human finger input properties and identified several useful properties such as contact area, contact shape and contact orientation which can be exploited to improve the performance of multi-touch selecting and pointing tasks. Based on the experimental results, we discuss some implications for the design of human finger input interfaces and propose several design prototypes which incorporate these implications. A set of raw data and several concrete recommendations which are useful for the research community are also presented.

Current interactions on direct-touch interactive surfaces are often modeled based on properties of the input channel that are common in traditional graphical user interfaces (GUI) such as x-y coordinate information. Leveraging additional information available on the surfaces could potentially result in richer and novel interactions. In this paper we specifically explore the role of finger orientation. This property is typically ignored in touch-based interactions partly because of the ambiguity in determining it solely from the contact shape. We present a simple algorithm that unambiguously detects the directed finger orientation vector in real-time from contact information only, by considering the dynamics of the finger landing process. Results of an experimental evaluation show that our algorithm is stable and accurate. We then demonstrate how finger orientation can be leveraged to enable novel interactions and to infer higher-level information such as hand occlusion or user position. We present a set of orientation-aware interaction techniques and widgets for direct-touch surfaces. ACM Classification: I.3.6 [Methodology and Techniques]: Interaction techniques.

This article presents an empirical study to determine the optimal size of a box for the input of handwriting characters on personal digital assistants (PDAs). The experiments involve the consideration of boxes for different kinds of characters, different box sizes and shapes (square and rectangular), different user postures, and the age differences of users. The results are assessed in terms of high performance (high character recognition rates, minimal stroke protrusions outside the character box, minimal number of error corrections, minimum writing time) and subjective ratings (e.g., ease of writing and minimum degree of fatigue). The results show that the optimal size of character boxes for the input of alphanumeric characters is 12 × 14 mm (rectangular), whereas for Kanji (Chinese characters) mixed with Kana characters and for Hiragana & Katakana characters the optimal size is 14 × 14 mm (square). We believe that knowledge of the optimal size of character input boxes will be useful for the design of user interfaces on PDAs.

In order to improve the stability of eye cursor, we introduce three methods, force field (FF), speed reduction (SR), and warping to target center (TC) to modulate eye cursor trajectories by counteracting eye jitter, which is the main cause of destabilizing the eye cursor. We evaluate these methods using two controlled experiments. One is an attention task experiment, which indicates that both FF and SR significantly alleviate the instability of eye cursor, but TC is not as we anticipated. The other is a 2D pointing task experiment, which shows that FF and SR as well as the improved implementation of SR (iSR) indeed improve human performance in dominant dwell-based eye pointing tasks of eye-based interactions. The method iSR is especially effective to accelerate eye pointing (10.5% and 8.5%) and reduce error rate (6.1% and 2.7%) when target diameter D = 45 and 60 pixels.

In this study six different mode switching techniques (i.e. timeout mode switching, non-preferred hand mode switching, barrel button mode switching, pressure mode switching, tilt mode switching and azimuth mode switching) based on multiple parameters pen input are proposed. The results indicate that the techniques utilizing tilt angle and azimuth offer faster performance than the others.

The fundamental technique of vision-based multi-touch technique is the high performance image processing. In this paper, two algorithms are presented to resolve the problems of blob recognition and tracking. Image contour transformation algorithm is adopted to analyze the existence of finger contact area and Minimum Distance First (MDF) algorithm is designed to identify and track the corresponding blob in two sequential images. The result shows that the performance of two algorithms fully meets the requirement of real time multi-touch system.

Three experiments were conducted in this study to investigate the human ability to control pen pressure and pen tilt input, coupling cursor position, angle and scale respectively. Comparisons between pressure input and tilt input have been made in the three experiments. Experimental results show that decreasing pressure input resulted in very bad performance and was not a good input technique for any of the three experiments. In "experiment1-coupling to cursor position", the tilt input technique performed relatively better than the increasing pressure input technique, even though the tilt technique had a slightly higher error rate. In "experiment2-coupling to cursor angle", the tilt input performed relatively better than the increasing pressure input in terms of time. In "experiment3-coupling to cursor scale", tilt input performed a little better than increasing pressure input in terms of adjustment time, but a little worse in terms of the degree of control that users exerted. These results will be useful for further user interface design.

"Book Review Conceptual Foundations of Human Factors Measurement by David Meister." International Journal of Human-Computer Interaction, 19(1), pp. 157–158

Informal presentations are a lightweight means for fast and convenient communication of ideas. People communicate their ideas to others on paper and whiteboards, which afford fluid sketching of graphs, words and other expressive symbols. Unlike existing authoring tools that are designed for formal presentations, we created SketchPoint to help presenters design informal presentations via freeform sketching. In SketchPoint, presenters can quickly author presentations by sketching slide content, overall hierarchical structures and hyperlinks. To facilitate the transition from idea capture to communication, a note-taking workspace was built for accumulating ideas and sketching presentation outlines. Informal feedback showed that SketchPoint is a promising tool for idea communication.

Two experiments were conducted to compare pen-based selection strategies and their characteristics.Two state transition models were also formulated which provide new vocabulary that will help in investigating interactions related to target selection issues.Six strategies, which can be described by the state transition models, were used in the experiments.We determined the best strategy of the six to be the "Slide Touch" strategy, where the target is selected at the moment the pen-tip touches the target for the first time after landing on the screen surface.The six strategies were also classified into strategy groups according to their characteristics.We determined the best strategy group to be the "In-Out" strategy group, where the target is selected by contact either inside or outside the target.Analyses show that differences between strategies are influenced by variations in target size; however, the differences between strategies are not affected by the distance to the target (i.e., pen-movement-distance) or the direction of pen movement (i.e., pen-movement-direction).We also found "the smallest maximum size" of five pixels, i.e., the boundary value for the target size below which there are significant differences, and above which there are no significant differences between the strategies in error rate.Relationships between interaction states, routes, and strategy efficiency were also investigated.