Environmental User Experience (UX) data collection is essential for user research, enabling evidence-based design decisions. However, traditional retrospective methods like micro-phenomenological interviews suffer from recall inaccuracies and memory distortions. Concurrent UX data collection methods with environmental contexts are promising but lack in-depth investigation. To examine this potential, we conducted a formative study with 34 participants, identifying design goals such as natural interaction, in-situ annotation, and spatial-temporal coupling. We developed JourneyCapturer, an interactive tool that fulfills these goals to integrate concurrent annotation within Immersive Virtual Environment (IVE), enabling real-time UX data capture within contextual scenarios. Using a mixed-method design, we comparatively evaluated concurrent IVE annotations, retrospective interviews, and the combined method with 20 participants, demonstrating how JourneyCapturer improves UX collection processes and outcomes. Our findings suggest that a consciously proactive concurrent IVE method with a first-person perspective advances UX research, offering implications for expert collaboration, multi-modal analytics, and IVE-based field studies.

The integration of Generative AI (GenAI) into Architecture, Engineering, and Construction (AEC) design marks a paradigm shift, yet empirical studies on AEC professional adoption are scarce. This study aims to bridge this gap through a mixed-methods study involving interviews with 20 professionals and a survey of 191 professionals. Findings show GenAI is mainly used in conceptual design, improving client and intra-team collaboration, especially in concept-only projects, strong site integration projects, and challenging client engagements projects for inspiration exploration, client communication and design service boundaries expansion. We further uncover a critical paradox: professionals perceive GenAI’s creativity as intrinsically linked to the unpredictability. Additionally, a nuanced “shame-yet-pride” dynamic is identified, wherein professionals publicly regard GenAI as a key asset but conceal its use to exaggerate manual effort. Based on these insights, we recommend fostering an industry-wide discussion for a mindset shift toward transparent collaboration with GenAI. This study offers the first comprehensive empirical study for understanding GenAI’s current role and future potential in transforming AEC design practices.

Engaging the younger generation is crucial to the preservation and innovation of intangible cultural heritage (ICH). However, the technical barriers to ICH participation often deter youth generation. Generative Artificial Intelligence (GenAI) provides opportunities to lower barriers but risks undermining ICH authenticity and leading to over-reliance on automation driven by GenAI. To address these challenges, we aim to develop a human-GenAI co-creation system to support the transmission and innovation of ICH. Using Canton Porcelain as a case study, we collaborated with experts to design EnamelAI Painter that simplifies the participation process while promoting creativity. In a study with 19 participants, this system demonstrated advantages over existing AI and non-AI tools, enhancing creative engagement and user experience. EnamelAI Painter balanced GenAI automation with young novices’ engagement, preserving cultural authenticity and fostering interest in ICH. This work contributes a human-GenAI co-creation framework that provides actionable recommendations for the digital preservation of ICH.

The transition from conceptual sketch to 3D model poses a major challenge in engineering design. While Mixed Reality (MR) enables direct 3D creation, designers face two key challenges: the Initiation Challenge (lack of meaningful starting points) and the Expression Challenge (difficulty in precise mid-air sketching). We introduce GenerativeSurface, leveraging AI to translate natural language descriptions into interactive 3D surfaces that aid designers in 3D sketching, simultaneously addressing both challenges. Our system transforms the design workflow into a fluid "describe, generate, sketch, iterate" cycle. Comparative evaluation against primitive scaffolding and freehand mid-air sketching demonstrates that our approach effectively bridges the gap between designers' conceptual intent and 3D sketch expression in immersive environments.

Micro-gestures provide a natural, efficient, and privacy-preserving input modality; however, existing techniques often depend on environmental conditions, limiting their robustness and applicability in real-world settings. In this work, we present three portable prototypes- - finger-cot, finger-worn, and surface-based-that integrate compact pressure array sensors to support environment-independent micro-gesture interaction. We further propose a deep learning-based recognition model that accurately classifies 14 micro-gestures by analyzing temporal pressure patterns. Building upon these components, we introduce PortInput, a real-time interactive system that enables robust micro-gesture tracking and detection. We conducted two user studies with augmented reality (AR) head-mounted displays (HMDs). The first study evaluates input performance under both sitting and walking conditions, while the second compares PortInput with a commercial pressure-based ring device. The results show that PortInput improves usability and user experience, achieves comparable accuracy, and enables faster input with lower perceived workload. Overall, PortInput have potential to offer efficient, robust, and comfortable input across diverse application scenarios-ranging from AR/Virtual Reality (VR) headsets to smart homes and in-car systems-even in noisy or cluttered environments. This work provides a foundation for integrating pressure array sensors into ring-based or other portable devices, advancing always-available micro-gesture interaction for ubiquitous computing environments.

The N-back task is widely recognized as a paradigm for cognitive training due to its adaptability and effectiveness in mitigating working memory decline. However, its repetitive and monotonous design often leads to reduced engagement and poor long-term adherence, lowering intervention effectiveness. To address these limitations, researchers have turned to game-based interventions and immersive technologies to enhance user motivation and sustain training participation. In this study, we propose VRN-back, a music rhythm-based gamified cognitive training system in virtual reality (VR). The system combines the adaptive N-back paradigm with rhythm-game mechanics to increase enjoyment and motivation, while immersive interaction fosters engagement and enhances training outcomes. We conducted a seven-day study with 15 young adults to evaluate the system's impact on their working memory, transfer to related cognitive domains, and user experience. Results showed significant improvements in N-back accuracy, reaction time, and maximum N level, as well as transfer effects on attention and inhibitory control tasks. Subjective evaluations indicated good usability and user experience, with the System Usability Scale scoring 80.83/100, the short User Experience Questionnaire rating pragmatic quality as "Good" and hedonic quality as "Above Average", and the Pleasure-Arousal-Dominance scale confirming sustained positive emotional states. Simulator sickness remained minimal before and after training, ensuring usability. These findings suggest the feasibility and effectiveness of immersive rhythm-based gamified VR systems for cognitive training, laying a foundation for future research on long-term cognitive intervention technologies.

This study explores the design space of two-handed input (i.e., clicking or tapping with the thumb) on the touchpads of controllers for virtual reality (VR) interaction. Four experiments were conducted to fulfill this purpose. Experiment 1 investigated how users employed two VR controllers to perform four representative interaction tasks in VR and identified 14 potentially usable two-handed operations that involved tapping or clicking. Experiments 2 and 3 analyzed user performance of the 14 operations, providing insights into their interaction characteristics in terms of completion time, accuracy, and subjective feedback. In Experiment 4, we designed a command-input technique based on the proposed operations. We verified its effectiveness compared to context menus and marking menus in a VR text entry scenario. Our technique generally had shorter times and similar accuracy to the two menu types. Our work contributes to the design of VR interactions using two-handed controllers.

Various locomotion techniques, such as teleportation, walking in place, redirected walking, and walking, have been proposed. However, conventional methods overlooked the human ability to coordinate multiple modalities (e.g., eyes and feet) for natural virtual locomotion within a small physical space. Inspired by the natural coordination of the eyes and feet in human walking, as well as insights from prior work, this study investigates how eye-foot coordination can be leveraged more effectively for VR teleportation. We present four novel teleportation techniques based on eye-foot coordination, using users' gaze behavior and 3D foot positions as input modalities. A user study with 20 participants compared our techniques with a state-of-the-art foot-based locomotion method. Results demonstrate the superiority of our approaches: task completion times were significantly reduced, NASA-TLX workload scores and SUS usability scores were markedly improved, and participants expressed a clear preference for our techniques over the baseline. These findings provide a strong foundation for the design and implementation of future eye-foot coordinated teleportation methods in VR.

Investigating users' plausible preferences for body-centric locomotion can help researchers better understand the impact of different factors on this locomotion method and optimize the locomotion configuration for providing a plausible locomotion experience. In this paper, we propose to evaluate users' plausible preferences for body-centric locomotion using a reinforcement learning method. This method can intelligently infer users' plausible preferences for different factor levels involved in the virtual locomotion by proposing possible modifications to the factor levels and asking users to accept or reject the modifications after experiencing the locomotion. We conducted a within-subject experiment to examine the impact of different factors (i.e., body parts used for virtual locomotion, the point of view, auditory feedback, the transfer function, and the coefficients of the transfer function) on users' plausible preferences for body-centric locomotion in sitting and standing postures. The results mainly indicated that (1) users preferred using arm swinging in standing posture, whereas they preferred using head tilting in sitting posture; (2) The point of view was identified as the most important factor in standing posture, while it was less important in sitting posture; (3) Participants showed consistent plausible preferences for auditory feedback, transfer function, and coefficient of the transfer function in both standing and sitting postures. Our research findings can guide the design of VR applications to enhance a plausible walking experience in different postures.

Immersive Virtual Environment (IVE) have demonstrated substantial potential in user research. However, the integration of IVE into Journey Map (JM) creation remains unexplored, as traditional JM creation still predominantly relies on 2D data. To bridge this research gap, we integrated IVE into the JM creation process, aiming to explore the potential of applying IVE in the JM creation process. Through a formative study with design experts ($\mathrm{N}=14$), six design goals were identified, reflecting designers' expectations for incorporating IVE into JM creation. Based on these goals, we developed ImmerJM, a tool that enables designers to analyze and document user journeys directly within immersive 3D environments. We conducted a comparative evaluation with User Experience (UX) participants ($\mathrm{N}=20$) to assess the JM creation process of ImmerJM against a traditional screen-based method. The quantitative and qualitative analyses revealed key improvements (e.g., phase-based JM mapping) in journey process and outputs. Our findings suggest that ImmerJM advances JM methodologies by leveraging spatial immersion, offering crucial implications for future JM creation and broader UX design practices involving IVE technologies.

Advances in artificial intelligence have led to breakthroughs in large language models (LLMs), like ChatGPT, opening up exciting possibilities for conceptual design. However, it’s essential to gain an in-depth understanding of how LLMs impact conceptual design output, process, and human designers’ perception. To this end, we chose ChatGPT as an example and conducted the investigation with 30 participants divided into Human-LLMs groups and human-human groups. The results indicated that there was no significant difference between their outputs, but the incorporation of LLMs shortened the completion time with fewer design steps and less time allocated to the late stages of design. Despite being perceived as less efficient and trusted, LLMs can still be viewed as potential collaborators, with humans holding the leadership. These findings offer the HCI community a thorough comprehension of how LLMs influence creativity-related practices, providing valuable insights for designing future interactions with LLMs.

In this research, we investigate the effectiveness of asymmetric interactions (HandStylus, HandController, and TwoHands) in Augmented Reality (AR), Virtual Reality (VR), and Extended Reality (XR) for 3D digital drawing overlaying on physical and virtual objects. We evaluate the input accuracy and fatigue of these object-based 3D drawing experiences using quantitative measurements and further explore the correlation between these outcomes with subjective questionnaires. We found significant independence between environments and interface types, which considerably influence the performance and usability of 3D immersive drawing. We noted discrepancies between users’ subjective experiences and objective performance. Specifically, although AR drawing on physical objects provides superior accuracy and minimal muscle fatigue due to tangible feedback, and the TwoHands interaction offers the highest precision, the subjective results show the reverse outcome. Based on these findings, we propose design recommendations and discuss directions for future research in immersive drawing environments.

Body-centric locomotion in Virtual Reality (VR) involves multiple factors, including the point of view, avatar representations, tracked body parts for locomotion control and transfer functions that map body movement to the displacement of the virtual viewpoint. Understanding the role of these factors in evoking a plausible walking experience using within- or between-subject experimental designs based on questionnaires and/or objective measurements can be time-consuming and challenging due to the interrelated effects of these factors. This study employed the subjective matching method to evaluate the sense of plausible walking experience during body-centric locomotion in VR. Five relevant factors that may affect locomotion experience were identified by analyzing existing studies, i.e., point of view, the avatar appearance, body parts for locomotion control, transfer functions and the coefficients of transfer functions. A virtual locomotion experiment with these five factors based on subjective matching was conducted. Results showed that participants regarded the point of view as the most critical factor for walking experience enhancement, followed by body parts, transfer functions, the coefficients of transfer functions and finally the avatar appearance. Additionally, participants’ preferences for different body parts and the coefficients of transfer functions affected the choice of transfer functions. These results could serve as the guidelines for virtual locomotion experience design that involves combinations of multiple factors and can help achieve a plausible walking experience in VR.

ABSTRACT Painting is a complex and creative process that involves the use of various drawing skills to create artworks. The concept of training artificial intelligence models to imitate this process is referred to as neural painting. To enable ordinary people to engage in the process of painting, we propose PainterAR, a novel interface that renders any paintings stroke‐by‐stroke in an immersive and realistic augmented reality (AR) environment. PainterAR is composed of two components: the neural painting model and the AR interface. Regarding the neural painting model, unlike previous models, we introduce the Kullback–Leibler divergence to replace the original Wasserstein distance existed in the baseline paint transformer model, which solves an important problem of encountering different scales of strokes (big or small) during painting. We then design an interactive AR interface, which allows users to upload an image and display the creation process of the neural painting model on the virtual drawing board. Experiments demonstrate that the paintings generated by our improved neural painting model are more realistic and vivid than previous neural painting models. The user study demonstrates that users prefer to control the painting process interactively in our AR environment.

Introduction: Generalized anxiety disorder (GAD) is a chronic uncontrollable excessive anxiety towards a variety of topics. It is one of the most prevalent anxiety diseases, affecting approximately 6% worldwide. Virtual reality (VR) based interventions have received much attention from researchers for the management of mental health problems. This systematic review was conducted to assess the effects of immersive virtual reality techniques and non-immersive techniques in treating patients with GAD.Methods: An extensive systematic literature review was implemented, from inception up to 9 July 2021. Comparative clinical studies that assessed the outcomes of VRT in patients with GAD were included for meta-analysis. Single arm studies were included for systematic review only.Results: A total of six articles with 239 participants with GAD were included in systematic review. The mean age of the included participants ranged from 38.33 to 59.87 years. There was no statistically significant difference between VRT and control groups regarding the anxiety levels (SMD 0.01; 95%CI −0.52, 0.54; P = 0.97), mean depression scores (SMD 1.34; 95%CI −0.80, 3.49; P = 0.22), PSWQ (MD −4.26; 95%CI −10.93, 2.40; P = 0.21), and discomfort status (MD −1.41; 95%CI −3.86, 1.05; P = 0.22).Conclusions: VR improved relatively the manifestations of GAD, reflecting on the patient's quality of life. However, the existing evidence is inconclusive to support the superiority of VR as a substitute for traditional therapies.

Simulator sickness induced by 360° stereoscopic video contents is a prolonged challenging issue in Virtual Reality (VR) system. Current machine learning models for simulator sickness prediction ignore the underlying interdependencies and correlations across multiple visual features which may lead to simulator sickness. We propose a model for sickness prediction by automatic learning and adaptive integrating multi-level mappings from stereoscopic video features to simulator sickness scores. Firstly, saliency, optical flow and disparity features are extracted from videos to reflect the factors causing simulator sickness, including human attention area, motion velocity and depth information. Then, these features are embedded and fed into a 3-dimensional convolutional neural network (3D CNN) to extract the underlying multi-level knowledge which includes low-level and higher-order visual concepts, and global image descriptor. Finally, an attentional mechanism is exploited to adaptively fuse multi-level information with attentional weights for sickness score estimation. The proposed model is trained by an end-to-end approach and validated over a public dataset. Comparison results with state-of-the-art models and ablation studies demonstrated improved performance in terms of Root Mean Square Error (RMSE) and Pearson Linear Correlation Coefficient.

Body-centric locomotion allows users to control both movement speed and direction with body parts (e.g., head tilt, arm swing or torso lean) to navigate in virtual reality (VR). However, there is little research to systematically investigate the effects of body parts for speed and direction control on virtual locomotion by taking in account different transfer functions(L: linear function, P: power function, and CL: piecewise function with constant and linear function). Therefore, we conducted an experiment to evaluate the combinational effects of the three factors (body parts for direction control, body parts for speed control, and transfer functions) on virtual locomotion. Results showed that (1) the head outperformed the torso for movement direction control in task completion time and environmental collisions; (2) Arm-based speed control led to shorter traveled distances than both head and knee. Head-based speed control had fewer environmental collisions than knee; (3) Body-centric locomotion with CL function was faster but less accurate than both L and P functions. Task time significantly decreased from P, L to CL functions, while traveled distance and overshoot significantly increased from P, L to CL functions. L function was rated with the highest score of USE-S, -pragmatic and -hedonic; (4) Transfer function had a significant main effect on motion sickness: the participants felt more headache and nausea when performing locomotion with CL function. Our results provide implications for body-centric locomotion design in VR applications.

Predicting visual attention facilitates an adaptive virtual museum environment and provides a context-aware and interactive user experience. Explorations toward development of a visual attention mechanism using eye-tracking data have so far been limited to 2D cases, and researchers are yet to approach this topic in a 3D virtual environment and from a spatiotemporal perspective. We present the first 3D Eye-tracking Dataset for Visual Attention modeling in a virtual Museum, known as the EDVAM. In addition, a deep learning model is devised and tested with the EDVAM to predict a user’s subsequent visual attention from previous eye movements. This work provides a reference for visual attention modeling and context-aware interaction in the context of virtual museums.

Task-irrelevant distractors affect visuo-motor control for target acquisition and studying such effects has already received much attention in human-computer interaction. However, there has been little research into distractor effects on crossing-based interaction. We thus conducted an empirical study on pen-based interfaces to investigate six crossing tasks with distractor interference in comparison to two tasks without it. The six distractor-related tasks differed in movement precision constraint (directional/amplitude), target size, target distance, distractor location and target-distractor spacing. We also developed and experimentally validated six quantitative models for the six tasks. Our results show that crossing targets with distractors had longer average times and similar accuracy than that without distractors. The effects of distractors varied depending on distractor location, target-distractor spacing and movement precision constraint. When spacing is smaller than 11.27 mm, crossing tasks with distractor interference can be regarded as pointing tasks or a combination of pointing and crossing tasks, which could be better fitted with our proposed models than Fitts’ law. According to these results, we provide practical implications to crossing-based user interface design.

Body-centric locomotion allows users to navigate virtual environments with body parts (e.g. head tilt, arm swing or torso lean). Transfer functions are an important determinant of the locus of such a locomotion method. However, there is little known about the effects of transfer functions on virtual locomotion with different body parts. In this work, we selected four typical transfer functions (linear function: L, power function: P, a piecewise function with constant and linear functions: CL, and a piecewise function with constant and power functions: CP) and four body parts (head, arm, torso, and knee) from existing works, and conducted an experiment to evaluate their effects on virtual locomotion under three distances (5, 10, and 15 m) in Virtual Reality (VR). Results show that (1) CP function generally led to the longest task time with a low rate of failed trials, while CL function had the shortest task time with a high rate of failed trials; (2) body parts significantly affected the rate of failed trials, but not task time and final position offset. Head and torso resulted in the lowest and highest rate of failed trials respectively; (3) body parts did not differ in User Experience Questionnaire-Short (UEQ-S), UEQ-S Pragmatic and UEQ-S Hedonic. L was rated as the highest score for UEQ-S, UEQ-S Pragmatic and UEQ-S Hedonic, but CP had the lowest score. According to the results, we provide implications of designing body-centric locomotion with different transfer functions in VR.

In dynamic optimization problems (DOPs), as the environment changes through time, the optima also dynamically change. How to adapt to the dynamic environment and quickly find the optima in all environments is a challenging issue in solving DOPs. Usually, a new environment is strongly relevant to its previous environment. If we know how it changes from the previous environment to the new one, then we can transfer the information of the previous environment, e.g., past solutions, to get new promising information of the new environment, e.g., new high-quality solutions. Thus, in this paper, we propose a neural network (NN)-based information transfer method, named NNIT, to learn the transfer model of environment changes by NN and then use the learned model to reuse the past solutions. When the environment changes, NNIT first collects the solutions from both the previous environment and the new environment and then uses an NN to learn the transfer model from these solutions. After that, the NN is used to transfer the past solutions to new promising solutions for assisting the optimization in the new environment. The proposed NNIT can be incorporated into population-based evolutionary algorithms (EAs) to solve DOPs. Several typical state-of-the-art EAs for DOPs are selected for comprehensive study and evaluated using the widely used moving peaks benchmark. The experimental results show that the proposed NNIT is promising and can accelerate algorithm convergence.

Visually induced motion sickness (MS) experienced in a 3D immersive virtual environment (VE) limits the widespread use of virtual reality (VR). This paper studies the effects of a saliency detection-based approach on the reduction of MS when the display on a user's retina is dynamic blurred. In the experiment, forty participants were exposed to a VR experience under a control condition without applying dynamic blurring, and an experimental condition applying dynamic blurring. The experimental results show that the participants under the experimental condition report a statistically significant reduction in the severity of MS symptoms on average during the VR experience compared to those under the control condition, which demonstrates that the proposed approach may alleviate visually induced MS in VR and enable users to remain in a VE for a longer period of time.

In a mixed reality (MR) environment that combines the physical objects with the virtual environments, users' feelings are immersed in the virtual world, while their bodies remain in the physical world. Compared to the purely physical environments, such characteristic has led to some special needs for users' long-term immersion. However, the deficiency needs that we have to face for long-term immersion still need further research. In this paper, we apply the theory of Maslow's Hierarchy of Needs (MHN) to guide the design of MR systems for long-term immersion. Taking the normal biological rhythm of human beings as the basic unit (24 hours), we propose the fundamental needs for long-term immersion in VEs through combining the theory of MHN with the special needs of virtual reality (VR). In order to verify whether those needs can satisfy users' long-term immersion, we design an MR office system for basic operations based on the theory of MHN. A long-term exposure experiment (duration of 8 hours) is designed to evaluate those needs by comparing the results with a physical work environment after a short-term preliminary study. The physiological and psychological effects are tested in both two environments and the deficiency needs for short-term immersion and long-term immersion are also compared. The results showed that the design based on the theory of MHN can support users' long-term immersion, which means that it can be a guideline for long-term use of MR systems.

Observing eye movement is a direct way to analyse human’s attention. Eye movement patterns in normal environment have been widely investigated. In virtual reality (VR) environment, previous studies of eye movement patterns are mainly based on content-unrelated influential factors. Considering this issue, in this paper, a novel content-related factor is studied. One crucial kind of region of interest (ROI), namely vision-penetrable entrance, is chosen to analyse eye movement pattern differences. The results suggest that users show more interest in vision-penetrable entrances than in other regions. Furthermore, this kind of difference is identified as higher average density of fixation. As far as we know, this paper is the first attempt to study specific types of ROI in virtual reality environments. The method utilised in this paper can be applied in other ROI analysis.

Accurate recognition of modern and traditional porcelain styles is a challenging issue in Cantonese porcelain management due to the large variety and complex elements and patterns. We propose a hybrid system with porcelain style identification and image recreation modules. In the identification module, prediction of an unknown porcelain sample is obtained by logistic regression of ensembled neural networks of top-ranked design signatures, which are obtained by discriminative analysis and transformed features in principal components. The synthesis module is developed based on a conditional generative adversarial network, which enables users to provide a designed mask with porcelain elements to generate synthesized images of Cantonese porcelain. Experimental results of 603 Cantonese porcelain images demonstrate that the proposed model outperforms other methods relative to precision, recall, area under curve of receiver operating characteristic, and confusion matrix. Case studies on image creation indicate that the proposed system has the potential to engage the community in understanding Cantonese porcelain and promote this intangible cultural heritage.

Mixed reality (MR) is an emerging technology which could potentially shape the future of education. A text mining method was applied to 3330 articles to classify them into six different topics. The classification reveals the trends relating to each topic and the relations between them. This paper focuses on recent developments in each topic area and summarizes the advantages and disadvantages of using MR for education. Our objective is to help researchers focus on the research challenges where technological advancements in education MR are most needed.

In 2008, Zhou et al. presented a survey paper summarizing the previous ten years of ISMAR publications, which provided invaluable insights into the research challenges and trends associated with that time period. Ten years later, we review the research that has been presented at ISMAR conferences since the survey of Zhou et al., at a time when both academia and the AR industry are enjoying dramatic technological changes. Here we consider the research results and trends of the last decade of ISMAR by carefully reviewing the ISMAR publications from the period of 2008-2017, in the context of the first ten years. The numbers of papers for different research topics and their impacts by citations were analyzed while reviewing them-which reveals that there is a sharp increase in AR evaluation and rendering research. Based on this review we offer some observations related to potential future research areas or trends, which could be helpful to AR researchers and industry members looking ahead.

There are few negative effects to make people discomfort using virtual reality systems. In this paper, we investigated the effects of visual fatigue when wearing head-mounted displays (HMD) and compared the results with those from the smartphones. Forty subjects were recruited and divided into two different groups. The visual fatigue scale was measured to assess the subjects' performance. The results indicated that visual fatigue caused by the conflict of focal distance and vergence distance was less severe than visual fatigue caused by long-term focus without accommodation.

Curves have long been used for graph visualization with increased popularity in recent years. Curves are mainly used for two purposes: one is to increase readability and the other is to enhance visual aesthetic pleasingness. Although curves can be visually pleasing, the introduction of curves in graph drawing does not increase readability automatically. Attempts have been made to investigate the usability of curved drawings. However, the results on the effect of curves per se on human graph comprehension has not been conclusive. This paper presents a user study that is to examine the effect of curves when they are introduced to remove crossings. Twenty-six participants were recruited to perform typical graph reading tasks. Task performance and user preference data were collected for analysis. The results indicate that curves can be a useful alternative when crossings are to be present in straight-line drawings. The findings of the study are also discussed along with some of our future research activities in this paper.

Increasing number of older adults manage their retirement savings online. A crucial element of better management is to take rational financial risk -- to strike a reasonable balance between expected gain and loss under uncertainty. With the emergence of Web 2.0 technologies, social trading networks can help individuals make better financial decisions by providing information about others' actions. It is, however, unclear whether these resources is beneficial to older adult's own financial decisions, especially because older adults are vulnerable to poor risk management. To address this question, we devise an experiment that improves upon an existing experimental economic task. We find that both peer information (detailed choices by a few individuals) and majority information (aggregated choices of the crowd) help older adults make more risk-neutral decisions. Furthermore, the combination of peer and majority information corrects more mistakes of more risk-averse older adults.

This paper presents 'DoDo's Catching Adventure,' a new color vision deficient screening test for young children. Early detection of color blindness among children is useful for parents and teachers to better understand children's needs, to overcome difficulties in learning, and for life and career planning. Unfortunately, current color screening tests are not designed for young children; most require more advanced verbal or cognitive skills. DoDo game has taken a new approach by embedding game elements into a color vision screening test. A user study conducted at Singapore National Eye Centre on twenty-eight children, identified fourteen as Red-Green deficient subjects as did by Ishihara screening test, showed that DoDo was adequately effective in identifying Red-Green color vision deficiency and comparable to two current gold standard colorblind tests, Ishihara and D15.

Goal-oriented visual search is performed when a person intentionally seeks a target in the visual environment. In augmented reality (AR) environments, visual search can be facilitated by augmenting virtual cues in the person's field of view. Traditional use of explicit AR cues can potentially degrade visual search performance due to the creation of distortions in the scene. An alternative to explicit cueing, known as subtle cueing, has been proposed as a clutter-neutral method to enhance visual search in video-see-through AR. However, the effects of subtle cueing are still not well understood, and more research is required to determine the optimal methods of applying subtle cueing in AR. We performed two experiments to investigate the variables of scene clutter, subtle cue opacity, size, and shape on visual search performance. We introduce a novel method of experimentally manipulating the scene clutter variable in a natural scene while controlling for other variables. The findings provide supporting evidence for the subtlety of the cue, and show that the clutter conditions of the scene can be used both as a global classifier, as well as a local performance measure.

Goal-oriented visual search in augmented reality can be facilitated by using visual cues to call attention to a target. However, traditional use of explicit cues can degrade visual search performance due to scene distortion, occlusion and addition of visual clutter. In contrast, Subtle Cueing has been previously proposed as an alter-native to explicit cueing, but little is known about how well it works for head-tracked head worn displays (HWDs). We investigated the effect of Subtle Cueing for head-tracked head worn displays, using visual search research methods in simulated augmented reality environments. Our user study found that Subtle Cueing improves visual search performance, and serves as a feasible cueing mechanism for AR environments using HWDs.

The purpose of this study was to investigate how a mobile collaborative augmented reality (AR) system affects learners¡¦ knowledge construction behaviors and learning performances. In this study, 20 undergraduate students were recruited and divided into dyads to discuss given questions with the assistance of mobile collaborative AR system named AR Physics. The participants¡¦ knowledge regarding elastic collision was evaluated through a pretest and a posttest that occurred right after the treatment. Learners¡¦ knowledge construction behaviors were qualitatively identified according to a selected coding scheme and then were analyzed by adopting sequential analysis. The results indicated that learners significantly gained their knowledge on the topic of elastic collision after completing the given task by manipulating AR Physics system. Furthermore, sequential patterns of learners¡¦ behaviors during knowledge construction activities were identified.

This paper attempts to explore students¡¦ collaborative learning behavior patterns in different simulation systems. Two mobile CSCL learning environments (Augmented Reality simulation and Traditional simulation) for helping university students to acquire physics knowledge were designed. A total of 40 students, grouped into twenty pairs, were randomly assigned to the two learning environments for conducting an inquiry task. Their collaborative learning behaviors were recorded and analyzed by quantitative content analysis and lag sequential analysis. The results indicated that the students¡¦ collaborative inquiry learning was supported in both environments. Particularly, the students using the AR-simulation showed a more cohesive collaborative inquiry learning behavior pattern. This study provides a new perspective for educators or system designers to deeply observe students¡¦ learning behaviors in a CSCL environment by using sequential analysis. Moreover, the AR-simulation system might be a suitable learning environment for enhancing students¡¦ collaborative inquiry learning. In addition, suggestions for designing a simulated learning environment are proposed.

Visual search in augmented reality environments is an important task that can be facilitated through different cueing methods. Current cueing methods rely on explicit cueing, which can potentially reduce visual search performance. In comparison, this paper proposes a subtle cueing method that improves visual search performance while being clutter-neutral. Two empirical user studies were conducted to evaluate our subtle cueing method in outdoor scenes. The results show that subtle cueing functions well within a narrow Feature Congestion range, and could be a feasible alternative to explicit cueing.

Augmented Reality (AR), using a mixture of the virtual and physical world, has been recognized as a promising environment for improving the quality of collaboration in educational domains. To identify how an AR-supported simulation affects collaborative learning, this study compared collaborative learning effectiveness between the conditions of AR-supported and traditional face-to-face collaborative learning for Physics. The findings revealed that collaborative use of the AR-supported simulation resulted in higher levels of perceived skill development, self-reported learning and learning interest. The individual learning achievement was also strengthened with the use of AR-supported simulation in collaboration. This study provides empirical evidence for supporting the potential value of AR technology in collaborative learning practices and has implications for research on learning effectiveness evaluation in CSCL.

Mobile collaborative augmented reality (AR) technology demonstrates great potential for augmenting learning experience in collaboration. The convergence of AR, collaboration and handheld devices opens up more opportunities for strengthening the effectiveness of social interaction in collaborative learning. We conceptualize it as a new form of collaborative learning, collaborative AR social learning. Based on the reflections of the sociality characteristics of mobile collaborative AR, we develop a model for better understanding this emerging instructional medium. It has implications for the development of mobile collaborative AR applications for educational purposes in the future.

This research demo describes the implementation of a mobile AR-supported educational course application, AR Circuit, which is designed to promote the effectiveness of remote collaborative learning for physics. The application employs the TCP/IP protocol enabling multiplayer functionality in a mobile AR environment. One phone acts as the server and the other acts as the client. The server phone will capture the video frames, process the video frame, and send the current frame and the markers transformation matrices to the client phone.

The paper describes a Yoga training system that is built based on motion replication technique (MoRep), including hardware, virtual scenario and feedback design. The motion replication technique proposed here can determine the similarity between Yoga master and student's postures and then provide feedback on the incorrect body posture of the student through multimodal channels. The key innovations of this project are also discussed.

10.1145/1753326.1753424

In this paper, a mobile approach for ambient learning is presented. As an example of the concept, modeling is done to deliver educational content on weather phenomenon to schoolchildren. Information visualization techniques are utilized to present an alternative approach of knowledge delivery that could be considered to be beyond conventional classroom based teachings. The concept is demonstrated with the `Interactive Weather Information System' (IWIS) platform. During the user tests, the effectiveness of the proposed method on knowledge conveyance was evaluated. Usability tests showed that IWIS serves as a good stimulus to captivate the learning interests of the target user group (children from 10 to 12 years old). In addition, the new issues that surfaced could be used as design considerations or factors in subsequent research work in this area.

Evidence from prior research has demonstrated that exocentric views of the environment can facilitate the acquisition of survey knowledge in a virtual environment. The present study examined the effect of different exocentric views on judging the relative direction of objects. During the participants' vertical movement in a virtual room, participants learned the spatial layout in one of three conditions: two-perspective, attentive-elevation, and normal-elevation conditions, where the number of the exocentric perspectives from which the spatial layout was observed was different. After spatial learning, they made the judgment of the relative direction of objects. The analysis of spatial judgment showed that as the number of exocentric perspectives increased, the accuracy improved in the mental representation of spatial vertical information and spatial information in novel directions. Results indicated that the increased number of exocentric perspectives during the vertical movement facilitated the flexible acquisition of survey knowledge. Applications of this study included the design of effective navigation aids in virtual multilevel buildings.

This paper simulates bonding effects inside pedestrian crowds. Based on the social force model, this paper derives an exponential formulation of the bonding force, as opposed to the repulsive force, and surveys the degree of interpersonal cohesion under various circumstances. Parameters associated with the model are calibrated by preliminary simulation runs. With the proper simulation environment configuration, the effect of the bonding force is extensively demonstrated. Results show that the bonding force results in pedestrians' walking speeds being different from their initial intended ones. Specifically, delays in walking and the overtaking phenomenon, which are empirically observed, are explained using this model. In the zigzag walkway defined in the experiment, up to approximately 4% fewer pedestrians are able to escape in the prescribed time, due to bonding effects. To sum up, the bonding forces cause negative effects on pedestrian evacuation and should be taken into consideration for crowd dynamics research.

In this paper, we present a novel sensing technique, optical linear encoder (OLE), in which the motion of an optical encoder on a reflective strip is converted to limb joints' goniometric data. A body sensing module is designed to integrate the OLE and an accelerometer. A sensor network of three sensing modules is established via Controller Areas Network (CAN) bus to capture full motion of human arm with a 7-DOF kinematic model. In addition, a statistical study was conducted to confirm the repeatability and reliability of our sensor network. Results demonstrate that the sensor system has strong potential to be used as a low-cost tool for motion capture, and objective arm function evaluation for both short-term and long-term monitoring.

Information empowers those who make sense from its successful interpretation; this is especially true on the subject of weather where the recipient is required to have prior forms of fundamental understanding towards the natural occurring phenomenon and its effects. This study facilitates the learning of complex weather phenomena for children using a unique educational approach using mobile technology. The initiative runs under Singapore's National Weather Study Project (NWSP), which already operates a network of mini weather stations in schools across the state to promote environmental awareness and support dynamic interactions with real time data. Its data is accessible via Web, Microsoft SensorMap and Google Earth interfaces. IWIS, an application for the recent Google Android mobile platform taps onto the fusion of mapping and the weather project server's data to present to children accessible local multi-variate weather data depicted using visual metaphors and propagates the development of mobile learning applications in the imminent future.

This paper develops a physical model of an inertial/magnetic measurement unit by effectively integrating an accelerometer, a magnetometer, and two gyroscopes for low-g motion tracking applications. The proposed model breaks down the errors contributed by individual components, then determines error elimination methods based on sensor behavior and characteristics, and finally constructs a feedback loop for continuous self-calibration. Measurement errors are reduced by adopting a systematic design methodology: 1) tilt errors are minimized through a careful selection of A/D convertor resolution and by making compensation on sensor bias and scale factor; 2) heading errors are reduced by cancelling out nearby ferrous distortions and making tilt-compensation on the magnetometer; 3) errors from gyroscope measurements are eliminated via the least squares algorithm and continuous corrections using orientation data at the steady-state position. Preliminary tests for low-g motion sensing show that the motion tracker can achieve less than ±0.5° accuracy in tilt and less than ±1° accuracy in yaw angle measurement with above-mentioned methods.

A novel self-calibrating sensing technology using miniature linear encoders and Inertial/magnetic Measurement Unit (IMU) provides the accuracy, fast response and robustness required by many body motion processing applications. Our sensor unit consists of an accelerometer, a 3-axis magnetic sensor, 2 gyroscopes and a miniature linear encoder. The fusion of data from the sensors is accomplished by extracting the gravity related term from the accelerometer and consistently calibrating the gyroscopes and linear encoder when the sensor unit is under static conditions. Using the fused sensors, we developed a complete motion processing system that consists of a gateway where the human kinematics modeling is embedded. A time divided multiple access wireless architecture is adopted to synchronize the sensor network at 100Hz. Experimental results show that the combination of the IMU and linear encoder produces a low RMS error of 3.5° and correlation coefficient of 99.01%. A video showing the capture a performer’s upper body motion is also realized.

Although Augmented Reality technology was first developed over forty years ago, there has been little survey work giving an overview of recent research in the field. This paper reviews the ten-year development of the work presented at the ISMAR conference and its predecessors with a particular focus on tracking, interaction and display research. It provides a roadmap for future augmented reality research which will be of great value to this relatively young field, and also for helping researchers decide which topics should be explored when they are beginning their own studies in the area.

With the growing popularity of mobile phone gaming, design issues with mobile phone games have become more important. Given the complexity of designing games for mobile phones, game developers must understand how mobile phone users' gaming experiences are like when using different types of phones. This paper examines users' gaming experiences in playing three types of games on three different types of phones. It is found that users have different preferences when playing different games on certain phones. Some games are more suitable to be played on mobile phone than others. Easy control and fewer levels of difficulties are the two main issues for consideration in designing mobile phone games. It is recommended that game designers design specific types of games for different mobile phone interfaces. Mobile phone manufacturers need to consider gamers' requirements for the type of games they intend to include in the mobile phone.

A robotic puppet is developed for studying motion generation and control of highly articulated biomimic mechatronic systems with anatomical motion data of human in real time. The system is controlled by a pair of data gloves tracking human fingers' actions. With the primitives designed in a multilayered motion synthesis structure, the puppet can realize some complex human-like actions. Continuous full body movements are produced on the robotic puppet by combining and sequencing the actions on different body parts using temporal and spatial information provided by the data gloves. Human is involved in the interactive design of the coordination and timing of the body movements of the robotic puppet in a natural and intuitive manner. The methods of motion generation exhibited on the robotic puppet may be applied to the interactive media, entertainment and biomedical engineering.

Usability testing of mobile devices is an emerging area of research in the field of Human-Computer Interaction. Guidelines had been established as to how usability tests should be conducted. However, there are limitations to the effectiveness of conventional usability tests, especially for mobile devices. Mobile devices typically are used in different situations, but current evaluation method cannot uncover problems in all type of situations. Hence, this study evaluates the effectiveness of conventional laboratory usability tests. It investigated the differences between usability tests on mobile phones conducted in laboratory and real life situation. Significant differences were found, including the frequency and severity of usability problems encountered, the users' behavior, and subjective responses to the device and the interaction.

The present study investigates the displays on the operation desk in two locomotive cabs. Planning the layout of displays provides the fundamental understanding for the design of operation desk. Before planning the layout, designers should know the relevant importance of each display. A method based on fuzzy logic to rank the displays on the operation desk in locomotive cabs was developed. After evaluating the factors which affect display unit rank, the proposed method takes six different fuzzy operators to comprehensively synthesize the results in order to compensate the integration provided by a mixed fuzzy operator. The result has been demonstrated to be rational in the practical design.

The effects of field-of-view (FOV) in a virtual environment (VE) on presence, enjoyment, memory and simulator sickness (SS) were studied. A refined scale, designed to assess subjects' engagement, enjoyment and immersion (E/sup 2/I), was developed. Items to examine subjects' memory of the VE were included. SS was examined using the Simulator Sickness Questionnaire (SSQ). Using a within-subjects design, data were collected from 10 subjects at four FOVs (60/spl deg/, 100/spl deg/, 140/spl deg/ and 180/spl deg/). The VE, called "Crayolaland", was presented in a driving simulator. The results indicated that presence, enjoyment and SS varied as a function of the display FOV. Subjects exhibited higher SSQ and presence subscale scores with increasing FOV. SSQ and presence values approached asymptotes for FOVs beyond 140/spl deg/. Presence and SS were positively correlated, while enjoyment and SS were negatively correlated.

Simulator sickness (SS) / virtual environment (VE) sickness is expected to become increasingly troublesome as VE technology evolves [20]. Procedures to alleviate SS / VE sickness have been of limited value [12]. This paper investigated a possible procedure to reduce SS and VE sickness. Postural disturbance was evoked by visual scene motion at different frequencies. Differences in disturbance were examined as a function of simultaneous exposure to an “independent visual background” (IVB). Eight subjects were tested at two scene motion frequencies and three different IVB conditions using a within-subjects design. An expected statistically significant interaction between IVB condition and frequency was observed. For low frequency scene movements, subjects exhibited less balance disturbance when the IVB was presented. We suggest that an IVB may alleviate disturbance when conflicting visual and inertial cues are likely to result in simulator or VE sickness.

With technological progress, wide field-of-view (FOV) displays will become increasingly common. Wide FOVs provide a more immersive environment and produce stronger self-motion perception. The objective of this study was to investigate the relationships between FOV and scene content on postural stability in an immersive environment. 10 subjects were tested using two different scenes (a simple radial pattern and a "meaningful" city scene) at six FOVs (30/spl deg/, 60/spl deg/, 90/spl deg/, 120/spl deg/, 150/spl deg/ 180/spl deg/) using a within-subjects design. Subjects exhibited more postural disturbance with increasing FOV. A surprisingly large increase in disturbance was found for the interval between 150/spl deg/ and 180/spl deg/ using the city scene. No statistically significant difference was found for effects of scene content. Two groups (postural stable group and postural unstable group) were identified during experiment. These groups performed differently in the two scene conditions. Future research plans are described.