The N-back task is widely recognized as a paradigm for cognitive training due to its adaptability and effectiveness in mitigating working memory decline. However, its repetitive and monotonous design often leads to reduced engagement and poor long-term adherence, lowering intervention effectiveness. To address these limitations, researchers have turned to game-based interventions and immersive technologies to enhance user motivation and sustain training participation. In this study, we propose VRN-back, a music rhythm-based gamified cognitive training system in virtual reality (VR). The system combines the adaptive N-back paradigm with rhythm-game mechanics to increase enjoyment and motivation, while immersive interaction fosters engagement and enhances training outcomes. We conducted a seven-day study with 15 young adults to evaluate the system's impact on their working memory, transfer to related cognitive domains, and user experience. Results showed significant improvements in N-back accuracy, reaction time, and maximum N level, as well as transfer effects on attention and inhibitory control tasks. Subjective evaluations indicated good usability and user experience, with the System Usability Scale scoring 80.83/100, the short User Experience Questionnaire rating pragmatic quality as "Good" and hedonic quality as "Above Average", and the Pleasure-Arousal-Dominance scale confirming sustained positive emotional states. Simulator sickness remained minimal before and after training, ensuring usability. These findings suggest the feasibility and effectiveness of immersive rhythm-based gamified VR systems for cognitive training, laying a foundation for future research on long-term cognitive intervention technologies.

Accurately representing and rendering dynamic scenes over time remains a central challenge in neural rendering and computer graphics. Existing dynamic Gaussian-based methods often suffer from limited temporal consistency, flickering under fast motion, and poor adaptability to non-human structures. To address these issues, we propose DG-4DGS, a deformation-graph-constrained 4D Gaussian splatting framework for temporally stable dynamic rendering. The method anchors all Gaussians in a canonical space and enforces cross-frame geometric alignment through a deformation graph. Based on neighborhood-consistency features, a multi-head residual decoder refines position, rotation/scale, and color attributes to achieve fine-detail fidelity without relying on online densification or pruning. Compared with 4DGS and avatar-based approaches, DG-4DGS achieves higher PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index measure) scores and significantly smaller model size on both the TalkBody4D (human) and Horse (non-human) datasets. It effectively suppresses temporal flickering and cross-frame drift in high-frequency regions such as hair strands, cloth wrinkles, and limb extremities. The framework does not depend on parametric templates, facilitating extension to non-human and complex clothing scenarios, though its performance still depends on deformation-tracking quality and neighborhood topology selection.

Inter-generational communication plays a vital role in bridging generational gaps and fostering mutual understanding. However, it remains challenging due to differences in cultural norms, communication styles, and geographical separation. While prior studies have shown Virtual Reality (VR) as a medium that fosters a relaxed atmosphere and companionship, its capacity to address the nuanced dynamics of inter-generational dialogue, such as divergent values and relational intricacies, remains limited. To address this gap, we explored the opportunities of Artificial Intelligence (AI) to support inter-generational communication in VR. We developed three technology probes (e.g., Content Generator, Communication Facilitator, and Info Assistant) in VR and employed them in a probe-based participatory design study with twelve inter-generational pairs. Our results show that AI-powered VR facilitates inter-generational communication by enhancing mutual understanding, fostering conversation fluency, and promoting active participation. We also introduce several challenges when using AI-powered VR in supporting inter-generational communication and derive design implications for future VR platforms, aiming to improve inter-generational communication.

Demonstrative learning in virtual reality (VR) is a pivotal learning strategy for knowledge transfer for embodied agents. While existing studies have extensively explored agents’ knowledge transfer through self-demonstrative learning (SDL) or teacher-demonstrative learning (TDL), there has been limited focus on a system that integrates both paradigms. This article proposes a comprehensive system that combines the SDL paradigm with the TDL paradigm in VR from a top-down perspective. The system involves using directed probabilistic graphs (DPGs) for knowledge representation, constructing detectors and actuators based on object fluents and atomic actions, representing knowledge acquired from both learning paradigms on the DPGs, and incorporating knowledge integration and visualization. Through system evaluation, we show the advantages of integrating two demonstrative learning paradigms, including increased learning efficiency, mitigating demonstrator’s deficiencies, and more logical task execution. The study also reveals that a dynamically decreasing fusion factor with the learning progresses, performs well across different correct percentages of teacher demonstrations. Additionally, we show a more decentralized demonstration in the middle of the agent’s learning progress maximizing learning efficiency when demonstrating only a few atomic actions. Finally, we assess the system’s generalizability and delineate its boundaries. With the ongoing development and the increasing availability of human data in VR, we anticipate that our system can be applied to future scenarios of efficient human-agent knowledge transfer in human-agent symbiosis.

The decline of cognitive inhibition significantly impacts older adults’ quality of life and well-being, making it a vital public health problem in today’s aging society. Previous research has demonstrated that Virtual reality (VR) exergames have great potential to enhance cognitive inhibition among older adults. However, existing commercial VR exergames were unsuitable for older adults’ long-term cognitive training due to the inappropriate cognitive activation paradigm, unnecessary complexity, and unbefitting difficulty levels. To bridge these gaps, we developed a customized VR cognitive training exergame (LightSword) based on Dual-task and Stroop paradigms for long-term cognitive inhibition training among healthy older adults. Subsequently, we conducted an eight-month longitudinal user study with 12 older adults aged 60 years and above to demonstrate the effectiveness of LightSword in improving cognitive inhibition. After the training, the cognitive inhibition abilities of older adults were significantly enhanced, with benefits persisting for 6 months. This result indicated that LightSword has both short-term and long-term effects in enhancing cognitive inhibition. Furthermore, qualitative feedback revealed that older adults exhibited a positive attitude toward long-term training with LightSword, which enhanced their motivation and compliance.

This paper presents PinchText, a mid-air technique with a condensed keys-based keyboard, which combines hand positions and pinch gestures, enabling one-handed text entry for Head-mounted displays (HMDs). Firstly, we conduct Study 1 to collect and analyze the typing data of PinchText with two arm postures and two movement directions, obtaining the range of hand position corresponding to the middle key set. Then, we conduct Study 2, a 6-block experiment, finding that PinchText with Hand-Up Vertical (UpV) and Hand-Down Vertical (DownV) modes could achieve a speed of 12.71 words-per-minute (WPM) and 11.14 WPM respectively with both uncorrected error rates less than 0.5%, which is 71% faster than the index finger pinch-based technique. Finally, Study 3 is conducted to explore the potential of reducing the size of the decoupled visual keyboard of PinchText, verifying that the occlusion of the virtual keyboard can be decreased. Overall, PinchText is an efficient, easy-to-learn, and comfortable text entry technique for HMDs.

Physical entity interactions in mixed reality (MR) environments aim to harness human capabilities in manipulating physical objects, thereby enhancing virtual environment (VEs) functionality. In MR, a common strategy is to use virtual agents as substitutes for physical entities, balancing interaction efficiency with environmental immersion. However, the impact of virtual agent size and form on interaction performance remains unclear. Two experiments were conducted to explore how virtual agent size and form affect interaction performance, immersion, and preference in MR environments. The first experiment assessed five virtual agent sizes (25%, 50%, 75%, 100%, and 125% of physical size). The second experiment tested four types of frames (no frame, consistent frame, half frame, and surrounding frame) across all agent sizes. Participants, utilizing a head-mounted display, performed tasks involving moving cups, typing words, and using a mouse. They completed questionnaires assessing aspects such as the virtual environment effects, interaction effects, collision concerns, and preferences. Results from the first experiment revealed that agents matching physical object size produced the best overall performance. The second experiment demonstrated that consistent framing notably enhances interaction accuracy and speed but reduces immersion. To balance efficiency and immersion, frameless agents matching physical object sizes were deemed optimal. Virtual agents matching physical entity sizes enhance user experience and interaction performance. Conversely, familiar frames from 2D interfaces detrimentally affect interaction and immersion in virtual spaces. This study provides valuable insights for the future development of MR systems.

This paper proposes a framework that utilizes skinning decomposition to automatically generate facial rigging from 4D expressions. The framework inputs a predefined rigging template and an actor's 4D facial expressions, including a neutral expression, as well as a group of arbitrary expressions. The output includes not only the linear blend skinning weights and joint positions of the actor's head mesh but also other facial components such as teeth and eyes. Compared to traditional methods, this paper applies a soft constraint to optimize joint positions and imposes a fixed sparsity distribution constraint to improve weight distribution. To further enhance rigging efficiency, this paper leverages GPU expression-parallel and CPU vertex-parallel strategies for joint transformation and weight updates, respectively. The experiments show that the proposed method generates high-fidelity facial rigging that outperforms existing solutions in terms of computational speed, joint position correctness, weight distribution correctness, or computational cost.

National Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), China Retrieving out-of-reach objects is a crucial task in virtual reality (VR). One of the most commonly used approaches for this task is the gesture-based approach, which allows for bare-hand, eyes-free, and direct retrieval. However, previous work has primarily focused on assigned gesture design, neglecting the context. This can make it challenging to accurately retrieve an object from a large number of objects due to the one-to-one mapping metaphor, limitations of finger poses, and memory burdens. There is a general consensus that objects and contexts are related, which suggests that the object expected to be retrieved is related to the context, including the scene and the objects with which users interact. As such, we propose a commonsense knowledge-driven joint reasoning approach for object retrieval, where human grasping gestures and context are modeled using an And-Or graph (AOG). This approach enables users to accurately retrieve objects from a large number of candidate objects by using natural grasping gestures based on their experience of grasping physical objects. Experimental results demonstrate that our proposed approach improves retrieval accuracy. We also propose an object retrieval system based on the proposed approach. Two user studies show that our system enables efficient object retrieval in virtual environments (VEs).

Virtual reality (VR) has become a powerful and promising tool for education, and numerous studies have investigated the application and effectiveness of VR education. However, few studies have focused on the expectations and concerns of teenagers regarding head-mounted displays (HMDs), which are used for this purpose. In this paper, we aim to explore the current problems and necessary advancements required in VR education based on a survey of 163 senior high school students who experience VR educational content for 1h. The usability and comfort of the HMD system, the physical and psychological effects on the students, and their preferences and concerns are investigated. The results show that HMDs increase students' interest, concentration, and enthusiasm for learning. However, isolated virtual environments make students feel nervous and afraid. The immersive environment also makes them worry about VR addiction and confusing the physical world with the virtual one. VR has great potential in the field of education, but the issue of safety needs to be considered in the future.

To investigate the effects of visual discomfort caused by long-term immersing in virtual environments (VEs), we conducted a comparative study to evaluate users’ visual discomfort in an eight-hour working rhythm and compared the differences between the VEs and the physical environments. Twenty-seven participants performed four different visual tasks with a head-mounted display (HMD) for the VE condition and with a monitor for the physical condition. Their subjective visual discomfort and objective oculomotor indicators were measured to evaluate their visual performances. The results show that the subjective visual fatigue symptoms, the objective pupil size, and the relative accommodation response vary across time for the two conditions, in which VEs affects visual fatigue the most compared to the physical environments. The results also show that pupil size is negatively related to subjective visual fatigue, and the long-term work based on displays only influences the maximum accommodation response of participants. This work is a supplement to the necessary but insufficient-researched field of visual fatigue in long-term immersing in VEs, which should be valuable to researchers involved in the evaluation of visual fatigue using HMDs.

Text entry in virtual reality (VR) is currently a common activity and a challenging problem. In this paper, we introduce HiPad, leveraging a circular touchpad with a circular virtual keyboard, to support the one-hand text entry in mobile head-mounted displays (HMDs). The design of HiPad’s layout is based on a circle and a square with rounded corners, where the outer circle is subdivided into six keys’ regions containing letters. This technique input text by a common hand-held controller with a circular touchpad for HMDs and disambiguates the word based on the sequence of keys pressed by the user. In our first study, three potential layouts are considered and evaluated, leading to the design containing six keys. By analyzing the touch behavior of users, we optimize the 6-keys layout and conduct the second study, showing that the optimized layout has better performance. Then the third study is conducted to evaluate the performance of 6-keys HiPad with VE-layout and TP-layout and to study the learning curves. The results show that novices can achieve 13.57 Words per Minute (WPM) with VE-layout and 11.60 WPM with TP-layout and the speeds increase by 74.42% for VE-layout users and by 81.53% for TP-layout users through a short 60-phrase training.

Different domain image sensors or imaging mechanisms provide cross-domain images when sensing the same scene. There is a domain shift between cross-domain images so that the image gap between different domains is the major challenge for measuring the similarity of the feature descriptors extracted from different domain images. Specifically, matching ground camera images and unmanned aerial vehicle (UAV) 3-D model-rendered images, which are two kinds of extremely challenging cross-domain images, is a way to establish indirectly the spatial relationship between 2-D and 3-D spaces. This provides a solution for the virtual-real registration of augmented reality (AR) in outdoor environments. However, during matching, handcrafted descriptors and existing learning-based feature descriptors limit the rendered images. In this letter, first, to learn robust and invariant 128-D local feature descriptors for ground camera and rendered images, we present a novel network structure, SiamAM-Net, which embeds the autoencoders with an attention mechanism into the Siamese network. Then, to narrow the gap between the cross-domain images during the optimizing of SiamAM-Net, we design an adaptive margin for the loss function. Finally, we match the ground camera-rendered images by using the learned local feature descriptors and explore the outdoor AR virtual-real registration. Experiments show that the local feature descriptors, learned by SiamAM-Net, are robust and achieve state-of-the-art retrieval performance on the cross-domain image data set of ground camera and rendered images. In addition, several outdoor AR applications also demonstrate the usefulness of the proposed outdoor AR virtual-real registration.

Human-computer interaction (HCI) plays an important role in the near-field mixed reality, in which the hand-based interaction is one of the most widely-used interaction modes, especially in the applications based on optical see-through head-mounted displays (OST-HMDs). In this paper, such interaction modes as gesture-based interaction (GBI) and physics-based interaction (PBI) are developed to construct a mixed reality system to evaluate the advantages and disadvantages of different interaction modes. The ultimate goal is to find an efficient hybrid paradigm for mixed reality applications based on OST-HMDs to deal with the situations that a single interaction mode cannot handle. The results of the experiment, which compares GBI and PBI, show that PBI leads to a better performance of users regarding their work efficiency in the proposed two tasks. Some statistical tests, including T-test and one-way ANOVA, have also been adopted to prove that the difference regarding the efficiency between different interaction modes is significant. Experiments for combining both interaction modes are put forward in order to seek a good experience for manipulation, which proves that the partially-overlapping style would help to improve work efficiency for manipulation tasks. The experimental results of the proposed two hand-based interaction modes and their hybrid forms can provide some practical suggestions for the development of mixed reality systems based on OST-HMDs.

In a mixed reality (MR) environment that combines the physical objects with the virtual environments, users' feelings are immersed in the virtual world, while their bodies remain in the physical world. Compared to the purely physical environments, such characteristic has led to some special needs for users' long-term immersion. However, the deficiency needs that we have to face for long-term immersion still need further research. In this paper, we apply the theory of Maslow's Hierarchy of Needs (MHN) to guide the design of MR systems for long-term immersion. Taking the normal biological rhythm of human beings as the basic unit (24 hours), we propose the fundamental needs for long-term immersion in VEs through combining the theory of MHN with the special needs of virtual reality (VR). In order to verify whether those needs can satisfy users' long-term immersion, we design an MR office system for basic operations based on the theory of MHN. A long-term exposure experiment (duration of 8 hours) is designed to evaluate those needs by comparing the results with a physical work environment after a short-term preliminary study. The physiological and psychological effects are tested in both two environments and the deficiency needs for short-term immersion and long-term immersion are also compared. The results showed that the design based on the theory of MHN can support users' long-term immersion, which means that it can be a guideline for long-term use of MR systems.

Long-term exposure to VR will become more and more important, but what we need for long term immersion to meet users fundamental needs is still under-researched. In this paper, we apply the theory of Maslows Hierarchy of Needs to guide the design of VR for longterm immersion based on the normal biological rhythm of human beings (24 hours). An office environment is designed to verify those needs. The efficiency, the physical and the psychological effects of this VR office system are tested. The results show that the VR office environment is as comfortable as the physical environment at short-term immersion and it can support users basic immersion. It means that the Maslows Hierarchy of Needs can be a guideline for long-term immersion.

This paper presents a novel virtual-real registration approach for augmented reality (AR) in large-scale outdoor environments. Essentially, it is a pose estimation for the mobile camera images (ground camera images) in 3D model recovered by Unmanned Aerial Vehicle (UAV) image sequence via Structure-From-Motion (SFM) technology. The approach considers to indirectly establish the spatial relationship between 2D and 3D space by inferring the transformation relationship between the ground camera images and the UAV 3D model rendered images. Specifically, the proposed approach can overcome the positioning errors, which are deterioration and drift in the GPS, and deviation of orientation. The experimental results demonstrate the possibility of the proposed virtual-real registration approach, and show that the approach is robust, efficient and intuitive for AR in large-scale outdoor environments.

The virtual reality (VR) and augmented reality (AR) applications have been widely used in a variety of fields; one of the key requirements in a VR or AR system is to understand how users perceive depth in the virtual environment and augmented reality. This paper conducts an experiment to compare users' performance of depth perception in VR and AR system using an optical see-through head-mounted display (HMD). The result shows that the accuracy of depth estimation in AR is higher than in VR. Besides, the matching error increases as the distance becomes farther.

In this paper, we review the background of physical reality, virtual reality, and some traditional mixed forms of them. Based on the current knowledge, we propose a new unified concept called symmetrical reality to describe the physical and virtual world in a unified perspective. Under the framework of symmetrical reality, the traditional virtual reality, augmented reality, inverse virtual reality, and inverse augmented reality can be interpreted using a unified presentation. We analyze the characteristics of symmetrical reality from two different observation locations (i.e., from the physical world and from the virtual world), where all other forms of physical and virtual reality can be treated as special cases of symmetrical reality.

Optical tracking has become the most commonly used virtual reality (VR) tracking technology because of its high precision and non-contact characteristics. The optical tracking system represented by HTC VIVE has the problem that signals of base stations interfere with each other, and the number of base stations cannot be extended by cascades, thereby limiting the scope of its work. In this paper, an extensible optical tracking system is proposed, which can distinguish the signals from different base stations and support the simultaneous operation of multiple base stations. Furthermore, we designed an encoding scheme to generate independent code for up to 32 base stations and proposed a highspeed decoding method. Experiments demonstrate that the system has high tracking accuracy and low system latency. Users can adjust the number and layout of base stations according to the actual demand, which greatly improves the flexibility of the system, and benefits promoting the development of large scale optical tracking equipment with low cost and high precision.

Since artificial intelligence has been integrated into virtual reality, a new branch of virtual reality, which is called inverse virtual reality (IVR), is created. A typical IVR system contains both the intelligence-driven virtual reality and the physical reality, thus constructing an intelligence-driven mutually mirrored world. We propose the concept of IVR, and describe the details about the definition, structure and implementation of a typical IVR system. The parallel living environment is proposed as a typical application of IVR, which reveals that IVR has a significant potential to extend the human living environment.

Hand-based interaction is one of the most widely-used interaction modes in the applications based on optical see-through head-mounted displays (OST-HMDs). In this paper, such interaction modes as gesture-based interaction (GBI) and physics-based interaction (PBI) are developed to construct a mixed reality system to evaluate the advantages and disadvantages of different interaction modes for near-field mixed reality. The experimental results show that PBI leads to a better performance of users regarding their work efficiency in the proposed tasks. The statistical analysis of T-test has been adopted to prove that the difference of efficiency between different interaction modes is significant.

Calibration accuracy is one of the most important factors to affect the user experience in mixed reality applications. For a typical mixed reality system built with the optical see-through head-mounted display (OST-HMD), a key problem is how to guarantee the accuracy of hand-eye coordination by decreasing the instability of the eye and the HMD in long-term use. In this paper, we propose a real-time latent active correction (LAC) algorithm to decrease hand-eye calibration errors accumulated over time. Experimental results show that we can successfully use the LAC algorithm to physics-inspired virtual input methods.

There are few negative effects to make people discomfort using virtual reality systems. In this paper, we investigated the effects of visual fatigue when wearing head-mounted displays (HMD) and compared the results with those from the smartphones. Forty subjects were recruited and divided into two different groups. The visual fatigue scale was measured to assess the subjects' performance. The results indicated that visual fatigue caused by the conflict of focal distance and vergence distance was less severe than visual fatigue caused by long-term focus without accommodation.

The most commonly used single point active alignment method (SPAAM) is based on a static pinhole camera model, in which it is assumed that both the eye and the HMD are fixed. This leads to a limitation for calibration precision. In this work, we propose a dynamic pinhole camera model according to the fact that the human eye would experience an obvious displacement over the whole calibration process. Based on such a camera model, we propose a new calibration data acquisition method called the region-induced data enhancement (RIDE) to revise the calibration data. The experimental results prove that the proposed dynamic model performs better than the traditional static model in actual calibration.

Yuanmingyuan is a relic park and only few cultural relics are left due to the looting and burning down in history, which makes that most of the scenic spots of the park look boring. To address such issue, a game-based guidance system for Yuanmingyuan and a time travel game called MAGIC-EYES has been proposed with Augmented Reality technology. Six interactive modes are designed in the proposed system to guide tourists to visit the specified place. The evaluation results of a pilot study shows that the proposed guidance system has significantly improved the tourist experiences.

We have developed omnidirectional-view 3D displays using multiple mini-projectors. Three types of synchronization structure are developed to ensure the accurate synchronization between the projectors and the rotating screen. Therefore, low-cost and low-speed display devices can be used to realize natural-looking three-dimensional (3D) scene with full color and high resolution.

The technology used to distort any projected image to fit the uneven-shaped physical screens perfectly is a key issue to the application of Spatial Augmented Reality. We proposed an image generation technique based on Structured Light, which can generate the image of physical screen at the projector angle. Through the image, we thus get the accurate matching information between the projected image and the physical screen. It can be used as the initial image for people to debug remotely and conveniently on the worksite. This paper also confirmed the high stability and accuracy of the technology on different physical screens with different materials.

We introduce a general technique of dynamically augmenting physical models. Using a handheld projector attached with an RGBD camera, we map the physical model with corresponding textures, and develop a real-time dynamic spatial augmented reality (SAR) system named iSarProjection. Compared to traditional huge and complex tracking system, this technique, based on KinectFusion, supports precise registration between the physical model point cloud and the reconstructed 3D scene, and provides real-time pose estimation to track the physical display surface effectively. Through projecting medical anatomy visualizations on a white body model, it presents that we ease the development of dynamic SAR system and make it possible to be applied in various research fields.

This paper introduces a novel and simple approach to realtime continuous keystone correction for hand-held projectors. An RGBD camera is attached to the projector to form a projector-RGBD-camera system. The system is first calibrated in an offline stage. At run-time, we then estimate the relative pose between the projector and the screen using the RGBD camera, which lets us correct the keystone distortion by warping the projected image accordingly. Experimental results show that our method outperforms existing techniques in terms of both accuracy and efficiency.

In this paper, we presented two prototype systems to analyze the role of touch in AR system. One was an AR evaluation prototype system used to design the aircraft cockpit. In this system, the pilots can touch the virtual buttons and meters to test the ergonomics performance of design results. The other was a touchable AR girl prototype system used to treat heterosexual-social anxiety. In this system, patients can touch and feel a virtual girl. We designed a series of subjective and objective experiments to prove the importance of “touchable” to enhancing the user experience in AR system.

Majority of present applications use buttons or touch panels as their human-computer interface. The limitation of these methods is that users can not interact with the equipments in a natural way, since people should operate the equipments according to certain rules which are arranged beforehand. This paper proposes a novel interaction system which contains a low cost full-field of view device to make sure users can see the display content from all directions. In addition, a tracking device is set up over users head to identify the user's natural gesture. A target following photography equipment is also combined with the system to provide on-line video chat and remote monitoring functions. Our system can be considered as a new form of human-computer interface and the advantages and potential are discussed in this paper. Finally, a demonstration and experimental results are given to evaluate the validity of the system.

This paper introduces a novel augmented reality shooting game named “Soul Hunter”, which has been successfully operating in a theme park in China. Soul Hunter adopts an innovative infrared marker scheme to build a mobile augmented reality application in a wide area. It is an extension of the traditional first person game, in which a player is able to fight with virtual ghost through a gunlike device in real environment. This paper describes the challenges of applying augmented reality in theme parks and shares some experiences in solving the problems encountered in practical applications.

Augmented reality (AR) Adventure Game is a wide-area indoor AR application for multiple users in Guangdong science center. Tracking the pose of users’ head in wide area is crucial for alignment between virtual and real scene. This paper studies the key issues of wide-area indoor tracking system. Different from previous inside-looking-out vision-based tracking, coded infrared (IR) markers are installed both on walls and ceiling in the proposed system. An automatic method is designed to calibrate the transformation between tracking camera and scene camera. The linear algorithm used for pose estimation is presented and problems of registration in rendering engine are also discussed. Experiments are conducted and applications in the AR Adventure Game prove that the proposed tracking system can provide precise and stable registration in actual systems.

A novel human-computer interaction (HCI) system based on real-time image processing is proposed in this paper. With the help of infrared tracking technology, the proposed system achieves real-time processing and stable operation on an ADSP-BF533 hardware platform. Compared with the conventional remote control methods, the proposed system enables a user to control the cursor on the screen of a TV by targeting it, and provides users with new experiences of remote control. To realize such a new remote control system, cross-ratio invariant, which is an important characteristic of the projective transformation, is also studied. Experimental results show the potential of the proposed system in TV remote control.

This paper presents a GPU-based real-time method to correct optical distortions in head-mounted displays (HMDs). The HMD to be corrected is a lightweight and wide field-of-view HMD system with free-form-surface (FFS) prism, in which the image distortion is not rectilinear and centrosymmetric. A special predistortion model is constructed to correct the distortion of the HMD. Although the distortion correction can be performed with an extensional optics system, the system will be too expensive and additional weight will be imposed to the HMD. With the method presented in this paper, each pixel in the original image is remapped to a new position with GPU and forms a predistortion image. The remapping process is based on a prior formatted distortion map, which is similar to the normal map used in the bump mapping process. The distortion map is an RGBA image that corresponds to the X and Y coordinates of a pixel offset from the original image. The remapping process accomplished by GPU is very fast. The performance of the proposed method is analyzed and validated via a demonstration system.

An indoor tracking system based on primary and assistant infrared markers is presented in this paper. The system can track the user's head in a large area with high stability and accuracy. And the price of the system is very low. The novel assistant infrared markers with particular spatial characteristic and the primary infrared markers with particular spatio-temporal characteristic are proposed, which can avoid the synchronization between infrared markers and user system. Various numbers of users can be supported by the proposed system and experimental result shows the effectiveness and robustness of the system.