Searching for information, objects, or places in virtual reality (VR) is often a cumbersome process that breaks user immersion. Existing interaction techniques like pointing or voice commands are context-dependent, yet current systems fail to help users select the optimal strategy for a given situation. To address this challenge, we introduce an adaptive VR search framework that recommends search strategies based on situational factors. We first propose a taxonomy of five common VR search strategies and construct a factorial dataset linking task types, object properties, and environmental conditions to strategy choices. We then conduct a two-stage study. In Stage 1, we capture users' natural strategy selections in a free-search environment to create a ground-truth dataset. In Stage 2, we use this data to train and evaluate two adaptive systems, one driven by a machine learning (ML) model and another by a Large Language Model (LLM), against the free-search baseline. Our results show that both adaptive systems significantly reduce the number of attempts and perceived workload compared to free search. While the ML-based system achieved the fastest task completion times, both adaptive approaches were rated as significantly more usable. This work demonstrates that adaptive, context-aware systems can enhance search efficiency and user experience in VR, paving the way for more intelligent and immersive information-seeking interfaces.

This paper presents a mobile augmented reality (AR) system designed to enhance user experiences at cultural heritage (CH) sites by providing personalized content and points of interest (POI) recommendations. Applying semantic relatedness, we address the heterogeneity of CH POIs to improve personalization accuracy. Our system constructs a POI-based co-occurrence graph to model semantic relationships, enriching users’ visited AR content items for better recommendations. The proposed method integrates content-filtering-based recommendations with this graph and recommends personalized POIs and AR content items. A user study demonstrated that our system outperforms conventional methods by 9.4% in recommendation precision and recall, significantly enhancing user engagement and attention focus during CH tours.

A 3D avatar typically has one of six cardinal facial expressions. To simulate realistic emotional variation, we should be able to render a facial transition between two arbitrary expressions. This study presents a new framework for instruction-driven facial expression generation that produces a 3D face and, starting from an image of the face, transforms the facial expression from one designated facial expression to another. The Instruction-driven Facial Expression Decomposer (IFED) module is introduced to facilitate multimodal data learning and capture the correlation between textual descriptions and facial expression features. Subsequently, we propose the Instruction to Facial Expression Transition (I2FET) method, which leverages IFED and a vertex reconstruction loss function to refine the semantic comprehension of latent vectors, thus generating a facial expression sequence according to the given instruction. Lastly, we present the Facial Expression Transition model to generate smooth transitions between facial expressions. Extensive evaluation suggests that the proposed model outperforms state-of-the-art methods on the CK+ and CelebV-HQ datasets. The results show that our framework can generate facial expression trajectories according to text instruction. Considering that text prompts allow us to make diverse descriptions of human emotional states, the repertoire of facial expressions and the transitions between them can be expanded greatly. We expect our framework to find various practical applications. More information about our project can be found athttps://vohoanganh.github.io/tg3dfet/

Object placement in Augmented Reality (AR) is crucial for creating immersive and functional experiences. However, a critical research gap exists in combining user input with efficient automated placement, particularly in understanding spatial relationships and optimal placement. This study addresses this gap by presenting a novel object placement pipeline for AR applications that balances automation with user-directed placement. The pipeline employs entity recognition, object detection, depth estimation along with spawn area allocation to create a placement system. We compared our proposed method against manual placement in a comprehensive evaluation involving 50 participants. The evaluation included user experience questionnaires, a comparative study of task performance, and post-task interviews. Results indicate that our pipeline significantly reduces task completion time while maintaining comparable accuracy to manual placement. The UEQ-S and TENS scores revealed high user satisfaction. While manual placement offered more direct control, our method provided a more streamlined, efficient experience. This study contributes to the field of object placement in AR by demonstrating the potential of automated systems to enhance user experience and task efficiency.

We present a new network whereby an agent navigates in the 3D environment to find a target object according to a language-based instruction. Such a task is challenging because the agent has to understand the instruction correctly and takes a series of actions to locate a target among others without colliding with obstacles. The essence of our proposed network consists of a coarse-to-fine fusion model to fuse language and vision and an autoencoder to encode visual information effectively. Then, an asynchronous reinforcement learning algorithm is used to coordinate detailed actions to complete the navigation task. Extensive evaluation using three different levels of the navigation task in the 3D Vizdoom environment suggests that our model outperforms the state-of-the-art. To see if the proposed network can deal with a real-world 3D environment for the navigation task, it is combined with Rec-BERT, which is based on REVERIE. The result suggests that it performs better, especially for unseen cases, and it is also useful to visualize what and when the agent pays attention to while it navigates in a complex indoor environment.

Abstract Synthesizing facial wrinkles has been tackled either by a long process of manual sculpting on 3D models, or using automatic methods that do not allow for user interaction or artistic expression. In this paper, we propose a method that accepts interactive sketchy drawings depicting wrinkle patterns, and synthesizes realistic looking wrinkles on faces. The method inherits the simplicity of sketching, making it possible for artists as well as novice users to generate realistic facial detail very efficiently, allowing fast preview for physical makeup, or aging simulations for fun and professional applications. All strokes are used to infer the wrinkles, retaining the expressiveness of the sketches and realism of the final result at the same time. This is achieved by designing novel multi‐scale statistics tailored to the wrinkle geometry and coupled to the sketch interpretation method. The statistics capture the cross‐sectional profiles of wrinkles at different scales and parts of a face. The strokes are augmented with the statistics extracted from given example face models, and applied to an input face model interactively. The interface gives the user control over the shapes and scales of wrinkles via sketching while adding extra details required for realism automatically.

ABSTRACT Recent advances in facial scanning technology provide highly detailed faces, including fine wrinkles. However, because of the increasing complexity of the resulting models, it is necessary to reduce the amount of data while preserving small‐scale facial features. In this paper, we propose a new adaptive surface splatting method to reduce the number of splats by optimizing the size and shape of splats using geometric and color information. Using the optimized splats, we can improve the rendering quality, especially in the visually sensitive feature areas. Our adaptive surface splatting is very effective to render massive facial data on tablet PCs or smartphones. Copyright © 2012 John Wiley & Sons, Ltd.

Abstract Facial symmetry is one of the number of aesthetic traits associated with health, physical attractiveness and beauty of a person. In this paper, we present a novel method for enhancing the symmetry of a scanned 3D face automatically. To handle detailed areas of the face, we developed a new local 3D shape descriptor based on facial features and surface curvatures. Our shape descriptor can improve the accuracy when deforming a 3D face towards a symmetric configuration, because it provides accurate point pairing with respect to the plane of symmetry. It can also reduce the computational complexity by dividing the given face into partitioned slices and facial feature regions. In addition, we use point‐based representation over all stages of symmetrization without generating a consistent triangle mesh or texture parameterization, which makes it much easier to support discrete processes such as symmetric deformation and to be combined with more sophisticated manipulations later such as cutting of surfaces. Finally, we performed a statistical analysis to assess subjects' preference for the symmetrized faces by our approach. Copyright © 2009 John Wiley & Sons, Ltd.

Abstract We present a time‐varying deformable model to visualize and analyze the motion of the left ventricle from a time series of 3‐D images. The model is composed of a non‐rigid body that deforms around a reference shape obtained from the previous time step. At each time step, the position and orientation of the left ventricle are extracted from the feature points of images. This information gives the position and orientation of the coordinate system attached to the non‐rigid body. To compute a dense non‐rigid motion field over the entire endocardial wall of the left ventricle, we introduce a 3‐D blob finite element and Galerkin interpolants based on 3‐D Gaussian, and use a physically based finite element method and a modal analysis. Then, cinematic attributes are visualized in pseudo colors on the reconstructed surface in order to help medical doctors in their interpretation of the data. Using the presented model, we estimate clinically useful quantitative parameters such as regional wall motion and ejection fraction. Experimental results are shown in a time series of X‐ray angiographic images. Copyright ©2001 John Wiley & Sons, Ltd.

A number of segmentation algorithms have been developed, but those algorithms are not effective on volume reconstruction because they are limited to operating only on two-dimensional (2-D) images. In this paper, we propose the volumetric object reconstruction method using the three-dimensional Markov random field (3D-MRF) model-based segmentation. The 3D-MRF model is known to be one of efficient ways to model spatial contextual information. The method is compared with the 2-D region growing scheme under three types of interpolation. The results show that the proposed method is better in the aspect of image quality than other methods.