Multi-view visualizations have gained prominence for their ability to simultaneously present multiple perspectives in a single display, aiding users in making informed decisions. Although several tools have been developed to facilitate the design of multi-view visualizations, there is a lack of support for effective color design in these systems. Thoughtful color design can enhance readability and provide cues, while careless color choices may lead to confusion. Moreover, human short-term memory imposes constraints on the number of colors that can be effectively employed in a multi-view visualization. To address these challenges, we introduce MVcolor, a color encoding recommendation system designed to maintain color encoding consistency and ensure adequate color discriminability within the constraints of human short-term memory. Our approach employs a unified color scheme across the entire multi-view visualization and groups views and visual objects based on their visual and semantic similarities. We conducted user studies to evaluate the effectiveness of our system and demonstrate its ability to improve color encoding in multi-view visualizations.

Chatbots hold promise as a technology to aid in sustained weight management. However, determining the optimal way for chatbots to deliver advice to effectively change user behaviors remains a significant hurdle. This research investigates the effects of different chatbot communication styles and expressions of care on user satisfaction, misinterpretation, and intent to adhere to the advice in weight-related conversations. A mixed method study with 97 participants classified as overweight was conducted, dividing them into four groups based on explicit/implicit communication styles and the presence or absence of caring language. Surprisingly, the study found that most participants in the explicit communication groups viewed the chatbot as non-offensive. These participants also reported higher levels of enjoyment and a greater intention to follow the chatbot's recommendations. Utilizing caring language may diminish users' perception of the chatbot as a marketing tool, thereby increasing their willingness to interact. The article discusses the implications for the design of healthcare chatbots.

In immersive virtual environments (IVEs), accurate size perception is critical, especially in training simulations designed to mimic real-world tasks, such as, nuclear power plant control room or medical procedures. These simulations have dials or instruments of varying sizes. Visual information of the objects alone, often fails to capture subtle size differences in virtual reality (VR). However, integrating haptic and hand-avatars may potentially improve accuracy and performance. This improvement could be especially beneficial for real-world scenarios where hand(s) are intermittently visible or obscured. To investigate how this intermittent presence or absence of body-scaled hand-avatars affects size perception when integrated with haptic information, we conducted 2x2 mixed-factorial experiment design using a near-field, size-estimation task in VR. The experiment conditions compared size estimations with or without virtual hand visibility in the perception and reporting phases. The task involved 16 graspable objects of varying sizes and randomly repeated 3 times across 48 trials per participant (total 80 participants). We employed Linear Mixed Models (LMMs) analysis to objective measures: perceived size, residual error and proportional errors. Results revealed that as the tangible-graspable size increases, overestimation reduces if the hand-avatars are visible in the reporting phase. Also, overestimation reduces as the number of trials increases, if the hand-avatars are visible in the reporting phase. Thus, the presence of hand-avatars facilitated perceptual calibration. This novel study, with different combinations of hand-avatar visibility, taking perception and reporting of size as two separate phases, could open future research directions in more complex scenarios for refined integration of sensory modalities and consequently enhance real-world application performance.

In driving scenarios, motion segmentation is a crucial and fundamental component that is needed for many tasks. Recently, a self-supervised multitasking framework was proposed for driving scenarios. It simultaneously trains motion segmentation, optical flow, depth, and ego-motion models without annotated data. The self-supervised architecture derives training signals from training data via loss functions. If these loss functions lack robustness, they may result in model inaccuracies. To reduce the bad influences of occlusion and optical flow estimation errors on motion segmentation, we propose two loss functions: (1) Soft-Per-Pixel-Minimum (Soft-PPM) loss that excludes occluded pixels while balancing the contribution of each frame on the loss function temporally; (2) Flow difference loss that excludes pixels with unclear motion states to diminish the effect of optical flow estimation errors. Our loss function design is based on the key insight that information such as depth and optical flow can be used to train motion segmentation models and act as a reliable measure for pixels during training. Our approach can improve segmentation accuracy for both moving and static objects and has achieved IoU scores on moving and static classes comparable to the state-of-the-art methods on the KITTI dataset.

Recent years have seen a shift towards learning-based methods for trajectory prediction, with challenges remaining in addressing uncertainty and capturing multi-modal distributions. This paper introduces Temporal Ensembling with Learning-based Aggregation, a meta-algorithm designed to mitigate the issue of missing behaviors in trajectory prediction, which leads to inconsistent predictions across consecutive frames. Unlike conventional model ensembling, temporal ensembling leverages predictions from nearby frames to enhance spatial coverage and prediction diversity. By confirming predictions from multiple frames, temporal ensembling compensates for occasional errors in individual frame predictions. Furthermore, trajectory-level aggregation, often utilized in model ensembling, is insufficient for temporal ensembling due to a lack of consideration of traffic context and its tendency to assign candidate trajectories with incorrect driving behaviors to final predictions. We further emphasize the necessity of learning-based aggregation by utilizing mode queries within a DETR-like architecture for our temporal ensembling, leveraging the characteristics of predictions from nearby frames. Our method, validated on the Argoverse 2 dataset, shows notable improvements: a 4% reduction in minADE, a 5% decrease in minFDE, and a 1.16% reduction in the miss rate compared to the strongest baseline, QCNet, highlighting its efficacy and potential in autonomous driving.

Scanning LiDAR is one of the widely used sensors in autonomous vehicles; however, the inherent sparsity of LiDAR point clouds often affects its performance. To address this issue, upsampling methods could be employed to enhance low-resolution LiDAR data. Although there have been methods on upsampling of single-object point clouds recently in computer vision, they tend to generate a considerable amount of artifacts when dealing with real-world LiDAR scenes consisting of multiple objects. In this paper, we propose a solution to tackle this problem by introducing an instance embedding auxiliary task and a context attention module. With our auxiliary learning architecture, the network can learn features that benefit both the primary upsampling task and the auxiliary instance embedding task. This training design enables the point generation process to be carried out separately and significantly reduces artifacts of the upsampling results on the SemanticKITTI dataset, particularly in areas surrounding instances. By leveraging these techniques to improve the model’s understanding of the relationship between objects and the background in LiDAR scenes, we achieve an overall 4% to 10% improvement in whole-scene upsampling.

Spatial perception in virtual reality (VR) has been a hot research topic for years. Most of the studies on this topic have focused on visual perception and distance perception. Fewer have examined auditory perception and room size perception, although these aspects are important for improving VR experiences. Recently, a number of studies have shown that perception can be calibrated to information that is relevant to the successful completion of everyday tasks in VR (such as distance estimation and spatial perception). Also, some recent studies have examined calibration of auditory perception as a way to compensate for the classic distance compression problem in VR. In this paper, we present a calibration method for both visual and auditory room size perception. We conducted experiments to investigate how people perceive the size of a virtual room and how the accuracy of their size perception can be calibrated by manipulating perceptible auditory and visual information in VR. The results show that people were more accurate in perceiving room size by means of vision than in audition, but that they could still use audition to perceive room size. The results also show that during calibration, auditory room size perception exhibits learning effects and its accuracy was greatly improved after calibration.

While existing visualization systems excel in exploring datasets and discovering data patterns and insights, challenges remain in automatically generating infographics from exploration-derived visualizations. We propose VisCollage, a computational pipeline that automatically organizes and renders charts from an exploration in a “visual collage”, which is inspired by data journalism and can be viewed as a kind of “partitioned poster infographic”. By analyzing the relation (e.g., drill-down or comparison) between charts established during exploration, VisCollage groups and merges them to reduce data redundancy. In addition, VisCollage automatically identifies a main chart of the exploration and arranges annotations and background charts around it. User studies evaluated from the perspectives of creators, professional data journalists, and general readers indicate that our system assists creators in generating satisfactory visualization summaries of data events, enables the general audience to extract insights from the data through visual collages, and are well received by professionals.

The perception of distance is a complex process that often involves sensory information beyond that of just vision. In this work, we investigated if depth perception based on auditory information can be calibrated, a process by which perceptual accuracy of depth judgments can be improved by providing feedback and then performing corrective actions. We further investigated if perceptual learning through carryover effects of calibration occurs in different levels of a virtual environment’s visibility based on different levels of virtual lighting. Users performed an auditory depth judgment task over several trials in which they walked where they perceived an aural sound to be, yielding absolute estimates of perceived distance. This task was performed in three sequential phases: pretest, calibration, posttest. Feedback on the perceptual accuracy of distance estimates was only provided in the calibration phase, allowing to study the calibration of auditory depth perception. We employed a 2 (Visibility of virtual environment) $\times 3$ (Phase) $\times 5$ (Target Distance) multi-factorial design, manipulating the phase and target distance as within-subjects factors, and the visibility of the virtual environment as a between-subjects factor. Our results revealed that users generally tend to underestimate aurally perceived distances in VR similar to the distance compression effects that commonly occur in visual distance perception in VR. We found that auditory depth estimates, obtained using an absolute measure, can be calibrated to become more accurate through feedback and corrective action. In terms of environment visibility, we find that environments visible enough to reveal their extent may contain visual information that users attune to in scaling aurally perceived depth.

Mixed reality (MR) interactions feature users interacting with a combination of virtual and physical components. Inspired by research investigating aspects associated with near-field interactions in augmented and virtual reality (AR & VR), we investigated how avatarization, the physicality of the interacting components, and the interaction technique used to manipulate a virtual object affected performance and perceptions of user experience in a mixed reality fundamentals of laparoscopic peg-transfer task wherein users had to transfer a virtual ring from one peg to another for a number of trials. We employed a 3 (Physicality of pegs) X 3 (Augmented Avatar Representation) X 2 (Interaction Technique) multi-factorial design, manipulating the physicality of the pegs as a between-subjects factor, the type of augmented self-avatar representation, and the type of interaction technique used for object-manipulation as within-subjects factors. Results indicated that users were significantly more accurate when the pegs were virtual rather than physical because of the increased salience of the task-relevant visual information. From an avatar perspective, providing users with a reach envelope-extending representation, though useful, was found to worsen performance, while co-located avatarization significantly improved performance. Choosing an interaction technique to manipulate objects depends on whether accuracy or efficiency is a priority. Finally, the relationship between the avatar representation and interaction technique dictates just how usable mixed reality interactions are deemed to be.

Active exploration in virtual reality (VR) involves users navigating immersive virtual environments, going from one place to another. While navigating, users often engage in secondary tasks that require attentional resources, as in the case of distracted driving. Inspired by research generally studying the effects of task demands on cybersickness (CS), we investigated how the attentional demands specifically associated with secondary tasks performed during exploration affect CS. Downstream of this, we studied how increased attentional demands from secondary tasks affect spatial memory and navigational performance. We discuss the results of a multi-factorial between-subjects study, manipulating a secondary task's demand across two levels and studying its effects on CS in two different sickness-inducing levels of an exploration experience. The secondary task's demand was manipulated by parametrically varying $n$ in an aural $n$-back working memory task and the provocativeness of the experience was manipulated by varying how frequently users experienced a yaw-rotational reorientation effect during the exploration. Results revealed that increases in the secondary task's demand increased sickness levels, also resulting in a higher temporal onset rate, especially when the experience was not already highly sickening. Increased attentional demand from the secondary task also vitiated navigational performance and spatial memory. Overall, increased demands from secondary tasks performed during navigation produce deleterious effects on the VR experience.

Cybersickness (CS) is one of the challenges that has hindered the widespread adoption of Virtual Reality (VR). Consequently, researchers continue to explore novel means to mitigate the undesirable effects associated with this affliction, one that may require a combination of remedies as opposed to a solitary stratagem. Inspired by research probing into the use of distractions as a means to control pain, we investigated the efficacy of this countermeasure against CS, studying how the introduction of temporally time-gated distractions affects this malady during a virtual experience featuring active exploration. Downstream of this, we studied how other aspects of the VR experience are affected by this intervention. We discuss the results of a between-subjects study manipulating the presence, sensory modality, and nature of periodic and short-lived (5-12 seconds) distractor stimuli across four experimental conditions: 1) no-distractors (ND); 2) auditory distractors (AD); 3) visual distractors (VD); 4) cognitive distractors (CD). Two of these conditions (VD and AD) formed a yoked control design wherein every matched pair of 'seers' and 'hearers' was periodically exposed to distractors that were identical in terms of content, temporality, duration, and sequence. In the CD condition, each participant had to periodically perform a 2-back working memory task, the duration and temporality of which was matched to distractors presented in each matched pair of the yoked conditions. These three conditions were compared to a baseline control group featuring no distractions. Results indicated that the reported sickness levels were lower in all three distraction groups in comparison to the control group. The intervention also increased the amount of time users were able to endure the VR simulation and avoided causing detriments to spatial memory and virtual travel efficiency. Overall, it appears that it may be possible to make users less consciously aware and bothered by the symptoms of CS, thereby reducing its perceived severity.

Knowledge graphs have been commonly used to represent relationships between entities and are utilized in the industry to enhance service qualities. As knowledge graphs integrate data from a variety of sources, they can also be useful references for data analysts. However, there is a lack of effective tools to make the most of the rich information in knowledge graphs. Existing knowledge graph exploration systems are ineffective because they did not consider various user needs and characteristics of knowledge graphs. Exploratory approaches specifically designed to uncover and summarize insights in knowledge graphs have not been well studied yet. In this article, we propose KGScope that supports interactive visual explorations and provides embedding-based guidance to derive insights from knowledge graphs. We demonstrate KGScope with usage scenarios and assess its efficacy in supporting the exploration of knowledge graphs with a user study. The results show that KGScope supports knowledge graph exploration effectively by providing useful information and helping explore the entire network.

LiDAR detection of long-range vehicles is challenging because very few and sparse points are measured in long distances and vehicles with similar shapes of targets could lead to false positives easily. To tackle these challenges, taking the environment information (HD maps) into account could be beneficial to predetermine where targets are more or less likely to appear. Compared with semantic maps, HD maps formed by point clouds provide much richer information from surrounding static objects and scenes. In this work, we construct a GNN-based feature extraction of point cloud maps to increase the receptive fields of learning map features. Our work is based on PVRCNN, the state-of-the-art LiDAR object detection method. With point-wise and voxel-wise features obtained from PVRCNN, residual feature fusion is proposed to fuse the features from PVRCNN and the map features from GNN. Our approach is evaluated on NuScenes dataset. It achieves a 24.78% average precision improvement for long-range objects at 40–50 meters, the farthest areas with ground truth annotation. Our approach also has a 4.22% reduction of false positives in the entire sensing areas.

We propose LiDAR-Inertial Odometry via Simultaneous EGo-motion estimation and Multiple Object Tracking (LIO-SEGMOT), an optimization-based odometry approach targeted for dynamic environments. LIO-SEGMOT is formulated as a state estimation approach with asynchronous state update of the odometry and the object tracking. That is, LIO-SEGMOT can provide continuous object tracking results while preserving the keyframe selection mechanism in the odometry system. Meanwhile, a hierarchical criterion is designed to properly couple odometry and object tracking, preventing system instability due to poor detections. We compare LIO-SEGMOT against the baseline model LIO-SAM, a state-of-the-art LIO approach, under dynamic environments of the KITTI raw dataset and the self-collected Hsinchu dataset. The former experiment shows that LIO-SEGMOT obtains an average improvement 1.61% and 5.41% of odometry accuracy in terms of absolute translational and rotational trajectory errors. The latter experiment also indicates that LIO-SEGMOT obtains an average improvement 6.97% and 4.21% of odometry accuracy.

Occlusion remains an issue in multiple object tracking, which could cause ambiguity in object detection, such as incorrect or missing detection. Under occlusion, a track could experience an early termination, resulting in identity switches and/or fragmentation. To recover from different lengths of occlusions, the track should be maintained by considering its occlusion status. To address the issues mentioned above, we propose an indicator that can model the track's occlusion extent via geometric information provided by LiDAR data. Through incorporating the indicator into the track management and data association process, it is feasible to prevent tracks from premature termination. The proposed method is evaluated on the collected dataset which undergoes frequent and severe occlusions. Compared to the state-of-the-art probabilistic tracking approach, our approach achieves improvements of 3.26% in MOTA and 5.36% in IDF1. Additionally, we obtain 9.89% improvements in IDF1 specifically for objects experiencing severe occlusions.

Automotive radars have been reliably used in most assisted and autonomous driving systems due to their robustness to extreme weather conditions. With radial velocity measurements from automotive radars, moving targets such as cars, trucks, and buses can be tracked robustly. However, due to the lack of elevation angles in measurements from automotive radars, stationary targets at different heights, such as maintenance holes and bridges, cannot be distinguished. Most autonomous systems rely on sensor fusion or ignore stationary targets to tackle the problem of missing the elevation angle dimension, which derives safety issues. We propose a simple yet effective approach to estimate the elevation angles of stationary targets from relative velocity and radial velocity measurements from an automotive radar. In contrast to structure from motion in computer vision, we utilize the instantaneous velocity generated from the motion of the ego vehicle. The radial velocity of each target is the projection of relative velocity onto the radial direction from radar to target. The radial velocity of each target can be inferred given the target's azimuth, elevation angle, and relative velocity. Accordingly, the elevation angle of each stationary target can be uniquely calculated given the velocity of radar and the target's azimuth and radial velocity measurements. The radar's velocity is estimated with the existing radar odometry algorithm and IMU. The proposed method is verified with real-world data. We evaluate the system's performance with a pre-built point cloud map and a good localization module in a real-world scenario. The proposed elevation angle reconstruction can reach a 1.41-degree mean error and standard deviation of 0.6 degrees in elevation angle.

Abstract Interface icons are prevalent in various digital applications. Due to limited time and budgets, many designers rely on informal evaluation, which often results in poor usability icons. In this paper, we propose a unique human‐in‐the‐loop framework that allows our target users, that is novice and professional user interface (UI) designers, to improve the usability of interface icons efficiently. We formulate several usability criteria into a perceptual usability function and enable users to iteratively revise an icon set with an interactive design tool, EvIcon. We take a large‐scale pre‐trained joint image‐text embedding (CLIP) and fine‐tune it to embed icon visuals with icon tags in the same embedding space (IconCLIP). During the revision process, our design tool provides two types of instant perceptual usability feedback. First, we provide perceptual usability feedback modelled by deep learning models trained on IconCLIP embeddings and crowdsourced perceptual ratings. Second, we use the embedding space of IconCLIP to assist users in improving icons' visual distinguishability among icons within the user‐prepared icon set. To provide the perceptual prediction, we compiled IconCEPT10K, the first large‐scale dataset of perceptual usability ratings over 10,000 interface icons, by conducting a crowdsourcing study. We demonstrated that our framework could benefit UI designers' interface icon revision process with a wide range of professional experience. Moreover, the interface icons designed using our framework achieved better semantic distance and familiarity, verified by an additional online user study.

User representations are critical to the virtual experience, and involve both the input device used to support interactions as well as how the user is virtually represented in the scene. Inspired by previous work that has shown effects of user representations on the perceptions of relatively static affordances, we attempt to investigate how end-effector representations affect the perceptions of affordances that dynamically change over time. Towards this end, we empirically evaluated how different virtual hand representations affect users' perceptions of dynamic affordances in an object retrieval task wherein users were tasked with retrieving a target from a box for a number of trials while avoiding collisions with its moving doors. We employed a 3 (virtual end-effector representation) X 13 (frequency of moving doors) X 2 (target object size) multi-factorial design, manipulating the input modality and its concomitant virtual end-effector representation as a between-subjects factor across three experimental conditions: (1) Controller (using a controller represented as a virtual controller); (2) Controller-hand (using a controller represented as a virtual hand); (3) Glove (using a hand tracked hi-fidelity glove represented as a virtual hand). Results indicated that the controller-hand condition produced lower levels of performance than both the other conditions. Furthermore, users in this condition exhibited a diminished ability to calibrate their performance over trials. Overall, we find that representing the end-effector as a hand tends to increase embodiment but can also come at the cost of performance, or an increased workload due to a discordant mapping between the virtual representation and the input modality used. It follows that VR system designers should carefully consider the priorities and target requirements of the application being developed when choosing the type of end-effector representation for users to embody in immersive virtual experiences.

Inspired by previous works showing promise for AR self-avatarization - providing users with an augmented self avatar, we investigated whether avatarizing users' end-effectors (hands) improved their interaction performance on a near-field, obstacle avoidance, object retrieval task wherein users were tasked with retrieving a target object from a field of non-target obstacles for a number of trials. We employed a 3 (Augmented hand representation) X 2 (density of obstacles) X 2 (size of obstacles) X 2 (virtual light intensity) multi-factorial design, manipulating the presence/absence and anthropomorphic fidelity of augmented self-avatars overlaid on the user's real hands, as a between subjects factor across three experimental conditions: (1) No-Augmented Avatar (using only real hands); (2) Iconic-Augmented Avatar; (3) Realistic Augmented Avatar. Results indicated that self-avatarization improved interaction performance and was perceived as more usable regardless of the anthropomorphic fidelity of avatar. We also found that the virtual light intensity used in illuminating holograms affects how visible one's real hands are. Overall, our findings seem to indicate that interaction performance may improve when users are provided with a visual representation of the AR system's interacting layer in the form of an augmented self-avatar.

Data exploration systems have become popular tools with which data analysts and others can explore raw data and organize their observations. However, users of such systems who are unfamiliar with their datasets face several challenges when trying to extract data events of interest to them. Those challenges include progressively discovering informative charts, organizing them into a logical order to depict a meaningful fact, and arranging one or more facts to illustrate a data event. To alleviate them, we propose VisGuide—a data exploration system that generates personalized recommendations to aid users’ discovery of data events in breadth and depth by incrementally learning their data exploration preferences and recommending meaningful charts tailored to them. As well as user preferences, VisGuide’s recommendations simultaneously consider sequence organization and chart presentation. We conducted two user studies to evaluate 1) the usability of VisGuide and 2) user satisfaction with its recommendation system. The results of those studies indicate that VisGuide can effectively help users create coherent and user-oriented visualization trees that represent meaningful data events.

Distance compression, which refers to the underestimation of ego-centric distance to objects, is a common problem in immersive virtual environments. Besides visually compensating the compressed distance, several studies have shown that auditory information can be an alternative solution for this problem. In particular, reverberation time (RT) has been proven to be an effective method to compensate distance compression. To further explore the feasibility of applying audio information to improve distance perception, we investigate whether users’ egocentric distance perception can be calibrated, and whether the calibrated effect can be carried over and even sustain for a longer duration. We conducted a study to understand the perceptual learning and carryover effects by using RT as stimuli for users to perceive distance in IVEs. The results show that the carryover effect exists after calibration, which indicates people can learn to perceive distances by attuning reverberation time, and the accuracy even remains a constant level after 6 months. Our findings could potentially be utilized to improve the distance perception in VR systems as the calibration of auditory distance perception in VR could sustain for several months. This could eventually avoid the burden of frequent training regimens.

Virtual Reality (VR) technologies are increasingly being applied to various contexts like those gaming, therapy, training, and education. Several of these applications require high degrees of accuracy in spatial and depth perception. Contemporary VR experiences continue to be confronted by the issue of distance compression wherein people systematically underestimate distances in the virtual world, leading to impoverished experiences. Consequently, a number of studies have focused on extensively understanding and exploring factors that influence this phenomenon to address the challenges it poses. Inspired by previous work that has sought to compensate distance compression effects in VR, we examined the potential of manipulating an auditory stimulus’ reverberation time (RT) to alter how users perceive depth. To this end, we conducted a two action forced choice study in which participants repeatedly made relative depth judgements between a pair of scenes featuring a virtual character placed at different distances with varying RTs. Results revealed that RT influences how users perceive depth with this influence being more pronounced in the near field. We found that users tend to associate longer RTs with farther distances and vice versa, indicating the potential to alter RT towards compensating distance underestimation in VR. However, it must be noted that excessively increasing RT (especially in the near field) could come at the cost of sensory segregation, wherein users are unable to unify visual and auditory sensory stimuli in their perceptions of depth. Researchers must hence strive to find the optimal amount of RT to add to a stimulus to ensure seamless virtual experiences.

Existing radar sensors can be classified into automotive and scanning radars. While most radar odometry (RO) methods are only designed for a specific type of radar, our RO method adapts to both scanning and automotive radars. Our RO is simple yet effective, where the pipeline consists of thresholding, probabilistic submap building, and an Normal Distribution Transform-based (NDT-based) radar scan matching. The proposed RO has been tested on two public radar datasets: the Oxford Radar RobotCar dataset and the nuScenes dataset, which provide scanning and automotive radar data respectively. The results show that our approach surpasses state-of-the-art RO using either automotive or scanning radar by reducing translational error by 51% and 30%, respectively, and rotational error by 17% and 29%, respectively. Besides, we show that our RO achieves centimeter-level accuracy as lidar odometry, and automotive and scanning RO have similar accuracy.

People increasingly use online video platforms, e.g., YouTube, to locate educational videos to acquire knowledge or skills to meet personal learning needs. However, most of existing video platforms display video search results in generic ranked lists based on relevance to queries. The design of relevance-oriented information display does not take into account the inner structure of the knowledge domain, and may not suit the need of online learners. In this paper, we present ConceptGuide, a prototype system for learning orientations to support ad hoc online learning from unorganized video materials. ConceptGuide features a computational pipeline that performs content analysis on the transcripts of YouTube videos retrieved for a topic, and generates concept-map-based visual recommendations of inter-concept and inter-video links, forming learning pathways as structures for learners to consume. We evaluated ConceptGuide by comparing the design to the general-purpose interface of YouTube in learning experiences and behaviors. ConceptuGuide was found to improve the efficiency of video learning and helped learners explore the knowledge of interest in many constructive ways.

While automotive radars are widely used in most assisted and autonomous driving systems, only a few works were proposed to tackle the calibration problems of automotive radars with other perception sensors. One of the key calibration challenges of automotive planar radars with other sensors is the missing elevation angle in 3D space. In this paper, extrinsic calibration is accomplished based on the observation that the radar cross section (RCS) measurements have different value distributions across radar's vertical field of view. An approach to accurately and efficiently estimate the time delay between radars and LiDARs based on spatial-temporal relationships of calibration target positions is proposed. In addition, a localization method for calibration target detection and localization in pre-built maps is proposed to tackle insufficient LiDAR measurements on calibration targets. The experimental results show the feasibility and effectiveness of the proposed Radar-LiDAR extrinsic and temporal calibration approaches.

Design discussion is a very important course in architecture education. In this paper, we developed a VR based architecture design discussion system that allows members to visualize and discuss the architectural models and to modify the models during discussion. Since the system is designed to work on top of Rhinoceros and Grasshopper, the object database is updated right after the object modification. Members communicate via voice, object manipulations, and mid-air sketching as well as on-surface sketching in the virtual environment. Several tools have been designed to enhance the sense of presence and to make the discussion more effective. We also developed a rollback mechanism to help users intuitively and quickly revert to a previous state of discussion to make some changes or to start a new direction of discussion. To evaluate the system, we conducted an initial user study with 14 participants to assess the user experience, user impression and effectiveness of the system. The feedback from participants were positive and suggested that the system could be effective and useful for supporting architecture design discussion.

The commercialization and lowering costs of consumer grade Virtual Reality (VR) devices has made the technology increasingly accessible to users around the world. The usage of VR technology is often accompanied by an undesirable side effect called cybersickness. Cyber-sickness is the feeling of discomfort that occurs during VR experiences, producing symptoms similar to those of motion sickness. It continues to remain one of the biggest hurdles to the widespread adoption of VR, making it increasingly important to explore and understand the factors that influence its onset. In this work, we investigated the influence of the presence/absence of motion control on the onset and severity of cybersickness in an HMD based VR driving simulation employing steering as a travel metaphor. Towards this end, we conducted a between subjects study manipulating the presence of control between three experimental conditions, two of which (Driving condition and Yoked Pair condition) formed a yoked control design where every pair of drivers and their yoked pairs were exposed to identical vehicular motion stimuli created by participants in the driving condition. In the other condition (Autonomous Car condition), participants experienced a program driven autonomous vehicle simulation. Results indicated that participants in the Driving condition experienced higher levels of cybersickness than participants in the Yoked Pair condition. While these results don’t conform to findings from previous research which suggests that having control over motion reduces cybersickness, it seems to point towards the importance of the fidelity of the control metaphor’s feedback response in alleviating cybersickness. Simply allowing one control their motion may not readily alleviate cybersickness but could instead increase it in such HMD based VR driving simulations. It may hence be important to consider how well the control metaphor and its feedback matches users’ expectations if we want to successfully mitigate cybersickness.

Cybersickness (CS) is one of the challenges that has hindered the widespread adoption of Virtual Reality and its applications. Consequently, a number of studies have focused on extensively understanding and reducing CS. Inspired by previous work that has sought to reduce CS using foveated rendering and Field of View (FOV) restrictions, we investigated how the presence and size of a static central window in peripheral FOV blurring affects CS. To facilitate this peripheral FOV blur, we applied a Gaussian blur effect in the display peripheral region, provisioning a full-resolution central window. Thirty participants took part in a three-session, within-subjects experiment, performing search and spatial updating tasks in a first-person, slow-walking, maze-traveling scenario. Two different central window sizes (small and large) were tested against a baseline condition that didn’t feature display peripheral blurring. Results revealed that the baseline condition produced higher levels of CS than both conditions with a central window. While there were no significant differences between the small and large windows, we observed interaction effects suggesting an influence of window size on “adaptation to CS.” When the central window is small, adaptation to CS seems to take more time but is more pronounced. The interventions had no effect on spatial updating and presence, but were detectable when the blurred area was larger (small central window). Lower sickness levels observed in both window conditions supports the use of peripheral FOV blurring to reduce CS, reducing our dependence on eye tracking. This being said, researchers must strive to find the right balance between window size and detectability to ensure seamless virtual experiences.

In this paper, we introduce a self-adaptive artificial bee colony (ABC) algorithm for learning the parameters of a Takagi-Sugeno-Kang-type (TSK-type) neuro-fuzzy system (NFS). The proposed NFS learns fuzzy rules for the premise part of the fuzzy system using an adaptive clustering method according to the input-output data at hand for establishing the network structure. All the free parameters in the NFS, including the premise and the following TSK-type consequent parameters, are optimized by the modified ABC (MABC) algorithm. Experiments involve two parts, including numerical optimization problems and dynamic system identification problems. In the first part of investigations, the proposed MABC compares to the standard ABC on mathematical optimization problems. In the remaining experiments, the performance of the proposed method is verified with other metaheuristic methods, including differential evolution (DE), genetic algorithm (GA), particle swarm optimization (PSO) and standard ABC, to evaluate the effectiveness and feasibility of the system. The simulation results show that the proposed method provides better approximation results than those obtained by competitors methods.

Prior studies have evaluated various designs for audio notifications. However, calls for more in-depth research on how such notifications work, especially at the level of users' cognitive states, have gone unanswered; and studies evaluating audio notifications with large numbers of participants in multiple environments have been rare. This study conducted an electroencephalography study (N=20) and an online study (N=967) to enhance understandings of how three musical parameters - melody (simple, complex), pitch (high, low), and tempo (fast, slow) - influenced users' cognition and behaviors. There are eight different notifications with different combinations of these parameters. The online study analyzed the effects of user-specific and environmental information on users' behaviors while they listened to these notifications. The results revealed that tempo and pitch have the main effect on the speed and strength (accuracy) of users' cognition and behaviors. The users' characteristics and environments influenced the effects of these musical parameters.

Auditory alarms that repeatedly interrupt users until they react are common, especially in the context of alarms. However, when an alarm repeats, our brains habituate to it and perceive it less and less, with reductions in both perception and attention-shifting: a phenomenon known as the repetition-suppression effect (RS). To retain users' perception and attention, this paper proposes and tests the use of pitch- and intensity-modulated alarms. Its experimental findings suggest that the proposed modulated alarms can reduce RS, albeit in different patterns, depending on whether pitch or intensity is the focus of the modulation. Specifically, pitch-modulated alarms were found to reduce RS more when the number of repetitions was small, while intensity-modulated alarms reduced it more as the number of repetitions increased. Based on these results, we make several recommendations for the design of improved repeating alarms, based on which modulation approach should be adopted in various situations.

Many crucial applications in the fields of filmmaking, game design, education, and cultural preservation - among others - involve the modeling, authoring, or editing of 3D objects and scenes. The two major methods of creating 3D models are 1) modeling, using computer software, and 2) reconstruction, generally using high-quality 3D scanners. Scanners of sufficient quality to support the latter method remain unaffordable to the general public. Since the emergence of consumer-grade RGBD cameras, there has been a growing interest in 3D reconstruction systems using depth cameras. However, most such systems are not user-friendly, and require intense efforts and practice if good reconstruction results are to be obtained. In this paper, we propose to increase the accessibility of depth-camera-based 3D reconstruction by assisting its users with augmented reality (AR) technology. Specifically, the proposed approach will allow users to rotate/move a target object freely with their hands and see the object being overlapped with its reconstructing model during the reconstruction process. As well as being more instinctual than conventional reconstruction systems, our proposed system will provide useful hints on complete 3D reconstruction of an object, including the best capturing range; reminder of moving and rotating the object at a steady speed; and which model regions are complex enough to require zooming-in. We evaluated our system via a user study that compared its performance against those of three other stateof- the-art approaches, and found our system outperforms the other approaches. Specifically, the participants rated it highest in usability, understandability, and model satisfaction.

Auditory saliency is an important mechanism that helps humans extract relevant information from environments. Audio notifications of mobile devices with high saliency can increase users' receptivity, yet overly high saliency could cause annoyance. Accurately measuring auditory saliency of a notification is critical for evaluating its usability. Previous studies adopted behavioral methods. However, their results may not accurately reflect auditory saliency as humans' perception of auditory saliency often involves complicated cognitive processes. Thus, we propose an electroencephalography (EEG)-based approach that can complement behavioral studies to provide a more nuanced analysis of auditory saliency. We evaluated our method by conducting an EEG experiment that measured the mismatch negativity and P3a of the sounds in realistic scenarios. We also conducted a behavioral experiment to link the EEG-based method with the behavioral method. The results suggested that EEG can provide detailed information about how human perceive auditory saliency and complement the behavioral measures.

Online learning platforms such as MOOCs have been prevalent sources of self-paced learning to people nowadays. However, the lack of peer accompaniment and social interaction may increase learners' sense of isolation and loneliness. Prior studies have shown the positive effects on visualizing peer students' appearances with virtual avatars or virtualized online learners in VR learning environments. In this work, we propose to build virtual classmates, which were constructed by synthesizing previous learners' messages (time-anchored comments). Configurations of virtual classmates, such as the number of classmates participating in a VR class and the behavioral features of the classmates, can also be adjusted. To build the characteristics of virtual classmates, we propose a technique called comment mapping to aggregate prior online learners' comments to shape virtual classmates' behaviors. We conduct a study with 100 participants to evaluate the effects of the virtual classmates built with and without the comment mapping and the amount of virtual classmates rendered in VR. The findings of our study suggest design implications for developing virtual classmates in VR environments.

Abstract Skiing is a popular recreational sport, and competitive skiing has been events at the Winter Olympic Games. Due to its wide moving range in the outdoor environment, motion capture of skiing is hard and usually not a good solution for generating skiing animations. Physical simulation offers a more viable alternative. However, skiing simulation is challenging as skiing involves many complicated motor skills and physics, such as balance keeping, movement coordination, articulated body dynamics and ski‐snow reaction. In particular, as no reference motions — usually from MOCAP data — are readily available for guiding the high‐level motor control, we need to synthesize plausible reference motions additionally. To solve this problem, sports techniques are exploited for reference motion planning. We propose a physics‐based framework that employs kinetic analyses of skiing techniques and the ski–snow contact model to generate realistic skiing motions. By simulating the inclination, angulation and weighting/unweighting techniques, stable and plausible carving turns and bump skiing animations can be generated. We evaluate our framework by demonstrating various skiing motions with different speeds, curvature radii and bump sizes. Our results show that employing the sports techniques used by athletes can provide considerable potential to generate agile sport motions without reference motions.

Abstract Sensorimotor control is an essential mechanism for human motions, from involuntary reflex actions to intentional motor skill learning, such as walking, jumping, and swimming. Humans perform various motions according to different task goals and physiological sensory perception; however, most existing computational approaches for motion simulation and generation rarely consider the effects of human perception. The assumption of perfect perception (i.e., no sensory errors) of existing approaches restricts the generated motion types and makes dynamical reactions less realistic. We propose a general framework for sensorimotor control, integrating a balance controller and a vestibular model, to generate perception‐aware motions. By exploiting simulated perception, more natural responses that are closer to human reactions can be generated. For example, motion sickness caused by the impairments in the function of the vestibular system induces postural instability and body sway. Our approach generates physically correct motions and reasonable reactions to external stimuli since the spatial orientation estimation by the vestibular system is essential to preserve balance. We evaluate our framework by demonstrating standing balance on a rotational platform with different angular speeds and duration. The generated motions show that either faster angular speeds or longer rotational duration cause more severe motion sickness. Our results demonstrate that sensorimotor control, integrating human perception and physically‐based control, offers considerable potential for providing more human‐like behaviors, especially for perceptual illusions of human beings, including visual, proprioceptive, and tactile sensations.

Abstract Realizing unrealistic faces is a complicated task that requires a rich imagination and comprehension of facial structures. When face matching, warping or stitching techniques are applied, existing methods are generally incapable of capturing detailed personal characteristics, are disturbed by block boundary artefacts, or require painting‐photo pairs for training. This paper presents a data‐driven framework to enhance the realism of sketch and portrait paintings based only on photo samples. It retrieves the optimal patches of adaptable shapes and numbers according to the content of the input portrait and collected photos. These patches are then seamlessly stitched by chromatic gain and offset compensation and multi‐level blending. Experiments and user evaluations show that the proposed method is able to generate realistic and novel results for a moderately sized photo collection.

Stylizing a 3D model with characteristic shapes or appearances is common in product design, particularly in the design of 3D model merchandise, such as souvenirs, toys, furniture, and stylized items. A model stylization approach is proposed in this study. The approach combines base and style models while preserving user-specified shape features of the base model and the attractive features of the style model with limited assistance from a user. The two models are first combined at the topological level. A tree-growing technique is utilized to search for all possible combinations of the two models. Second, the models are combined at textural and geometric levels by employing a morphing technique. Results show that the proposed approach generates various appealing models and allows users to control the diversity of the output models and adjust the blending degree between the base and style models. The results of this work are also experimentally compared with those of a recent work through a user study. The comparison indicates that our results are more appealing, feature-preserving, and reasonable than those of the compared previous study. The proposed system allows product designers to easily explore design possibilities and assists novice users in creating their own stylized models.

As virtual reality (VR) hardware technology becomes more mature and affordable, it is timely to develop visualization applications making use of such technology. How to interact with data in an immersive 3D space is both an interesting and challenging problem, demanding more research investigations. In this paper, we present a gesture input system for graph visualization in a stereoscopic 3D space. We compare desktop mouse input with gesture input with bare hands for performing a set of tasks on graphs. Our study results indicate that users are able to effortlessly manipulate and analyze graphs using gesture input. Furthermore, the results also show that using gestures is more efficient when exploring the complicated graph.

Abstract MOOCs (Massive Open Online Courses) are increasingly prevalent as an online educational resource open to everyone and have attracted hundreds of thousands learners enrolling these online courses. At such scale, there is potentially rich information of learners' behaviors embedded in the interactions between learners and videos that may help instructors and content producers adjust the instructions and refine the online courses. However, the lack of tools to visualize information from interactive data, including messages left to the videos at particular timestamps as well as the temporal variations of learners' online participation and perceived experience, has prevented people from gaining more insights from video‐watching logs. In this paper, we focus on extracting and visualizing useful information from time‐anchored comments that learners left to specific time points of the videos when watching them. Timestamps as a kind of metadata of messages can be useful to recover the interactive dynamics of learners occurring around the videos. Therefore, we present a visualization system to analyze and categorize time‐anchored comments based on topics and content types. Our system integrates visualization methods of temporal text data, namely ToPIN and ThemeRiver, which can help people understand the quality and quantity of online learners' feedback and their states of learning. To evaluate the proposed system, we visualized time‐anchored commenting data from two online course videos, and conducted two user studies participated by course instructors and third‐party educational evaluators. The results validate the usefulness of the approach and show how the quantitative and qualitative visualizations can be used to gain interesting insights around learners' online learning behaviors.

Graphic icons play an increasingly important role in interface design due to the proliferation of digital devices in recent years. Their ability to express information in a universal fashion allows us to immediately interact with new applications, systems, and devices. Icons can, however, cause user confusion and frustration if designed poorly. Several studies have evaluated icons using behavioral-performance metrics such as reaction time as well as self-report methods. However, determining the usability of icons based on behavioral measures alone is not straightforward, because users' interpretations of the meaning of icons involve various cognitive processes and perceptual mechanisms. Moreover, these perceptual mechanisms are affected not only by the icons themselves, but by usage scenarios. Thus, we need a means of sensitively and continuously measuring users' different cognitive processes when they are interacting with icons. In this study, we propose an EEG-based approach to icon evaluation, in which users' EEG signals are measured in multiple usage scenarios. Based on a combination of EEG and behavioral results, we provide a novel interpretation of the participants' perception during these tasks, and identify some important implications for icon design.

We propose an electroencephalography (EEG) prediction system based on a recurrent fuzzy neural network (RFNN) architecture to assess drivers' fatigue degrees during a virtual-reality (VR) dynamic driving environment. Prediction of fatigue degrees is a crucial and arduous biomedical issue for driving safety, which has attracted growing attention of the research community in the recent past. Meanwhile, combined with the benefits of measuring EEG signals facilitates, many EEG-based brain-computer interfaces (BCIs) have been developed for use in real-time mental assessment. In the literature, EEG signals are severely blended with stochastic noise; therefore, the performance of BCIs is constrained by low resolution in recognition tasks. For this rationale, independent component analysis (ICA) is usually used to find a source mapping from original data that has been blended with unrelated artificial noise. However, the mechanism of ICA cannot be used in real-time BCI design. To overcome this bottleneck, the proposed system in this paper utilizes a recurrent self-evolving fuzzy neural work (RSEFNN) to increase memory capability for adaptive noise cancellation when assessing drivers' mental states during a car driving task. The experimental results without the use of ICA procedure indicate that the proposed RSEFNN model remains superior performance compared with the state-of-the-arts models.

A brain-computer interface (BCI) system provides a convenient means of communication between the human brain and a computer, which is applied not only to healthy people but also for people that suffer from motor neuron diseases (MNDs). Motor imagery (MI) is one well-known basis for designing Electroencephalography (EEG)-based real-life BCI systems. However, EEG signals are often contaminated with severe noise and various uncertainties, imprecise and incomplete information streams. Therefore, this study proposes spectrum ensemble based on swam-optimized fuzzy integral for integrating decisions from sub-band classifiers that are established by a sub-band common spatial pattern (SBCSP) method. Firstly, the SBCSP effectively extracts features from EEG signals, and thereby the multiple linear discriminant analysis (MLDA) is employed during a MI classification task. Subsequently, particle swarm optimization (PSO) is used to regulate the subject-specific parameters for assigning optimal confidence levels for classifiers used in the fuzzy integral during the fuzzy fusion stage of the proposed system. Moreover, BCI systems usually tend to have complex architectures, be bulky in size, and require time-consuming processing. To overcome this drawback, a wireless and wearable EEG measurement system is investigated in this study. Finally, in our experimental result, the proposed system is found to produce significant improvement in terms of the receiver operating characteristic (ROC) curve. Furthermore, we demonstrate that a robotic arm can be reliably controlled using the proposed BCI system. This paper presents novel insights regarding the possibility of using the proposed MI-based BCI system in real-life applications.

We present an agent-based approach to animate microscopic mixed traffic involving cars and motorcycles in complex scenarios, including signalized and nonsignalized road intersections, and traffic jams due to blockage. Based on our new car-following and lateral movement models, our method can reproduce lane-based and nonlane-based traffic behaviors that are commonly seen in urban scenes. Our dynamic routing and intersection procedures enable a user to interactively control the movement of a car, and our system generates the microscopic behaviors of the influenced vehicles accordingly. Experimental results show that our approach can animate appealing microscopic mixed traffic with various behaviors. Our approach will benefit applications in virtual cities, computer games, and driving simulators.

Online learning is increasingly prevalent as an option for self-learning and as a resource for instructional design. Prerecorded video is currently the main medium of online education content delivery and instruction; this affords asynchronicity and flexibility, and enables the dissemination of lecture content in a distributed and scalable manner. However, the same properties may impede learners' engagement due to the lack of social interaction and peer support. In this paper, we propose a time-anchored commenting interface to allow online learners who watch the same video clips to exchange comments on them. Comments left by previous learners at specific time points of a video are displayed to new learners when they watch the same video and reach those time points. We investigated how the display of time-anchored comments (dynamic or static) and type of comments (content-related or social-oriented) influenced users' perceived engagement, perceived social interactivity, and learning outcomes. Our results show that dynamically displaying time-anchored comments can indeed enhance learners' perceived social interactivity. Moreover, the content of comments would further affect learners' intention of commenting. Based on our findings, we make various recommendations for the improvement of social interaction and learning experience in online education.

In this paper, we develop a physics based approach which enables users to use a brush for manipulating water. The users can drip and drag water on a non-absorbent hydrophilic surface to create water artworks. We consider factors, such as cohesive force and adhesive force, to compute the motion of water. Water drops with different shapes can be formed. We also develop a converter for converting input pictures into water-styled pictures. Our system can be applied in advertisements, movies, games, and education.

Abstract Visualization of vehicle detection (VD) data is essential because the data play an important role in traffic control and policy development. Most previous works focus on visualizing trajectories obtained from global positioning system (GPS), which are detailed but less representative. In contrast, VD data report the traffic statistic at each sensing site during a time span, including speed, flow, and occupancy of each lane, which contain comprehensive traffic information for analysis. In this work, we visualize three‐year VD data of freeways in Taiwan. The visualization depicts the traffic situation at a site over time using a color‐coded chart that extends from left to right over time. The charts are vertically stacked and horizontally aligned according to VD's located mileage and data time, respectively, to provide global insight. Our system allows semantic zoom, which changes the chart appearance in a continuous manner, to enable macro‐ and micro‐ scopic visualizations. Analysts can explore events that span an area with different sizes and that persist a time span with various lengths. To ensure the feasibility of our visualization, before the system design, we conducted a study with experts who work in the national freeway bureau and the institute of transportation of Taiwan. We also showed our results to the experts after the prototype system was built. The feedback shows that our VD data visualization is helpful to traffic control and policy development.

Abstract Flow visualization is recognized as an essential tool for many scientific research fields and different visualization approaches are proposed. Several studies are also conducted to evaluate their effectiveness but these studies rarely examine the performance from the perspective of visual perception. In this paper, we aim at exploring how users’ visual perception is influenced by different 2D flow visualization methods. An eye tracker is used to analyze users’ visual behaviors when they perform the free viewing, advection prediction, flow feature detection, and flow feature identification tasks on the flow field images generated by different visualizations methods. We evaluate the illustration capability of five representative visualization algorithms. Our results show that the eye‐tracking‐based evaluation provides more insights to quantitatively analyze the effectiveness of these visualization methods.

Audio notifications are an important means of prompting users of electronic products. Although useful in most environments, audio notifications are ineffective in certain situations, especially against particular auditory backgrounds or when the user is distracted. Several studies have used behavioral performance to evaluate audio notifications, but these studies failed to achieve consistent results due to factors including user subjectivity and environmental differences; thus, a new method and more objective indicators are necessary. In this study, we propose an approach based on electroencephalography (EEG) to evaluate audio notifications by measuring users' auditory perceptual responses (mismatch negativity) and attention shifting (P3a). We demonstrate our approach by applying it to the usability testing of audio notifications in realistic scenarios, such as users performing a major task amid ambient noises. Our results open a new perspective for evaluating the design of the audio notifications.

The radial view-based culling (RVBC) method has been presented for continuous self-collision detection to efficiently cull away non-colliding regions. While this technique mainly relies on the segmented clusters of the reference pose and the associated fixed observer points, it has several drawbacks during the animation and the reduced cost of executing collision detection is limited. We thus present a modified framework to improve the culling efficiency of RVBC. At the preprocessing stage, we segment the closed deformable mesh according to not only the attached skeleton but also the triangle orientations, in order to minimize the collision checks of triangles in a cluster. At the runtime stage, we dynamically merge adjacent clusters and update the positions of observer points if the merged shape is nearly convex. This strategy minimizes the number of triangles in different clusters that required collision check. Our framework can be easily integrated with bounding volume hierarchies to boost the culling efficiency. Experimental results show that our framework achieves up to 5.2 times speedup over the original RVBC method and even more times over the recent techniques.

We present a chess visualization to convey the changes in a game over successive generations. It contains a score chart, an evolution graph and a chess board, such that users can understand a game from global to local viewpoints. Unlike current graphical chess tools, which focus only on highlighting pieces that are under attack and require sequential investigation, our visualization shows potential outcomes after a piece is moved and indicates how much tactical advantage the player can have over the opponent. Users can first glance at the score chart to roughly obtain the growth and decline of advantages from both sides, and then examine the position relations and the piece placements, to know how the pieces are controlled and how the strategy works. To achieve this visualization, we compute the decision tree using artificial intelligence to analyze a game, in which each node represents a chess position and each edge connects two positions that are one-move different. We then merge nodes representing the same chess position, and shorten branches where nodes on them contain only two neighbors, in order to achieve readability. During the graph rendering, the nodes containing events such as draws, effective checks and checkmates, are highlighted because they show how a game is ended. As a result, our visualization helps players understand a chess game so that they can efficiently learn strategies and tactics. The presented results, evaluations, and the conducted user studies demonstrate the feasibility of our visualization design.

Realistic rendering of human faces with makeup is critical for many applications in the 3D facial animation and cosmetic industry. Facial makeup is perhaps one of the most important routines for many females or even for some males. Makeup is a multi-layered process. For example, people usually do the skin care first and smear some cosmetics (such as foundation, blush, lipstick and eye-shadow) on their face. By smearing the cosmetics on the face, facial appearance changes obviously.

Abstract Lighting design plays a crucial role in indoor lighting design, computer cinematograph and many other applications. Computer‐assisted lighting design aims to find a lighting configuration that best approximates the illumination effect specified by designers. In this paper, we present an automatic approach for lighting design, in which discrete and continuous optimization of the lighting configuration, including the number, intensity, and position of lights, are achieved. Our lighting design algorithm consists of two major steps. The first step estimates an initial lighting configuration by light sampling and clustering. The initial light clusters are then recursively merged to form a light hierarchy. The second step optimizes the lighting configuration by alternatively selecting a light cut on the light hierarchy to determine the number of representative lights and optimizing the lighting parameters using the simplex method. To speed up the optimization computation, only illumination at scene vertices that are important to rendering are sampled and taken into account in the optimization. Using the proposed approach, we develop a lighting design system that can compute appropriate lighting configurations to generate the illumination effects iteratively painted and modified by a designer interactively.

Abstract This paper presents a novel example‐based stippling technique that employs a simple and intuitive concept to convert a color image into a pointillism painting. Our method relies on analyzing and imitating the color distributions of Seurat's paintings to obtain a statistical color model. Then, this model can be easily combined with the modified multi‐class blue noise sampling to stylize an input image with characteristics of color composition in Seurat's paintings. The blue noise property of the output image also ensures that the color points are randomly located but remain spatially uniform. In our experiments, the multivariate goodness‐of‐fit tests were adopted to quantitatively analyze the results of the proposed and previous methods, further confirming that the color composition of our results are more similar to Seurat's painting style than that of previous approaches. Additionally, we also conducted a user study participated by artists to qualitatively evaluate the synthesized images of the proposed method.

ABSTRACT Horse locomotion exhibits rich variations in gaits and styles. Although there have been many approaches proposed for animating quadrupeds, there is not much research on synthesizing horse locomotion. In this paper, we present a horse locomotion synthesis approach. A user can arbitrarily change a horse's moving speed and direction, and our system would automatically adjust the horse's motion to fulfill the user's commands. At preprocessing, we manually capture horse locomotion data from Eadweard Muybridge's famous photographs of animal locomotion and expand the captured motion database to various speeds for each gait. At runtime, our approach automatically changes gaits based on speed, synthesizes the horse's root trajectory, and adjusts its body orientation based on the horse's turning direction. We propose an asynchronous time warping approach to handle gait transition, which is critical for generating realistic and controllable horse locomotion. Our experiments demonstrate that our system can produce smooth, rich, and controllable horse locomotion in real time. Copyright © 2012 John Wiley & Sons, Ltd.

ABSTRACT Simulating realistic makeup effects is one of the important research issues in the 3D facial animation and cosmetic industry. Existing approaches based on image processing techniques, such as warping and blending, have been mostly applied to transfer one's makeup to another's. Although these approaches are intuitive and need only makeup images, they have some drawbacks, for example, distorted shapes and fixed viewing and lighting conditions. In this paper, we propose an integrated approach, which combines the Kubelka–Munk model and a screen‐space skin rendering approach, to simulate 3D makeup effects. The Kubelka–Munk model is used to compute total transmittance when light passes through cosmetic layers, whereas the screen‐space translucent rendering approach simulates the subsurface scattering effects inside human skin. The parameters of Kubelka–Munk model are obtained by measuring the optical properties of different cosmetic materials, such as foundations, blushes, and lipsticks. Our results demonstrate that the proposed approach is able to render realistic cosmetic effects on human facial models, and different cosmetic materials and styles can be flexibly applied and simulated in real time. Copyright © 2013 John Wiley & Sons, Ltd.

We present a novel radial-view-based culling method for continuous self-collision detection (CSCD) of skeletal models. Our method targets closed triangular meshes used to represent the surface of a model. It can be easily integrated with bounding volume hierarchies (BVHs) and used as the first stage for culling non-colliding triangle pairs. A mesh is decomposed into clusters with respect to a set of observer primitives (i.e., observer points and line segments) on the skeleton of the mesh so that each cluster is associated with an observer primitive. One BVH is then built for each cluster. At the runtime stage, a radial view test is performed from the observer primitive of each cluster to check its collision state. Every pair of clusters is also checked for collisions. We evaluated our method on various models and compared its performance with prior methods. Experimental results show that our method reduces the number of the bounding volume overlapping tests and the number of potentially colliding triangle pairs, thereby improving the overall process of CSCD.

In a virtual world, a group of virtual characters can interact with each other, and these characters may leave a group to join another. The interaction among individuals and groups often produces interesting events in a sequence of animation. The goal of this paper is to discover social events involving mutual interactions or group activities in multicharacter animations and automatically plan a smooth camera motion to view interesting events suggested by our system or relevant events specified by a user. Inspired by sociology studies, we borrow the knowledge in Proxemics, social force, and social network analysis to model the dynamic relation among social events and the relation among the participants within each event. By analyzing the variation of relation strength among participants and spatiotemporal correlation among events, we discover salient social events in a motion clip and generate an overview video of these events with smooth camera motion using a simulated annealing optimization method. We tested our approach on different motions performed by multiple characters. Our user study shows that our results are preferred in 66.19 percent of the comparisons with those by the camera control approach without event analysis and are comparable (51.79 percent) to professional results by an artist.

This paper presents a novel parametric representation for bidirectional texture functions. Our method mainly relies on two original techniques, namely, multivariate spherical radial basis functions (SRBFs) and optimized parameterization. First, since the surface appearance of a real-world object is frequently a mixed effect of different physical factors, the proposed sum-of-products model based on multivariate SRBFs especially provides an intrinsic and efficient representation for heterogenous materials. Second, optimized parameterization particularly aims at overcoming the major disadvantage of traditional fixed parameterization. By using a parametric model to account for variable transformations, the parameterization process can be tightly integrated with multivariate SRBFs into a unified framework. Finally, a hierarchical fitting algorithm for bidirectional texture functions is developed to exploit spatial coherence and reduce computational cost. Our experimental results further reveal that the proposed representation can easily achieve high-quality approximation and real-time rendering performance.

We present a physics-based approach to generate 3D biped character animation that can react to dynamical environments in real time. Our approach utilizes an inverted pendulum model to online adjust the desired motion trajectory from the input motion capture data. This online adjustment produces a physically plausible motion trajectory adapted to dynamic environments, which is then used as the desired motion for the motion controllers to track in dynamics simulation. Rather than using Proportional-Derivative controllers whose parameters usually cannot be easily set, our motion tracking adopts a velocity-driven method which computes joint torques based on the desired joint angular velocities. Physically correct full-body motion of the 3D character is computed in dynamics simulation using the computed torques and dynamical model of the character. Our experiments demonstrate that tracking motion capture data with real-time response animation can be achieved easily. In addition, physically plausible motion style editing, automatic motion transition, and motion adaptation to different limb sizes can also be generated without difficulty.

Abstract We present a novel approach for real‐time rendering of translucent surfaces. The computation of subsurface scattering is performed by first converting the integration over the 3D model surface into an integration over a 2D texture space and then applying importance sampling based on the irradiance stored in the texture. Such a conversion leads to a feasible GPU implementation and makes real‐time frame rate possible. Our implementation shows that plausible images can be rendered in real time for complex translucent models with dynamic light and material properties. For objects with more apparent local effect, our approach generally requires more samples that may downgrade the frame rate. To deal with this case, we decompose the integration into two parts, one for local effect and the other for global effect, which are evaluated by the combination of available methods [DS03, MKB* 03a] and our texture space importance sampling, respectively. Such a hybrid scheme is able to steadily render the translucent effect in real time with a fixed amount of samples.

We introduce a novel technique to generate painterly art map (PAM) for 3D non-photorealistic rendering. Our technique can automatically transfer brush stroke textures and color changes to 3D models from samples of a painted image. Therefore, the generation of stylized images/animation in the style of a given artwork can be achieved. This new approach works particularly well for a rich variety of brush strokes ranging from simple 1D and 2D line-art strokes to very complicated ones with significant variations in stroke characteristics. During the rendering/animation process, the coherence of brush stroke textures and color changes over 3D surfaces can be well maintained. With PAM, we can also easily generate the illusion of flow animation over a 3D surface to convey the shape of a model.

A near-regular texture (NRT) is a geometric and photometric deformation from its regular origin--a congruent wallpaper pattern formed by 2D translations of a single tile. A dynamic NRT is an NRT under motion. Although NRTs are pervasive in man-made and natural environments, effective computational algorithms for NRTs are few. This paper addresses specific computational challenges in modeling and tracking dynamic NRTs, including ambiguous correspondences, occlusions, and drastic illumination and appearance variations. We propose a lattice-based Markov-Random-Field (MRF) model for dynamic NRTs in a 3D spatiotemporal space. Our model consists of a global lattice structure that characterizes the topological constraint among multiple textons and an image observation model that handles local geometry and appearance variations. Based on the proposed MRF model, we develop a tracking algorithm that utilizes belief propagation and particle filtering to effectively handle the special challenges of the dynamic NRT tracking without any assumption on the motion types or lighting conditions. We provide quantitative evaluations of the proposed method against existing tracking algorithms and demonstrate its applications in video editing.

Near regular textures are pervasive in man-made and natural world. Their global regularity and local randomness pose new difficulties to the state of the art texture analysis and synthesis algorithms. We carry out a systematic comparison study on the performance of four texture synthesis algorithms on near-regular textures. Our results confirm that faithful near-regular texture synthesis remains a challenging problem for the state of the art general purpose texture synthesis algorithms. In addition, we provide comparison of human perception with computer evaluations on the quality of the texture synthesis results.

A near-regular texture deviates geometrically and photometrically from a regular congruent tiling. Although near-regular textures are ubiquitous in the man-made and natural world, they present computational challenges for state of the art texture analysis and synthesis algorithms. Using regular tiling as our anchor point, and with user-assisted lattice extraction, we can explicitly model the deformation of a near-regular texture with respect to geometry, lighting and color. We treat a deformation field both as a function that acts on a texture and as a texture that is acted upon, and develop a multi-modal framework where each deformation field is subject to analysis, synthesis and manipulation. Using this formalization, we are able to construct simple parametric models to faithfully synthesize the appearance of a near-regular texture and purposefully control its regularity.

Motion recognition has received increasing attention in recent years owing to heightened demand for computer vision in many domains, including the surveillance system, multimodal human computer interface, and traffic control system. Most conventional approaches classify the motion recognition task into partial feature extraction and time-domain recognition subtasks. However, the information of motion resides in the space-time domain instead of the time domain or space domain independently, implying that fusing the feature extraction and classification in the space and time domains into a single framework is preferred. Based on this notion, this work presents a novel Space-Time Delay Neural Network (STDNN) capable of handling the space-time dynamic information for motion recognition. The STDNN is unified structure, in which the low-level spatiotemporal feature extraction and high-level space-time-domain recognition are fused. The proposed network possesses the spatiotemporal shift-invariant recognition ability that is inherited from the time delay neural network (TDNN) and space displacement neural network (SDNN), where TDNN and SDNN are good at temporal and spatial shift-invariant recognition, respectively. In contrast to multilayer perceptron (MLP), TDNN, and SDNN, STDNN is constructed by vector-type nodes and matrix-type links such that the spatiotemporal information can be accurately represented in a neural network. Also evaluated herein is the performance of the proposed STDNN via two experiments. The moving Arabic numerals (MAN) experiment simulates the object's free movement in the space-time domain on image sequences. According to these results, STDNN possesses a good generalization ability with respect to the spatiotemporal shift-invariant recognition. In the lipreading experiment, STDNN recognizes the lip motions based on the inputs of real image sequences. This observation confirms that STDNN yields a better performance than the existing TDNN-based system, particularly in terms of the generalization ability. In addition to the lipreading application, the STDNN can be applied to other problems since no domain-dependent knowledge is used in the experiment.