Playing video games can enhance older adults’ well-being and social connections. However, most mainstream games rely on button-based controls that require fine motor skills, limiting accessibility. We present ElderPlay, a real-time game input translation system that enables older adults to play unmodified commercial games using intuitive, motion-based interaction. We first conducted a gesture elicitation study to derive user-defined gestures grounded in everyday experiences, which informed the design of a proof-of-concept system translating gestures into controller inputs. We then evaluated ElderPlay with two commercial Nintendo Switch games. Results show that gesture-based interaction improves enjoyment, perceived physical engagement, and performance. Rather than replacing controllers, our findings highlight the effectiveness of hybrid interaction, where gesture and controller inputs support different gameplay actions. We discuss implications for context-dependent and inclusive game interaction design.

Generative AI systems allow visual designers to rapidly produce many image alternatives, but preserving, revisiting, and building on promising partial results across iterations remains challenging. In practice, generated images are rarely valuable as complete wholes; instead, designers often reuse fragments, revisit earlier materials, and combine partial ideas into new directions. We present Trinketry, an element-centered visual exploration system built around trinkets: reusable visual and semantic elements extracted from prior generations. Trinketry allows users to extract image regions, whole images, text labels, and composite elements; recombine them in intention trays to generate new variations; trace the provenance of specific elements across the exploration process; and retrieve semantically related prior outcomes. By foregrounding extraction, recombination, tracing, and retrieval, this demo presents an alternative interaction model for AI-assisted visual exploration—one that supports traceable reuse, intentional iteration, and reflective design exploration.

We define authoritative presence as letting people experience authority through human-made technology in sensory or non-sensory ways. Our goal is to design a robot that creates the impression of possessing capabilities worthy of respect as a source of authority, thereby enhancing compliance without attributing that authority to external sources, such as a specific person or organization. We hypothesized that strategies commonly used in the Wizard-of-Oz method could help manipulate authoritative presence as it strives to ensure that the robot is not perceived as having additional abilities beyond those introduced by the manipulations. Wizards typically need to maintain the robot’s functionality and abilities at an appropriate level to minimize unwanted influence on participants’ perceptions and interactions. By interviewing HRI researchers who have wizarded, we summarized their usual strategies and implemented the opposite behaviors in a robot to investigate if this would contribute to authoritative presence. Based on the findings, we designed four behaviors that include (1) let the robot have an open-ended conversation with people, (2) randomize the robot’s reaction delay timing, (3) let the robot move with inconsistent velocity, and (4) let the robot perceive people’s status without looking at them. To evaluate the impact of these behaviors, we conducted a video-based online experiment with 942 participants, using a between-subjects design. The experiment aimed to determine whether the behaviors conveying authoritative presence would make people perceive the robot as having more authority and increase their likelihood of complying with its requests. A mediation analysis indicated that despite a decrease in perceived authority, the imply authoritative presence condition had a positive effect on participant compliance. Our study formally introduces the concept of authoritative presence, providing a proof-of-concept for how robots can create authoritative presence through specific behaviors. This work lays the groundwork for future research on authority and robotics.

Concept designers in the entertainment industry create highly detailed, often imaginary environments for movies, games, and TV shows. Their early ideation phase requires intensive research, brainstorming, visual exploration, and combination of various design elements to form cohesive designs. However, existing AI tools focus on image generation from user specifications, lacking support for the unique needs and complexity of concept designers' workflows. Through a formative study with 12 professional designers, we captured their workflows and identified key requirements for AI-assisted ideation tools. Leveraging these insights, we developed AIdeation to support early ideation by brainstorming design concepts with flexible searching and recombination of reference images. A user study with 16 professional designers showed that AIdeation significantly enhanced creativity, ideation efficiency, and satisfaction (all p<.01) compared to current tools and workflows. A field study with 4 studios for 1 week provided insights into AIdeation's benefits and limitations in real-world projects. After the completion of the field study, two studios, covering films, television, and games, have continued to use AIdeation in their commercial projects to date, further validating AIdeation's improvement in ideation quality and efficiency.

Sketches are an important medium of expression and recently many works concentrate on automatic sketch creations. One such ability very useful for amateurs is text-based completion of a partial sketch to create a complex scene, while preserving the style of the partial sketch. Existing methods focus solely on generating sketch that match the content in the input prompt in a predefined style, ignoring the styles of the input partial sketches, e.g., the global abstraction level and local stroke styles. To address this challenge, we introduce AutoSketch, a style-aware vector sketch completion method that accommodates diverse sketch styles and supports iterative sketch completion. AutoSketch completes the input sketch in a style-consistent manner using a two-stage method. In the first stage, we initially optimize the strokes to match an input prompt augmented by style descriptions extracted from a vision-language model (VLM). Such style descriptions lead to non-photorealistic guidance images which enable more content to be depicted through new strokes. In the second stage, we utilize the VLM to adjust the strokes from the previous stage to adhere to the style present in the input partial sketch through an iterative style adjustment process. In each iteration, the VLM identifies a list of style differences between the input sketch and the strokes generated in the previous stage, translating these differences into adjustment codes to modify the strokes. We compare our method with existing methods using various sketch styles and prompts, perform extensive ablation studies and qualitative and quantitative evaluations, and demonstrate that AutoSketch can support diverse sketching scenarios.

Environment designers in the entertainment industry create imaginative 2D and 3D scenes for games, films, and television, requiring both fine-grained control of specific details and consistent global coherence. Designers have increasingly integrated generative AI into their workflows, often relying on large language models (LLMs) to expand user prompts for text-to-image generation, then iteratively refining those prompts and applying inpainting. However, our formative study with 10 designers surfaced two key challenges: (1) the lengthy LLM-generated prompts make it difficult to understand and isolate the keywords that must be revised for specific visual elements; and (2) while inpainting supports localized edits, it can struggle with global consistency and correctness. Based on these insights, we present GenTune, an approach that enhances human--AI collaboration by clarifying how AI-generated prompts map to image content. Our GenTune system lets designers select any element in a generated image, trace it back to the corresponding prompt labels, and revise those labels to guide precise yet globally consistent image refinement. In a summative study with 20 designers, GenTune significantly improved prompt--image comprehension, refinement quality, and efficiency, and overall satisfaction (all $p < .01$) compared to current practice. A follow-up field study with two studios further demonstrated its effectiveness in real-world settings.

We present Surrogate Avatar, an adaptive telepresence method that enhances user mobility and situated co-presence in symmetric avatar-mediated communication. The system enables a remote user’s avatar to autonomously position itself in socially and environmentally appropriate locations within the local user’s space—based on spatial affordances, interactional norms, and environmental constraints—supporting fluid interaction without requiring a shared environmental context. Through a formative study, we derived key adaptation objectives and implemented them using a distributed optimization framework based on the AUIT system. The framework distributes adaptation tasks across server and client to balance responsiveness and computational efficiency. A user study involving both stationary and nomadic scenarios demonstrated consistently high usability and presence, with some limitations observed under walking conditions. An additional exploratory field study in a semi-structured public setting demonstrated the system’s viability beyond controlled lab conditions. These findings motivate future designs of mobile telepresence systems that dynamically adapt to spatial and conversational context while mitigating misunderstandings that can arise from asymmetric environmental awareness and supporting privacy-sensitive interaction.

Rekindling old social bonds with individuals who were once a part of our lives but have since faded away is crucial for our well-being. Such connections with dormant ties help us overcome loneliness and provide social support. Recently, virtual reality (VR) emerged as a promising tool for facilitating social interactions, such as online gatherings for formal or casual activities. VR can offer immersive and shared experiences, facilitating genuine connections between people. This provides a unique advantage over traditional computer-mediated communication methods. However, while prior research has explored how VR can aid in forming new social connections, its potential to reconnect dormant ties is largely unexplored. This paper aims to bridge this gap by examining how different features of VR, specifically avatar appearance and virtual environments, influence reactivations of dormant ties. We conducted an experiment involving 24 dyads to investigate the effect of different avatar-self similarities and virtual environments on the perceptions and interactions between dormant ties. Our findings indicate that avatars resembling oneself and dormant ties promote social closeness. Familiar virtual environments evoke shared memories, while unfamiliar ones stimulate more conversations. We discuss the impact of VR features on reconnecting dormant ties and provide implications for re-connecting relationships in VR.

Abstract Direct part-level manipulation of man-made shapes in an image is desired given its simplicity. However, it is not intuitive given the existing manually created cuboid and cylinder controllers. To tackle this problem, we present StylePart, a framework that enables direct shape manipulation of an image by leveraging generative models of both images and 3D shapes. Our key contribution is a shape-consistent latent mapping function that connects the image generative latent space and the 3D man-made shape attribute latent space. Our method “forwardly maps” the image content to its corresponding 3D shape attributes, where the shape part can be easily manipulated. The attribute codes of the manipulated 3D shape are then “backwardly mapped” to the image latent code to obtain the final manipulated image. By using both forward and backward mapping, an user can edit the image directly without resorting to any 3D workflow. We demonstrate our approach through various manipulation tasks, including part replacement, part resizing, and shape orientation manipulation, and evaluate its effectiveness through extensive ablation studies.

Mixed-reality (MR) soundscapes blend real-world sound with virtual audio from hearing devices, presenting intricate auditory information that is hard to discern and differentiate. This is particularly challenging for blind or visually impaired individuals, who rely on sounds and descriptions in their everyday lives. To understand how complex audio information is consumed, we analyzed online forum posts within the blind community, identifying prevailing challenges, needs, and desired solutions. We synthesized the results and propose SoundShift for increasing MR sound awareness, which includes six sound manipulations: Transparency Shift, Envelope Shift, Position Shift, Style Shift, Time Shift, and Sound Append. To evaluate the effectiveness of SoundShift, we conducted a user study with 18 blind participants across three simulated MR scenarios, where participants identified specific sounds within intricate soundscapes. We found that SoundShift increased MR sound awareness and minimized cognitive load. Finally, we developed three real-world example applications to demonstrate the practicality of SoundShift.

While mathematical formulas are widely applied across various fields, the abundance of information they contain poses challenges for comprehension. Understanding the meanings of individual symbols in formulas is a significant obstacle for readers. They have to shift their attention between formulas and their accompanying descriptions frequently. Colorizing the symbol and its corresponding definitions in the same color can build visual links to guide the readers’ attention and reduce the cognitive load, which is one of the most pervasive designs that can enhance the readability of formulas. However, the colorization process is tedious and time-consuming since the authors must manually locate these symbols and definitions and change their colors one by one. Therefore, we propose MaugVLink, a prototype of LATEX-based and AI-assisted authoring tool, to expedite these processes. With easy-to-use symbol selection UI design and human-AI collaboration mechanism, the author can rapidly build the visual links to augment the formulas. We also conduct a preliminary two-part user study with eight participants to evaluate the effectiveness and usefulness of MaugVLink.

Brainstorming, a creative activity that aims to generate ideas, plays a crucial part in problem-solving processes and is widely employed to explore innovative solutions across various contexts. It may suffer from repetitive results, inefficient time management, and stagnation in discussion. Seeing the rich opportunities of virtual reality (VR) in visualizing the environments, we propose to use (1) scene-switching and (2) creative visualization of time constraints to facilitate this process. The current study sought to explore the effects of these two factors on brainstorming performance in VR. By conducting a mixed-method study of 20 three-participant groups, we found that scene-switching and implicit time limitations can stimulate creativity, establish diverse atmospheres, and open up new conversations during brainstorming. Additionally, we provide suggestions for visualizing time constraints and various virtual environments based on research results.

Laser-cutting is a promising fabrication method that empowers makers, including blind or visually-impaired (BVI) creators, to create technologies that fit their needs. Existing work on laser-cut accessibility has facilitated easier assembly as a workaround for existing models. However, laser-cut models are still not designed to accommodate the needs of BVI users. Integrating BVI needs can enrich the greater maker community by enabling cross-group discourse on laser-cut making. To investigate how laser-cut model design can be more accessible overall, we study laser-cut assembly as a process deeply intertwined with the fundamental design of laser-cut models. We present a study with seven sighted and seven BVI participants to compare their usage of laser-cut model affordances during assembly. Data for the BVI participants in this study originate from a previous work [13]. We identify assembly cues common or unique to sighted and BVI users, and discuss implications to improve general accessibility in laser-cut design.

Blind and visually impaired (BVI) people primarily rely on non-visual senses to interact with a physical environment. Doing so requires a high cognitive load to perceive and memorize the presence of a large set of objects, such as at home or in a learning setting. In this work, we explored opportunities to enable object-centric note-taking by using a 3D printing pen for interactive, personalized tactile annotations. We first identified the benefits and challenges of self-created tactile graphics in a formative diary study. Then, we developed TacNote, a system that enables BVI users to annotate, explore, and memorize critical information associated with everyday objects. Using TacNote, the users create tactile graphics with a 3D printing pen and attach them to the target objects. They capture and organize the physical labels by using TacNote’s camera-based mobile app. In addition, they can specify locations, ordering, and hierarchy via finger-pointing interaction and receive audio feedback. Our user study with ten BVI participants showed that TacNote effectively alleviated the memory burden, offering a promising solution for enhancing users’ access to information.

Abstract Colorizing icon is a challenging task, even for skillful artists, as it involves balancing aesthetics and practical considerations. Prior works have primarily focused on colorizing pixel‐based icons, which do not seamlessly integrate into the current vector‐based icon design workflow. In this paper, we propose a palette‐based colorization algorithm for vector icons without the need for rasterization. Our algorithm takes a vector icon and a five‐color palette as input and generates various colorized results for designers to choose from. Inspired by the common icon design workflow, we developed our algorithm to consist of two steps: generating a colorization template and performing the palette‐based color transfer. To generate the colorization templates, we introduce a novel vector icon colorization model that employs an MRF‐based loss and a color harmony loss. The color harmony loss encourages the alignment of the resulting color template with widely used harmony templates. We then map the predicted colorization template to chroma‐like palette colors to obtain diverse colorization results. We compare our results with those generated by previous pixel‐based icon colorization methods and validate the effectiveness of our algorithm by evaluations in both qualitative and quantitative measurements. Our method enables icon designers to explore diverse colorization results for a single icon using different color palettes while also efficiently evaluating the suitability of a color palette for a set of icons.

Abstract Interface icons are prevalent in various digital applications. Due to limited time and budgets, many designers rely on informal evaluation, which often results in poor usability icons. In this paper, we propose a unique human‐in‐the‐loop framework that allows our target users, that is novice and professional user interface (UI) designers, to improve the usability of interface icons efficiently. We formulate several usability criteria into a perceptual usability function and enable users to iteratively revise an icon set with an interactive design tool, EvIcon. We take a large‐scale pre‐trained joint image‐text embedding (CLIP) and fine‐tune it to embed icon visuals with icon tags in the same embedding space (IconCLIP). During the revision process, our design tool provides two types of instant perceptual usability feedback. First, we provide perceptual usability feedback modelled by deep learning models trained on IconCLIP embeddings and crowdsourced perceptual ratings. Second, we use the embedding space of IconCLIP to assist users in improving icons' visual distinguishability among icons within the user‐prepared icon set. To provide the perceptual prediction, we compiled IconCEPT10K, the first large‐scale dataset of perceptual usability ratings over 10,000 interface icons, by conducting a crowdsourcing study. We demonstrated that our framework could benefit UI designers' interface icon revision process with a wide range of professional experience. Moreover, the interface icons designed using our framework achieved better semantic distance and familiarity, verified by an additional online user study.

NFC (Near-Field Communication) has been widely applied for human-computer interaction (HCI). However, the short sensing distance of NFC requires the users to initiate the tasks with extra effort mostly using their hands, so it is inconvenient to use NFC in hands-busy scenarios. This paper presents an investigation of body-centric interactions between the NFC device users and their surroundings. The exploration is based on the recent development of near-field enabled clothing, which can passively extend an NFC-enabled device’s reading distance to users’ body landmarks. We present an accessible method for fabricating flexible, extensible, and scalable NFC extenders on clothing pieces, and an easy-to-use toolkit for facilitating designers to realize the interactive experiences. The method and toolkit were tested in technical experiments and in a co-creation workshop. The elicited design outcomes and the further exploratory makings generated knowledge for future research and embodied interaction design opportunities.

Blind people typically access videos via audio descriptions (AD) crafted by sighted describers who comprehend, select, and describe crucial visual content in the videos. 360° video is an emerging storytelling medium that enables immersive experiences that people may not possibly reach in everyday life. However, the omnidirectional nature of 360° videos makes it challenging for describers to perceive the holistic visual content and interpret spatial information that is essential to create immersive ADs for blind people. Through a formative study with a professional describer, we identified key challenges in describing 360° videos and iteratively designed OmniScribe, a system that supports the authoring of immersive ADs for 360° videos. OmniScribe uses AI-generated content-awareness overlays for describers to better grasp 360° video content. Furthermore, OmniScribe enables describers to author spatial AD and immersive labels for blind users to consume the videos immersively with our mobile prototype. In a study with 11 professional and novice describers, we demonstrated the value of OmniScribe in the authoring workflow; and a study with 8 blind participants revealed the promise of immersive AD over standard AD for 360° videos. Finally, we discuss the implications of promoting 360° video accessibility.

In this paper, we propose NFCStack, which is a physical building block system that supports stacking and frictionless interaction and is based on near-field communication (NFC). This system consists of a portable station that can support and resolve the order of three types of passive identifiable stackable: bricks, boxes, and adapters. The bricks support stable and sturdy physical construction, whereas the boxes support frictionless tangible interactions. The adapters provide an interface between the aforementioned two types of stackable and convert the top of a stack into a terminal for detecting interactions between NFC-tagged objects. In contrast to existing systems based on NFC or radio-frequency identification technologies, NFCStack is portable, supports simultaneous interactions, and resolves stacking and interaction events responsively, even when objects are not strictly aligned. Evaluation results indicate that the proposed system effectively supports 12 layers of rich-ID stacking with the three types of building block, even if every box is stacked with a 6-mm offset. The results also indicate possible generalized applications of the proposed system, including 2.5-dimensional construction. The interaction styles are described using several educational application examples, and the design implications of this research are explained.

Research has enabled virtual reality (VR) users to interact with the physical world by blending the physical world view into the virtual environment. However, current solutions are designed for specific use cases and hence are not capable of covering users’ varying needs for accessing information about the physical world. This work presents RealityLens, a user interface that allows users to peep into the physical world in VR with the reality lenses they deployed for their needs. For this purpose, we first conducted a preliminary study with experienced VR users to identify users’ needs for interacting with the physical world, which led to a set of features for customizing the scale, placement, and activation method of a reality lens. We evaluated the design in a user study (n=12) and collected the feedback of participants engaged in two VR applications while encountering a range of interventions from the physical world. The results show that users’ VR presence tends to be better preserved when interacting with the physical world with the support of the RealityLens interface.

Body-controlled avatars provide a more intuitive method to real-time control virtual avatars but require larger environment space and more user effort. In contrast, hand-controlled avatars give more dexterous and fewer fatigue manipulations within a close-range space for avatar control but provide fewer sensory cues than the body-based method. This paper investigates the differences between the two manipulations and explores the possibility of a combination. We first performed a formative study to understand when and how users prefer manipulating hands and bodies to represent avatars’ actions in current popular video games. Based on the top video games survey, we decided to represent human avatars’ motions. Besides, we found that players used their bodies to represent avatar actions but changed to using hands when they were too unrealistic and exaggerated to mimic by bodies (e.g., flying in the sky, rolling over quickly). Hand gestures also provide an alternative to lower-body motions when players want to sit during gaming and do not want extensive effort to move their avatars. Hence, we focused on the design of hand gestures and upper-body postures. We present Puppeteer, an input prototype system that allows players directly control their avatars through intuitive hand gestures and upper-body postures. We selected 17 avatar actions discovered in the formative study and conducted a gesture elicitation study to invite 12 participants to design best representing hand gestures and upper-body postures for each action. Then we implemented a prototype system using the MediaPipe framework to detect keypoints and a self-trained model to recognize 17 hand gestures and 17 upper-body postures. Finally, three applications demonstrate the interactions enabled by Puppeteer.

Welcome to this issue of the Proceedings of the ACM on Human-Computer Interaction, the first to bring together contributions from the community on Mobile Human-Computer Interaction (MHCI). The pervasive impact of mobile technologies over the last few years of the global pandemic is clear. They have become the de facto means for connecting children to their virtual classrooms, for remote doctor visits, to keep us 'in touch' with loved ones, for allowing us to keep working, and everything else in between. Yet, as researchers and practitioners, we are humbled by the many gaps that have needed our renewed attention. The call for articles for this first issue focused on contributions on the Mobile HCI theme received 113 valid submissions from across the world. After the first round of reviews, 56 contributions were invited to submit revised versions of their articles. After second round reviews 41 articles were accepted, giving an acceptance rate of 36%. We would like to thank our dedicated team of 35 associate editors and the external reviewers they recruited who have ensured the articles in this issue were rigorously reviewed to form a collection of high-quality articles on the theme of Mobile HCI.

Current haptic devices are usually designed to provide one type of force feedback; however, most VR scenarios require versatile force feedback, which may require the integration of different devices to provide various types of forces. In addition, besides the main haptic effects caused by the forces, multiple types of oscillation may also commonly accompany them, which are crucial for improving VR realism and immersion. Therefore, we simulate versatile force feedback by rendering the corresponding types of oscillation as the effects caused by those forces. We take inertia and impact forces as examples in this paper, and achieve versatility using the proposed device, OsciHead, on a head-mounted display (HMD), instead of integrating different devices. By controlling elastic bands' elasticity and stored power, OsciHead uses two rotatable oscillators on both sides of the HMD, in order to render various multilevel and multidimensional oscillation feedback in 2D translation and 2D rotation directions on a head. In an exploratory study, we explored different scenarios in which multiple types of oscillation could be simulated by OsciHead. We then observed oscillation level distinguishability in two just-noticeable difference (JND) studies, and evaluated the oscillation type recognition rates in a recognition study. Based on the results, we performed a VR study, which verified that the inertia and impact feedback simulated by OsciHead enhances realism and achieves versatility.

This article presents a novel deep learning-based approach for automatically vectorizing and synthesizing the clipart of man-made objects. Given a raster clipart image and its corresponding object category (e.g., airplanes), the proposed method sequentially generates new layers, each of which is composed of a new closed path filled with a single color. The final result is obtained by compositing all layers together into a vector clipart image that falls into the target category. The proposed approach is based on an iterative generative model that (i) decides whether to continue synthesizing a new layer and (ii) determines the geometry and appearance of the new layer. We formulated a joint loss function for training our generative model, including the shape similarity, symmetry, and local curve smoothness losses, as well as vector graphics rendering accuracy loss for synthesizing clipart recognizable by humans. We also introduced a collection of man-made object clipart, ClipNet, which is composed of closed-path layers, and two designed preprocessing tasks to clean up and enrich the original raw clipart. To validate the proposed approach, we conducted several experiments and demonstrated its ability to vectorize and synthesize various clipart categories. We envision that our generative model can facilitate efficient and intuitive clipart designs for novice users and graphic designers.

Many haptic feedback methods have been proposed to enhance realism in virtual reality (VR). However, friction on the feet in VR, which renders feedback as if walking on different terrains or ground textures or stepping on objects is still less explored. Herein, we propose a wearable device, FrictShoes a pair of foot accessories, to provide multilevel nonuniform friction feedback to feet. This is achieved by the independent functioning of six brakes on six wheels underneath each FrictShoe, which allows the friction levels of the wheels from each to be either matched or to vary. We conducted a magnitude estimation study to understand users' distinguishability of friction force magnitudes (or levels). Based on the results, we performed an exploratory study to realize how users adjust and map the multilevel nonuniform friction patterns to common VR terrains or ground textures. Finally, a VR experience study was conducted to evaluate the performance of the proposed multilevel nonuniform friction feedback to the feet in VR experiences.

In this paper, we propose the designs for low cost and 3D-printable add-on components to adapt existing breadboards, circuit components and electronics tools for blind or low vision (BLV) users. Through an initial user study, we identified several barriers to entry for beginners with BLV in electronics and circuit prototyping. These barriers guided the design and development of our add-on components. We focused on developing adaptations that provide additional information about the specific component pins and breadboard holes, modify tools to make them easier to use for users with BLV, and expand non-visual feedback (e.g., audio, tactile) for tasks that require vision. Through a second user study, we demonstrated that our adaptations can effectively overcome the accessibility barriers in breadboard circuit prototyping for users with BLV.

Tactile feedback is widely used to enhance realism in virtual reality (VR). When touching virtual objects, stiffness and roughness are common and obvious factors perceived by the users. Furthermore, when touching a surface with complicated surface structure, differences from not only stiffness and roughness but also surface height are crucial. To integrate these factors, we propose a pin-based handheld device, HairTouch, to provide stiffness differences, roughness differences, surface height differences and their combinations. HairTouch consists of two pins for the two finger segments close to the index fingertip, respectively. By controlling brush hairs’ length and bending direction to change the hairs’ elasticity and hair tip direction, each pin renders various stiffness and roughness, respectively. By further independently controlling the hairs’ configuration and pins’ height, versatile stiffness, roughness and surface height differences are achieved. We conducted a perception study to realize users’ distinguishability of stiffness and roughness on each of the segments. Based on the results, we performed a VR experience study to verify that the tactile feedback from HairTouch enhances VR realism.

This research investigates the design space of combining touchscreens with passive rich-ID building block systems to support the physical construction of contexts in touchscreen interactions. With two proof-of-concept systems, RFIPillars and RFITiles, we explore various schemes for using tangible inputs for context enrichment in touchscreen interactions. Instead of incorporating an electronic touchscreen module that requires per-module maintenance, this work intentionally makes each tangible object passive. We explore rear-projection solutions to integrate touchscreen interactions into these passive building blocks with capacitive touch sensing techniques and deliberate physical forgiving to retain the merits of being both batteryless and wireless. The presented research artifacts embody the interaction designs and elucidate scalability challenges in integrating touchscreen interactions into this emerging tangible user interface.

For haptic guidance, vibrotactile feedback is a commonly-used mechanism, but requires users to interpret its complicated patterns especially in 3D guidance, which is not intuitive and increases their mental effort. Furthermore, for haptic guidance in virtual reality (VR), not only guidance performance but also realism should be considered. Since vibrotactile feedback interferes with and reduces VR realism, it may not be proper for VR haptic guidance. Therefore, we propose a wearable device, GuideBand, to provide intuitive 3D multilevel force guidance upon the forearm, which reproduces an effect that the forearm is pulled and guided by a virtual guider or telepresent person in VR. GuideBand uses three motors to pull a wristband at different force levels in 3D space. Such feedback usually requires much larger and heavier robotic arms or exoskeletons. We conducted a just-noticeable difference study to understand users’ force level distinguishability. Based on the results, we performed a study to verify that compared with state-of-the-art vibrotactile guidance, GuideBand is more intuitive, needs a lower level of mental effort, and achieves similar guidance performance. We further conducted a VR experience study to observe how users combine and complement visual and force guidance, and prove that GuideBand enhances realism in VR guidance.

Design tools and research regarding laser-cut architectures have been widely explored in the past decade. However, such discussion has mostly revolved around technical and structural design questions instead of another essential element of laser-cut models — assembly — a process that relies heavily on components’ visual affordance, therefore less accessible to blind or low vision (BLV) people. To narrow the gap in this area, we co-designed with 7 BLV people to examine their assembly experience with different laser-cut architectures. From their feedback, we proposed several design heuristics and guidelines for Daedalus, a generative design tool that can produce tactile aids for laser-cut assembly given a few high-level manual inputs. We validate the proposed aids in a user study with 8 new BLV participants. Our results revealed that BLV users can manage laser-cut assembly more efficiently with Daedalus. Going forth from this design iteration, we discuss implications for future research on accessible laser-cut assembly.

Abstract We present an assistive system for clipart design by providing visual scaffolds from the unseen viewpoints. Inspired by the artists' creation process, our system constructs the visual scaffold by first synthesizing the reference 3D shape of the input clipart and rendering it from the desired viewpoint. The critical challenge of constructing this visual scaffold is to generate a reference 3D shape that matches the user's expectations in terms of object sizing and positioning while preserving the geometric style of the input clipart. To address this challenge, we propose a user‐assisted curve extrusion method to obtain the reference 3D shape. We render the synthesized reference 3D shape with a consistent style into the visual scaffold. By following the generated visual scaffold, the users can efficiently design clipart with their desired viewpoints. The user study conducted by an intuitive user interface and our generated visual scaffold suggests that our system is especially useful for estimating the ratio and scale between object parts and can save on average 57% of drawing time.

This paper introduces Glissade, a digital pen that generates balance shifting feedback by changing the weight distribution of the pen. A pulley system shifts a brass mass inside the pen to change the pen's center of mass and moment of inertia. When the mass is stationary, the pen delivers a constant yet natural sensation of weight, which can be used to convey a status. The pen can also generate a variety of haptic clues by actuating the mass according to the tilt or rotation of the pen, two commonly-used auxiliary pen input channels. Glissade demonstrates new possibilities that balance shifting feedback can bring to digital pen interactions. We validated the usability of this feedback by determining the recognizability of six balance patterns – a mix of static and dynamic patterns chosen based on our design considerations – in two controlled experiments. The results show that, on average, the participants could distinguish between the patterns with a 94.25% accuracy. At the end, we demonstrate a set of novel interactions enabled by Glissade and discuss the directions for future research.

Force feedback from damped oscillation is a common effect in our daily lives, especially when shaking an elastic object, an object hanging or containing other stuff, or a container with liquid, e.g., casting with a fishing pole or wine-swirling. Such a force, affected by complex physical variations and collisions, is difficult to properly simulate using current force feedback methods. Therefore, we propose ElastOscillation on a virtual reality (VR) controller to provide 3D multilevel force feedback for damped oscillation to enhance VR experiences. ElastOscillation consists of a proxy, six elastic bands and DC motors. It leverages the motors to control the bands' elasticity to restrain the movement of the proxy, which is connected with the bands. Therefore, when users shake the ElastOscillation device, the proxy shakes or moves in corresponding ranges of movement. The users then perceive the force from oscillation at different levels. In addition, elastic force from the bands further reinforces the oscillation force feedback. We conducted a force perception study to understand users' distinguishability for perceiving oscillation forces in 1D and 2D movement, respectively. Based on the results, we performed a VR experience study to show that the force feedback provided by ElastOscillation enhances VR realism.

We propose generating two-dimensional skin stretch feedback on the user's legs. Skin stretch is useful cutaneous feedback to induce the perception of virtual textures and illusory forces and to deliver directional cues. This feedback has been applied to the head, body, and upper limbs to simulate rich physical properties in virtual reality (VR). However, how to expand the benefit of skin stretch feedback and apply it to the lower limbs, remains to be explored. Our first two psychophysical studies examined the minimum changes in skin stretch distance and stretch angle that are perceivable by participants. We then designed and implemented Gaiters, a pair of ungrounded, leg-worn devices, each of which is able to generate multiple two-dimensional skin stretches on the skin of the user's leg. With Gaiters, we conducted an exploratory study to understand participants' experiences when coupling skin stretch patterns with various lower limb actions. The results indicate that rich haptic experiences can be created by our prototype. Finally, a user evaluation indicates that participants enjoyed the experiences when using Gaiters and considered skin stretch as compelling haptic feedback on the legs.

Force feedback is commonly used to enhance realism in virtual reality (VR). However, current works mainly focus on providing different force types or patterns, but do not investigate how a proper point of application of force (PAF), which means where the resultant force is applied to, affects users' experience. For example, users perceive resistive force without torque when pulling a virtual bow, but with torque when pulling a virtual slingshot. Therefore, we propose a set of handheld controllers, ElastiLinks, to provide force feedback between controllers with dynamic PAFs.A rotatable track on each controller provides a dynamic PAF, and two common types of force feedback, resistive force and impact, are produced by two links, respectively. We performed a force perception study to ascertain users' resistive and impact force level distinguishability between controllers. Based on the results, we conducted another perception study to understand users' distinguishability of PAF offset and rotation differences. Finally, we performed a VR experience study to prove that force feedback with dynamic PAFs enhances VR experience.

Abstract In recent years, personalized fabrication has received considerable attention because of the widespread use of consumer‐level three‐dimensional (3D) printers. However, such 3D printers have drawbacks, such as long production time and limited output size, which hinder large‐scale rapid‐prototyping. In this paper, for the time‐ and cost‐effective fabrication of large‐scale objects, we propose a hybrid 3D fabrication method that combines 3D printing and the Zometool construction set, which is a compact, sturdy and reusable structure for infill fabrication. The proposed method significantly reduces fabrication cost and time by printing only thin 3D outer shells. In addition, we design an optimization framework to generate both a Zometol structure and printed surface partitions by optimizing several criteria, including printability, material cost and Zometool structure complexity. Moreover, we demonstrate the effectiveness of the proposed method by fabricating various large‐scale 3D models.

The human ear is highly sensitive and accessible, making it especially suitable for being used as an interface for interacting with smart earpieces or augmented glasses. However, previous works on ear-based input mainly address gesture sensing technology and researcher-designed gestures. This paper aims to bring more understandings of gesture design. Thus, for a user elicitation study, we recruited 28 participants, each of whom designed gestures for 31 smart device-related tasks. This resulted in a total of 868 gestures generated. Upon the basis of these gestures, we compiled a taxonomy and concluded the considerations underlying the participants' designs that also offer insights into their design rationales and preferences. Thereafter, based on these study results, we propose a set of user-defined gestures and share interesting findings. We hope this work can shed some light on not only sensing technologies of ear-based input, but also the interface design of future wearable interfaces.

We propose autocomplete for the design and development of virtual breadboard circuits using software prototyping tools. With our system, a user inserts a component into the virtual breadboard, and it automatically provides a user with a list of suggested components. These suggestions complete or ex- tend the electronic functionality of the inserted component to save the user's time and reduce circuit error. To demon- strate the effectiveness of autocomplete, we implemented our system on Fritzing, a popular open source breadboard circuit prototyping software, used by novice makers. Our autocomplete suggestions were implemented based upon schematics from datasheets for standard components, as well as how components are used together from over 4000 circuit projects from the Fritzing community. We report the results of a controlled study with 16 participants, evaluating the effectiveness of autocomplete in the creation of virtual breadboard circuits, and conclude by sharing insights and directions for future research.

We propose a new type of haptic output for foreground interactions on an interactive chair, where input is carried out explicitly in the foreground of the user's consciousness. This type of force output restricts a user's motion by modulating the resistive force when rotating a seat, tilting the backrest, or rolling the chair. These interactions are useful for many applications in a ubiquitous computing environment, ranging from immersive VR games to rapid and private query of information for people who are occupied with other tasks (e.g. in a meeting). We carefully designed and implemented our proposed haptic force output on a standard office chair and determined the recognizability of five force profiles for rotating, tilting, and rolling the chair. We present the result of our studies, as well as a set of novel interaction techniques enabled by this new force output for chairs.

Resistive force (e.g., due to object elasticity) and impact (e.g., due to recoil) are common effects in our daily life. However, resistive force continuously changes due to users' movements while impact instantly occurs when an event triggers it. These feedback are still not realistically provided by current VR haptic methods. In this paper, a wearable device, ElasticVR, which consists of an elastic band, servo motors and mechanical brakes, is proposed to provide the continuously-changing resistive force and instantly-occurring impact upon the user's hand to enhance VR realism. By changing two physical properties, length and extension distance, of the elastic band, ElasticVR provides multilevel resistive force with no delay and impact with little delay, respectively, for realistic and versatile VR applications. A force perception study was performed to observe users' force distinguishability of the resistive force and impact, and the prototype was built based on its results. A VR experience study further proves that the resistive force and impact from ElasticVR both outperform those from current approaches in realism. Applications using ElasticVR are also demonstrated.

This work explores and evaluates the designs of finger-worn radio-frequency identification (RFID) motion tracking for activity recognition on tagged objects. We propose an index-finger-worn device that consists of a short-range (~2cm) RFID reader and a pair of two inertial measurement units (IMUs), which are mounted at the locations where an artificial nail and a ring are worn. The short-range RFID reader recognizes the tagged object on finger touch, and then the IMU data are used for activity recognition. Data collected from the user of this device allows for a post-hoc analysis, which informs the activity recognition performance in various RFID+IMU and IMU-only configurations on the same task. The results of a ten-participant user study show that when the objects have similar physical form factors, the hybrid RFID motion tracking significantly outperforms the IMU-only tracking, especially in a larger-number set of objects. In our test, three IMU configurations (i.e., NailOnly, RingOnly, and Nail+Ring) achieved comparable action recognition performances, i.e., ≥90% accuracy, with 500 ms recognition time, though the NailOnly RFID+IMU configuration provided the highest wearability. The practical challenges toward a real-world deployment of a finger-worn RFID motion tracking system are also discussed.

Rendering haptic feedback in virtual reality is a common approach to enhancing the immersion of virtual reality content. However, current editing tools allow developers to access the haptic feedback only through physical contact with the actuators, making it difficult to fast iterate haptic interaction designs. This paper introduces SeeingHaptics, an authoring tool which visualizes haptic properties in 3D scenes. The active area of certain feedback is simulated with mesh shapes, while the 2D icons allow for indicating the type of haptic sensation. Our evaluation showed that SeeingHaptics helps developers rapidly create haptic feedbacks after a short training session.

This paper presents RFTouchPads, a system of batteryless and wireless modular hardware designs of two-dimensional (2D) touch sensor pads based on the ultra-high frequency (UHF) radio-frequency identification (RFID) technology. In this system, multiple RFID IC chips are connected to an antenna in parallel. Each chip connects only one of its endpoints to the antenna; hence, the module normally turns off when it gets insufficient energy to operate. When a finger touches the circuit trace attached to another endpoint of the chip, the finger functions as part of the antenna that turns the connected chip on, while the finger touch location is determined according to the chip's ID. Based on this principle, we propose two hardware designs, namely, StickerPad and TilePad. StickerPad is a flexible 3×3 touch-sensing pad suitable for applications on curved surfaces such as the human body. TilePad is a modular 3×3 touch-sensing pad that supports the modular area expansion by tiling and provides a more flexible deployment because its antenna is folded. Our implementation allows 2D touch inputs to be reliability detected 2 m away from a remote antenna of an RFID reader. The proposed batteryless, wireless, and modular hardware design enables fine-grained and less-constrained 2D touch inputs in various ubiquitous computing applications.

We present TilePoP, a new type of pneumatically-actuated interface deployed as floor tiles which dynamically pop up by inflating into large shapes constructing proxy objects for whole-body interactions in Virtual Reality. TilePoP consists of a 2D array of stacked cube-shaped airbags designed with specific folding structures, enabling each airbag to be inflated into a physical proxy and then deflated down back to its original tile shape when not in use. TilePoP is capable of providing haptic feedback for the whole body and can even support human body weight. Thus, it allows new interaction possibilities in VR. Herein, the design and implementation of TilePoP are described in detail along with demonstrations of its applications and the results of a preliminary user evaluation conducted to understand the users' experience with TilePoP.

Impact is a common effect in both daily life and virtual reality (VR) experiences, e.g., being punched, hit or bumped. Impact force is instantly produced, which is distinct from other force feedback, e.g., push and pull. We propose ElastImpact to provide 2.5D instant impact on a head-mounted display (HMD) for realistic and versatile VR experiences. ElastImpact consists of three impact devices, also called impactors. Each impactor blocks an elastic band with a mechanical brake using a servo motor and extending it using a DC motor to store the impact power. When releasing the brake, it provides impact instantly. Two impactors are affixed on both sides of the head and connected with the HMD to provide the normal direction impact toward the face (i.e., 0.5D in z-axis). The other impactor is connected with a proxy collider in a barrel in front of the HMD and rotated by a DC motor in the tangential plane of the face to provide 2D impact (i.e., xy-plane). By performing a just-noticeable difference (JND) study, we realize users' impact force perception distinguishability on the heads in the normal direction and tangential plane, separately. Based on the results, we combine normal and tangential impact as 2.5D impact, and performed a VR experience study to verify that the proposed 2.5D impact significantly enhances realism.

We propose integrating an array of skin stretch modules with an head-mounted display (HMD) to provide two-dimensional skin stretch feedback on the user's face. Skin stretch has been found effective to induce the perception of force (e.g. weight or inertia) and to enable directional haptic cues. However, its potential as an HMD output for virtual reality (VR) remains to be exploited. Our explorative study firstly investigated the design of shear tactors. Based on our results, Masque has been implemented as an HMD prototype actuating six shear tactors positioned on the HMD's face interface. A comfort study was conducted to ensure that skin stretches generated by Masque are acceptable to all participants. The following two perception-based studies examined the minimum changes in skin stretch distance and stretch angles that are detectable by participants. The results help us to design haptic profiles as well as our prototype applications. Finally, the user evaluation indicates that participants welcomed Masque and regarded skin stretch feedback as a worthwhile addition to HMD output.

We present RFIBricks, an interactive building block system based on ultrahigh frequency radio-frequency identification (RFID) sensing. The system enables geometry resolution based on a simple yet highly generalizable mechanism: an RFID contact switch, which is made by cutting each RFID tag into two parts, namely antenna and chip. A magnetic connector is then coupled with each part. When the antenna and chip connect, an interaction event with an ID is transmitted to the reader. On the basis of our design of RFID contact switch patterns, we present a system of interactive physical building blocks that resolves the stacking order and orientation when one block is stacked upon another, determines a three-dimensional (3D) geometry built on a two-dimensional base plate, and detects user inputs by incorporating electromechanical sensors. Because it is calibration-free and does not require batteries in each block, it facilitates straightforward maintenance when deployed at scale. Compared with other approaches, this RFID-based system resolves several critical challenges in human-computer interaction, such as 1) determining the identity and the built 3D geometry of passive building blocks, 2) enabling stackable token+constraint interaction on a tabletop, and 3) tracking in-hand assembly.

This paper presents a technique enabling distributed batteryless near-field identification (ID) between two passive radio frequency ID (RFID) tags. Each conventional ultra-high-frequency (UHF) RFID tag is modified by connecting its antenna and chip to a reed switch and then attaching a magnet to one of the reed switch's terminals, thus transforming it into an always-on switch. When the two modules approach each other, the magnets counteract each other and turn off both switches at the same time. The coabsence of IDs thus indicates a unique interaction event. In addition to sensing, the module also provides native haptic feedback through magnetic repulsion force, enabling users to perceive the system's state eyes-free, without physical constraints. Additional visual feedback can be provided through an energy-harvesting module and a light emitting diode. This specific hardware design supports contactless, orientation-invariant sensing, with a form factor compact enough for embedded and wearable use in ubiquitous computing applications.

We propose using a single slip tactile pixel on virtual reality controllers to produce sensations of finger sliding and textures. When a user moves the controller on a virtual surface, we add a slip opposite to the movement, creating an illusion of a finger that is sliding on the surface, while varying the slip feedback changes lateral forces on fingertip. When coupled with hand motion the lateral forces can be used to create perceptions of artificial textures. RollingStone has been implemented as a prototype VR controller consisting of a ball-based slip display positioned under the user's fingertip. Within the slip display, a pair of motors actuates the ball, which is capable of gener- ating both short- and long-term two-degree-of-freedom slip feedback. An exploratory study was conducted to ensure that changing the relative motion between the finger and the ball could alter the perceptions conveying the properties of a tex- ture. The following two perception-based studies examined the minimum changes in speed of slip and angle of slip that are detectable by users. The results help us to design haptic patterns as well as our prototype applications. Finally, our preliminary user evaluation indicated that participants wel- comed RollingStone as a useful addition to the range of VR controllers.

The sensation of being able to feel the shape of an object when grasping it in Virtual Reality (VR) enhances a sense of presence and the ease of object manipulation. Though most prior works focus on force feedback on fingers, the haptic emulation of grasping a 3D shape requires the sensation of touch using the entire hand. Hence, we present Pop-up Prop on Palm (PuPoP), a light-weight pneumatic shape-proxy interface worn on the palm that pops several airbags up with predefined primitive shapes for grasping. When a user's hand encounters a virtual object, an airbag of appropriate shape, ready for grasping, is inflated by way of the use of air pumps; the airbag then deflates when the object is no longer in play. Since PuPoP is a physical prop, it can provide the full sensation of touch to enhance the sense of realism for VR object manipulation. For this paper, we first explored the design and implementation of PuPoP with multiple shape structures. We then conducted two user studies to further understand its applicability. The first study shows that, when in conflict, visual sensation tends to dominate over touch sensation, allowing a prop with a fixed size to represent multiple virtual objects with similar sizes. The second study compares PuPoP with controllers and free-hand manipulation in two VR applications. The results suggest that utilization of dynamically-changing PuPoP, when grasped by users in line with the shapes of virtual objects, enhances enjoyment and realism. We believe that PuPoP is a simple yet effective way to convey haptic shapes in VR.

We present Dwell+, a method that boosts the effectiveness of typical dwell selection by augmenting the passive dwell duration with active haptic ticks which promptly drives rapid switches of modes forward through the user's skin sensations. In this way, Dwell+ enables multi-level dwell selection using rapid haptic ticks. To select a mode from a button, users dwell-touch the button until the mode of selection is haptically prompted. Our haptic stimulation design consists of a short 10ms vibrotacile feedback that indicates a mode arriving and a break that separates consecutive modes. We first tested the effectiveness of 170ms, 150ms, 130ms, and 110ms intervals between modes for a 10-level selection. The results reveal that 3-beats-per-chunk rhythm design, e.g., displaying longer 25ms vibrations initially for all three modes, could potentially achieve higher accuracy. The second study reveals significant improvement wherein a 94.5% accuracy was achieved for a 10-level Dwell+ selection using the 170ms interval with 3-beats-per-chunk design, and a 93.82% rate of accuracy using the more frequent 150ms interval with similar chunks for 5-level selection. The performance of conducting touch and receiving vibration from disparate hands was investigated for our final study to provide a wider range of usage. Our applications demonstrated implementing Dwell+ across interfaces, such as text input on a smartwatch, enhancing touch space for HMDs, boosting modalities of stylus-based tool selection, and extending the input vocabulary of physical interfaces.

360-degree video contains a full field of environmental content. However, browsing these videos, either on screens or through head-mounted displays (HMDs), users consume only a subset of the full field of view per a natural viewing experience. This causes a search problem when a region-of-interest (ROI) in a video is outside of the current field of view (FOV) on the screen, or users may search for non-existing ROIs. We propose Outside-In, a visualization technique which re-introduces off-screen regions-of-interest (ROIs) into the main screen as spatial picture-in-picture (PIP) previews. The geometry of the preview windows further encodes a ROI's relative location vis-à-vis the main screen view, allowing for effective navigation. In an 18-participant study, we compare Outside-In with traditional arrow-based guidance within three types of 360-degree video. Results show that Outside-In outperforms in regard to understanding spatial relationship, the storyline of the content and overall preference. Two applications are demonstrated for use with Outside-In in 360-degree video navigation with touchscreens, and live telepresence.

Automatic photo cropping is an important tool for improving visual quality of digital photos without resorting to tedious manual selection. Traditionally, photo cropping is accomplished by determining the best proposal window through visual quality assessment or saliency detection. In essence, the performance of an image cropper highly depends on the ability to correctly rank a number of visually similar proposal windows. Despite the ranking nature of automatic photo cropping, little attention has been paid to learning-to-rank algorithms in tackling such a problem. In this work, we conduct an extensive study on traditional approaches as well as ranking-based croppers trained on various image features. In addition, a new dataset consisting of high quality cropping and pairwise ranking annotations is presented to evaluate the performance of various baselines. The experimental results on the new dataset provide useful insights into the design of better photo cropping algorithms.

Abstract In virtual reality (VR) applications, the contents are usually generated by creating a 360° Video panorama of a real‐world scene. Although many capture devices are being released, getting high‐resolution panoramas and displaying a virtual world in real‐time remains challenging due to its computationally demanding nature. In this paper, we propose a real‐time 360° Video foveated stitching framework, that renders the entire scene in different level of detail, aiming to create a high‐resolution panoramic Video in real‐time that can be streamed directly to the client. Our foveated stitching algorithm takes Videos from multiple cameras as input, combined with measurements of human visual attention (i.e. the acuity map and the saliency map), can greatly reduce the number of pixels to be processed. We further parallelize the algorithm using GPU to achieve a responsive interface and validate our results via a user study. Our system accelerates graphics computation by a factor of 6 on a Google Cardboard display.

Online bookstores have highly thrived and changed consumer behaviors in these years. However, most customers go to online bookstores only when they have specific targets. One reason is that the current web interfaces are usually too complex and cluttered for users to browse. In addition, current visualization interfaces only display the results associated with a single attribute, thus requiring users to interact intensively to find their targets. Inspired by the user experiences (UX) in physical bookstores, we present Bookwall, an online bookstore interface which comprises two components: Category Map and Wall View, enabling users to find their targets more efficiently and releasing users from the burden of complicated operations. Specifically, the category map produces a map with a "natural" map-like look, providing an overview of the clusters and neighborhood of book categories. The wall view enables displaying query results satisfying dual query attributes simultaneously. The results show that Bookwall can provide the users a favourable alternative visualization.

Thumb-to-fingers interfaces augment touch widgets on fingers, which are manipulated by the thumb. Such interfaces are ideal for one-handed eyes-free input since touch widgets on the fingers enable easy access by the stylus thumb. This study presents DigitSpace, a thumb-to-fingers interface that addresses two ergonomic factors: hand anatomy and touch precision. Hand anatomy restricts possible movements of a thumb, which further influences the physical comfort during the interactions. Touch precision is a human factor that determines how precisely users can manipulate touch widgets set on fingers, which determines effective layouts of the widgets. Buttons and touchpads were considered in our studies to enable discrete and continuous input in an eyes-free manner. The first study explores the regions of fingers where the interactions can be comfortably performed. According to the comfort regions, the second and third studies explore effective layouts for button and touchpad widgets. The experimental results indicate that participants could discriminate at least 16 buttons on their fingers. For touchpad, participants were asked to perform unistrokes. Our results revealed that since individual participant performed a coherent writing behavior, personalized $1 recognizers could offer 92% accuracy on a cross-finger touchpad. A series of design guidelines are proposed for designers, and a DigitSpace prototype that uses magnetic-tracking methods is demonstrated.

This work develops a system of spherical magnetic tangibles, GaussMarbles, that exploits the unique affordances of spherical tangibles for interacting with portable physical constraints. The proposed design of each magnetic sphere includes a magnetic polyhedron in the center. The magnetic polyhedron provides bi-polar magnetic fields, which are expanded in equal dihedral angles as robust features for tracking, allowing an analog Hall-sensor grid to resolve the near-surface 3D position accurately in real-time. Possible interactions between the magnetic spheres and portable physical constraints in various levels of embodiment were explored using several example applications.

We present GaussRFID, a hybrid RFID and magnetic-field tag sensing system that supports interactivity when embedded in retrofitted or new physical objects. The system consists of two major components - GaussTag, a magnetic-RFID tag that is combined with a magnetic unit and an RFID tag, and GaussStage, which is a tag reader that is combined with an analog Hall-sensor grid and an RFID reader. A GaussStage recognizes the ID, 3D position, and partial 3D orientation of a GaussTag near the sensing platform, and provides simple interfaces for involving physical constraints, displays and actuators in tangible interaction designs. The results of a two-day toy-hacking workshop reveal that all six groups of 31 participants successfully modified physical toys to interact with computers using the GaussRFID system.

This paper investigates the feasibility of using a nail-mounted array of tactors, NailTactors, as an eyes-free output device. By rim-attached eccentric-rotating-mass (ERM) vibrators to artificial nails, miniature high-resolution tactile displays were realized as an eyes-free output device. To understand how to deliver rich signals to users for valid signal perception, three user studies were conducted. The results suggest that users can not only recognized absolute and relative directional cues, but also recognized numerical characters in EdgeWrite format with an overall 89% recognition rate. Experiments also identified the optimal placement of ERM actuators for maximizing information transfer.

The analog Hall-sensor grid, GaussSense, is a thin-form magnetic-field camera technology for designing expressive occlusion-free, near-surface tangible interactions on conventional portable displays. The studio will provide hands-on experiences that combine physical designs and the GaussSense technology. Through a series of brainstorming and making exercises, participants will learn how to exploit natural hand and micro interactions through designing the expressions and affordances of physical objects, and know how to utilize physical constraints to provide additional kinesthetic awareness and haptic feedback. The exercises will be including form-giving, electronic prototyping, and hacking physical toys that are prepared by either the organizers or participants.

This paper presents EdgeVib, a system of spatiotemporal vibration patterns for delivering alphanumeric characters on wrist-worn vibrotactile displays. We first investigated spatiotemporal pattern delivery through a watch-back tactile display by performing a series of user studies. The results reveal that employing a 2×2 vibrotactile array is more effective than employing a 3×3 one, because the lower-resolution array creates clearer tactile sensations in less time consumption. We then deployed EdgeWrite patterns on a 2×2 vibrotactile array to determine any difficulties of delivering alphanumerical characters, and then modified the unistroke patterns into multistroke EdgeVib ones on the basis of the findings. The results of a 24-participant user study reveal that the recognition rates of the modified multistroke patterns were significantly higher than the original unistroke ones in both alphabet (85.9% vs. 70.7%) and digits (88.6% vs. 78.5%) delivery, and a further study indicated that the techniques can be generalized to deliver two-character compound messages with recognition rates higher than 83.3%. The guidelines derived from our study can be used for designing watch-back tactile displays for alphanumeric character output.

Abstract In this paper, we introduce an interactive method suitable for retargeting both 3D objects and scenes. Initially, the input object or scene is decomposed into a collection of constituent components enclosed by corresponding control bounding volumes which capture the intra‐structures of the object or semantic grouping of objects in the 3D scene. The overall retargeting is accomplished through a constrained optimization by manipulating the control bounding volumes. Without inferring the intricate dependencies between the components, we define a minimal set of constraints that maintain the spatial arrangement and connectivity between the components to regularize the valid retargeting results. The default retargeting behavior can then be easily altered by additional semantic constraints imposed by users. This strategy makes the proposed method highly flexible to process a wide variety of 3D objects and scenes under an unified framework. In addition, the proposed method achieved more general structure‐preserving pattern synthesis in both object and scene levels. We demonstrate the effectiveness of our method by applying it to several complicated 3D objects and scenes.

This paper presents Cyclops, a single-piece wearable device that sees its user's whole body postures through an ego-centric view of the user that is obtained through a fisheye lens at the center of the user's body, allowing it to see only the user's limbs and interpret body postures effectively. Unlike currently available body gesture input systems that depend on external cameras or distributed motion sensors across the user's body, Cyclops is a single-piece wearable device that is worn as a pendant or a badge. The main idea proposed in this paper is the observation of limbs from a central location of the body. Owing to the ego-centric view, Cyclops turns posture recognition into a highly controllable computer vision problem. This paper demonstrates a proof-of-concept device, and an algorithm for recognizing static and moving bodily gestures based on motion history images (MHI) and a random decision forest (RDF). Four example applications of interactive bodily workout, a mobile racing game that involves hands and feet, a full-body virtual reality system, and interaction with a tangible toy are presented. The experiment on the bodily workout demonstrates that, from a database of 20 body workout gestures that were collected from 20 participants, Cyclops achieved a recognition rate of 79% using MHI and simple template matching, which increased to 92% with the more advanced machine learning approach of RDF.

This work presents WonderLens, a system of optical lenses and mirrors for enabling tangible interactions on printed paper. When users perform spatial operations on the optical components, they deform the visual content that is printed on paper, and thereby provide dynamic visual feedback on user interactions without any display devices. The magnetic unit that is embedded in each lens and mirror allows the unit to be identified and tracked using an analog Hall-sensor grid that is placed behind the paper, so the system provides additional auditory and visual feedback through different levels of embodiment, further enhancing the interactivity with the printed content on the physical paper.

In this paper, an inter-video mapping approach is proposed to integrate video footages from two dashcams installed on a preceding and its following vehicle to provide the illusion that the driver of the following vehicle can see-through the preceding one. The key challenge is to adapt the perspectives of the two videos based on a small number of common features since a large portion of the common region in the video captured by the following vehicle is occluded by the preceding one. Inspired by the observation that images with the most similar viewpoints yield dense and high-quality matches, the proposed inter-video mapping estimates spatially-varying motions across two videos utilizing images of very similar contents. Specifically, we estimate frame-to-frame motions of each two consecutive images and incrementally add new views into a merged representation. In this way, long-range motion estimation is achieved, and the observed perspective discrepancy between the two videos can be well approximated our motion estimation. Once the inter-video mapping is established, the correspondences can be updated incrementally, so the proposed method is suitable for on-line applications. Our experiments demonstrate the effectiveness of our approach on real-world challenging videos demonstrate the effectiveness of our approach.

This work presents LEaD, a helmet-based visual guidance system utilizing light movement in scooter drivers' peripheral vision for turn-by-turn navigation. A linear light strip mounted on a helmet navigates for scooter drivers using simple 1D light movement, which can be easily acquired and identified by peripheral vision with the on-going foveal vision task. User studies suggest that this novel system can effectively direct scooter drivers without introducing visual distractions in route-guided experiences.

In this paper, we explored eyes-free gesture interactions on palms, which enables users to interact with devices by drawing stroke gestures on palms without looking at palms. We conducted a 24-person user study to understand how users draw gestures on the palm with varying characteristics including regions, orientation and starting points. Based on the findings, we proposed two new interaction techniques for palm-based gesture interface. To explore and demonstrate the feasibility of the interaction, we implemented EyeWrist, a wrist-mounted prototype which detects gestures on palms by using an IR camera and laser-line projector. The preliminary evaluation revealed that EyeWrist enabled users to draw graffiti letter and multi-stroke gestures with above 90% accuracy and that both the concept of using palms as gesture interfaces for eyes-free input and the proposed two interaction techniques were appealing to users.

This paper presents CyclopsRing, a ring-style fisheye imaging wearable device that can be worn on hand webbings to en- able whole-hand and context-aware interactions. Observing from a central position of the hand through a fisheye perspective, CyclopsRing sees not only the operating hand, but also the environmental contexts that involve with the hand-based interactions. Since CyclopsRing is a finger-worn device, it also allows users to fully preserve skin feedback of the hands. This paper demonstrates a proof-of-concept device, reports the performance in hand-gesture recognition using random decision forest (RDF) method, and, upon the gesture recognizer, presents a set of interaction techniques including on-finger pinch-and-slide input, in-air pinch-and-motion input, palm-writing input, and their interactions with the environ- mental contexts. The experiment obtained an 84.75% recognition rate of hand gesture input from a database of seven hand gestures collected from 15 participants. To our knowledge, CyclopsRing is the first ring-wearable device that supports whole-hand and context-aware interactions.

This paper presents FlexiBend, an easily installable shape-sensing strip that enables interactivity of multi-part, deformable fabrications. The flexible sensor strip is composed of a dense linear array of strain gauges, therefore it has shape sensing capability. After installation, FlexiBend can simultaneously sense user inputs in different parts of a fabrication or even capture the geometry of a deformable fabrication.

A person's handwriting appears differently within a typical range of variations, and the shapes of handwriting characters also show complex interaction with their nearby neighbors. This makes automatic synthesis of handwriting characters and paragraphs very challenging. In this paper, we propose a method for synthesizing handwriting texts according to a writer's handwriting style. The synthesis algorithm is composed by two phases. First, we create the multidimensional morphable models for different characters based on one writer's data. Then, we compute the cursive probability to decide whether each pair of neighboring characters are conjoined together or not. By jointly modeling the handwriting style and conjoined property through a novel trajectory optimization, final handwriting words can be synthesized from a set of collected samples. Furthermore, the paragraphs’ layouts are also automatically generated and adjusted according to the writer's style obtained from the same dataset. We demonstrate that our method can successfully synthesize an entire paragraph that mimic a writer's handwriting using his/her collected handwriting samples.

We present an algorithm that integrates image co-segmentation into feature matching, and can robustly yield accurate and dense feature correspondences. Inspired by the fact that correct feature correspondences on the same object typically have coherent transformations, we cast the task of feature matching as a density estimation problem in the homography space. Specifically, we project the homographies of correspondence candidates into the parametric Hough space, in which geometric verification of correspondences can be activated by voting. The precision of matching is then boosted. On the other hand, we leverage image co-segmentation, which discovers object boundaries, to determine relevant voters and speed up Hough voting. In addition, correspondence enrichment can be achieved by inferring the concerted homographies that are propagated between the features within the same segments. The recall is hence increased. In our approach, feature matching and image co-segmentation are tightly coupled. Through an iterative optimization process, more and more correct correspondences are detected owing to object boundaries revealed by co-segmentation. The proposed approach is comprehensively evaluated. Promising experimental results on four datasets manifest its effectiveness.

With the aim to improve the performance of feature matching, we present an unsupervised approach for adaptive description selection in the space of homographies. Inspired by the observation that the homographies of correct feature correspondences vary smoothly along the spatial domain, our approach stands on the unsupervised nature of feature matching, and can choose a good descriptor locally for matching each feature point, instead of using one global descriptor. To this end, the homography space serves as the domain for selecting various heterogeneous descriptors. Correspondences obtained by any descriptors are considered as points in the space, and their geometric coherence and spatial continuity are measured via computing the geodesic distances. In this way, mutual verification across different descriptors is allowed, and correct correspondences will be highlighted with a high degree of consistency short geodesic distances here. It follows that one-class SVM can be applied to identifying these correct correspondences, and achieves adaptive descriptor selection. The proposed approach is comprehensively compared with the state-of-the-art approaches, and evaluated on five benchmarks of image matching. The promising results manifest its effectiveness.

Building LEGO sculptures requires accounting for the target object's shape, colors, and stability. In particular, finding a good layout of LEGO bricks that prevents the sculpture from collapsing (due to its own weight) is usually challenging, and it becomes increasingly difficult as the target object becomes larger or more complex. We devise a force-based analysis for estimating physical stability of a given sculpture. Unlike previous techniques for Legolization, which typically use heuristic-based metrics for stability estimation, our force-based metric gives 1) an ordering in the strength so that we know which structure is more stable, and 2) a threshold for stability so that we know which one is stable enough. In addition, our stability analysis tells us the weak portion of the sculpture. Building atop our stability analysis, we present a layout refinement algorithm that iteratively improves the structure around the weak portion, allowing for automatic generation of a LEGO brick layout from a given 3D model, accounting for color information, required workload (in terms of the number of bricks) and physical stability. We demonstrate the success of our method with real LEGO sculptures built up from a wide variety of 3D models, and compare against previous methods.

This paper presents a patch-based synthesis framework for stereoscopic image editing. The core of the proposed method builds upon a patch-based optimization framework with two key contributions: First, we introduce a depth-dependent patch-pair similarity measure for distinguishing and better utilizing image contents with different depth structures. Second, a joint patch-pair search is proposed for properly handling the correlation between two views. The proposed method successfully overcomes two main challenges of editing stereoscopic 3D media: (1) maintaining the depth interpretation, and (2) providing controllability of the scene depth. The method offers patch-based solutions to a wide variety of stereoscopic image editing problems, including depth-guided texture synthesis, stereoscopic NPR, paint by depth, content adaptation, and 2D to 3D conversion. Several challenging cases are demonstrated to show the effectiveness of the proposed method. The results of user studies also show that the proposed method produces stereoscopic images with good stereoscopics and visual quality.

Email archives contain rich information about how we interact with different contacts and how events evolve throughout time. Making sense of the archived messages can be a good way to understand how things evolved and progressed in the past. Although much work has been devoted to email visualization, most work has focused on presenting one of the two aspects of email archives: discovering the evolution of emails and events, or the relationship between the email owner and his/her contacts over time. In this paper, we present Email Map, an email visualization which integrates the information of both events and contacts into a single view, enabling users to make sense of their email archives with complementary contextual information. Two visualization components are designed to portray complex information within the email archives: event flow and contact tracks. The event flow illustrates the evolution of past events, helping the users to grasp high-level pictures and patterns of their email archives. The contact tracks reveal the interaction between the email owner and his/her contacts.

This work describes a novel building block system for tangible interaction design, GaussBricks, which enables real-time constructive tangible interactions on portable displays. Given its simplicity, the mechanical design of the magnetic building blocks facilitates the construction of configurable forms. The form constructed by the magnetic building blocks, which are connected by the magnetic joints, allows users to stably manipulate with various elastic force feedback mechanisms. With an analog Hall-sensor grid mounted to its back, a portable display determines the geometrical configuration and detects various user interactions in real time. This work also introduce several methods to enable shape changing, multi-touch input, and display capabilities in the construction. The proposed building block system enriches how individuals interact with the portable displays physically.

This work presents GaussStones, a system of shielded magnetic tangibles design for supporting multi-token interactions on portable displays. Unlike prior works in sensing magnetic tangibles on portable displays, the proposed tangible design applies magnetic shielding by using an inexpensive galvanized steel case, which eliminates interference between magnetic tangibles. An analog Hall-sensor grid can recognize the identity of each shielded magnetic unit since each unit generates a magnetic field with a specific intensity distribution and/or polarization. Combining multiple units as a knob further allows for resolving additional identities and their orientations. Enabling these features improves support for applications involving multiple tokens. Thus, using prevalent portable displays provides generic platforms for tangible interaction design.

Abstract Visual obstruction caused by a preceding vehicle is one of the key factors threatening driving safety. One possible solution is to share the first‐person‐view of the preceding vehicle to unveil the blocked field‐of‐view of the following vehicle. However, the geometric inconsistency caused by the camera‐eye discrepancy renders view sharing between different cars a very challenging task. In this paper, we present a first‐person‐perspective image rendering algorithm to solve this problem. Firstly, we contour unobstructed view as the transferred region, then by iteratively estimating local homography transformations and performing perspective‐adaptive warping using the estimated transformations, we are able to locally adjust the shape of the unobstructed view so that its perspective and boundary could be matched to that of the occluded region. Thus, the composited view is seamless in both the perceived perspective and photometric appearance, creating an impression as if the preceding vehicle is transparent. Our system improves the driver's visibility and thus relieves the burden on the driver, which in turn increases comfort. We demonstrate the usability and stability of our system by performing its evaluation with several challenging data sets collected from real‐world driving scenarios.

We present a technique for computing the shape of a transparent object that can generate user-defined caustic patterns. The surface of the object generated using our method is smooth. Thanks to this property, the resulting caustic pattern is smooth, natural, and highly detailed compared to the results btained using previous methods. Our method consists of two processes. First, we use a differential geometry approach to compute a smooth mapping between the distributions of the incident light and the light reaching the screen. Second, we utilize this mapping to compute the surface of the object. We solve Poisson's equation to compute both the mapping and the surface of the object.

We present GaussBits, which is a system of the passive magnetic tangible designs that enables 3D tangible interactions in the near-surface space of portable displays. When a thin magnetic sensor grid is attached to the back of the display, the 3D position and partial 3D orientation of the GaussBits can be resolved by the proposed bi-polar magnetic field tracking technique. This portable platform can therefore enrich tangible interactions by extending the design space to the near-surface space. Since non-ferrous materials, such as the user's hand, do not occlude the magnetic field, interaction designers can freely incorporate a magnetic unit into an appropriately shaped non-ferrous object to exploit the metaphors of the real-world tasks, and users can freely manipulate the GaussBits by hands or using other non-ferrous tools without causing interference. The presented example applications and the collected feedback from an explorative workshop revealed that this new approach is widely applicable.

This work presents a novel and always-available nail mounted display known as NailDisplay. The proposed display augments the use of a finger by allowing for always-available visual feedback owing to its fast accessibility and binding user controls with the display, i.e. what you control is what you see (through the display). Potential benefits of NailDisplay are demonstrated in three applications: from displaying to combining it with user controls. In the first application, NailDisplay can reveal what is occluded under a finger touch, making it a solution to operate small UI elements. In the second application, NailDisplay is complementary to an imaginary interface, helping users to learn an imaginary interface (e.g., on the users' arms) and allowing them to reassure the interface when their memory of it becomes unclear. In the third application, NailDisplay is integrated with rich finger interactions, such as swiping in the air. We also report users' feedbacks gathered from an explorative user study.

We present an algorithm that carries out alternate Hough transform and inverted Hough transform to establish feature correspondences, and enhances the quality of matching in both precision and recall. Inspired by the fact that nearby features on the same object share coherent homographies in matching, we cast the task of feature matching as a density estimation problem in the Hough space spanned by the hypotheses of homographies. Specifically, we project all the correspondences into the Hough space, and determine the correctness of the correspondences by their respective densities. In this way, mutual verification of relevant correspondences is activated, and the precision of matching is boosted. On the other hand, we infer the concerted homographies propagated from the locally grouped features, and enrich the correspondence candidates for each feature. The recall is hence increased. The two processes are tightly coupled. Through iterative optimization, plausible enrichments are gradually revealed while more correct correspondences are detected. Promising experimental results on three benchmark datasets manifest the effectiveness of the proposed approach.

Thanks to the advanced digital technologies, people can take pictures everyday to record their lives. However, many people almost have no time to browse these tremendous amount of photos and no good way to revisit the treasured photos. Though many methods can display the photos, people still need to allocate a specific timeslot to browse them. To address this problem, a novel design concept, The memory, is developed to utilize users' photos to synthesize program themes, so that the photos will become a part of users' working space and evoke their valuable memories while using their computers. As a user-centric design process, we first recruited 10 professional designers to design program themes according to our described concept, which were further ranked by 31 recruited users. The top ranked templates were implemented according to the extracted design guidelines through the ranked results.

We present FingerPad, a nail-mounted device that turns the tip of the index finger into a touchpad, allowing private and subtle interaction while on the move. FingerPad enables touch input using magnetic tracking, by adding a Hall sensor grid on the index fingernail, and a magnet on the thumbnail. Since it permits input through the pinch gesture, FingerPad is suitable for private use because the movements of the fingers in a pinch are subtle and are naturally hidden by the hand. Functionally, FingerPad resembles a touchpad, and also allows for eyes-free use. Additionally, since the necessary devices are attached to the nails, FingerPad preserves natural haptic feedback without affecting the native function of the fingertips. Through user study, we analyze the three design factors, namely posture, commitment method and target size, to assess the design of the FingerPad. Though the results show some trade-off among the factors, generally participants achieve 93% accuracy for very small targets (1.2mm-width) in the seated condition, and 92% accuracy for 2.5mm-width targets in the walking condition.

Abstract A QR code is a two‐dimensional barcode that encodes information. A standard QR code contains only regular black and white squares, and thus is unattractive. This paper proposes a novel framework for embellishing a standard QR code, to make it both attractive and recognizable by any human while maintaining its scanability. The proposed method is inspired by artistic methods. A QR code is typically embellished by stylizing the squares and embedding images into it. In the proposed framework, the regular squares are reshaped using a binary examplar, to make their local appearances resemble the example shape. Additionally, an error‐aware warping technique for deforming the embedded image is proposed; it minimizes the error in the QR code that is generated by the embedding of the image to optimize the readability of the code. The proposed algorithm yields lower data error than previous global transformation techniques because the warping can locally deform the embedded image to conform to the squares that surround it. The proposed framework was examined by using it to embellish an extensive set of QR codes and to test the readability with various commercial QR code readers.

Abstract Creating variations of an image object is an important task, which usually requires manipulating the skeletal structure of the object. However, most existing methods (such as image deformation) only allow for stretching the skeletal structure of an object: modifying skeletal topology remains a challenge. This paper presents a technique for synthesizing image objects with different skeletal structures while respecting to an input image object. To apply this technique, a user firstly annotates the skeletal structure of the input object by specifying a number of strokes in the input image, and draws corresponding strokes in an output domain to generate new skeletal structures. Then, a number of the example texture pieces are sampled along the strokes in the input image and pasted along the strokes in the output domain with their orientations. The result is obtained by optimizing the texture sampling and seam computation. The proposed method is successfully used to synthesize challenging skeletal structures, such as skeletal branches, and a wide range of image objects with various skeletal structures, to demonstrate its effectiveness.

This paper presents dynamic media assemblage, which is a new presentation and summarization method for images and videos on a 2-D canvas. Instead of using the keyframes of the videos to generate a still image summarization, our method allows the videos to play simultaneously on the canvas while utilizing the limited space efficiently. This technique uses an efficient iterative packing algorithm, and as a result is well suited for interactive manipulations of media files within the assemblages in real time, such as insertion, deletion, and rearrangement. Our method starts by detecting shot boundaries and dividing longer input videos into individual shots. Within each shot, its temporal-spatial salient regions are extracted and used to recover camera motions. These saliency information further defines important regions within individual videos or images, which allows us to preserve visually important regions while packing the media files more efficiently into media assemblages. Our algorithm is iterative and can therefore quickly adjust to new canvas sizes or other user intentions. We demonstrate the effectiveness of our techniques by applying our methods to several applications, including media collection presentation, single video dynamic summary, personal media file browser, and interactive video wall.

This work presents GaussSense, which is a back-of-device sensing technique for enabling input on an arbitrary surface using stylus by exploiting magnetism. A 2mm-thick Hall sensor grid is developed to sense magnets that are embedded in the stylus. Our system can sense the magnetic field that is emitted from the stylus when it is within 2cm of any non-ferromagnetic surface. Attaching the sensor behind an arbitrary thin surface enables the stylus input to be recognized by analyzing the distribution of the applied magnetic field. Attaching the sensor grid to the back of a touchscreen device and incorporating magnets into the corresponding stylus enable the system 1) to distinguish touch events that are caused by a finger from those caused by the stylus, 2) to sense the tilt angle of the stylus and the pressure with which it is applied, and 3) to detect where the stylus hovers over the screen. A pilot study reveals that people were satisfied with the novel sketching experiences based on this system.

Abstract Super‐deformed, SD, is a specific artistic style for Japanese manga and anime which exaggerates characters in the goal of appearing cute and funny. The SD style characters are widely used, and can be seen in many anime, CG movies, or games. However, to create an SD model often requires professional skills and considerable time and effort. In this paper, we present a novel technique to generate an SD style counterpart of a normal 3D character model. Our approach uses an optimization guided by a number of constraints that can capture the properties of the SD style. Users can also customize the results by specifying a small set of parameters related to the body proportions and the emphasis of the signature characteristics. With our technique, even a novel user can generate visually pleasing SD models in seconds.

Abstract Pixel art is a kind of digital art that through per‐pixel manipulation enables production of a diverse array of artistic images. In this paper, we present a new way for people to experience and express pixel art. Our digital art consists of a set of sticks made of acrylate resin, each of which refracts light from a parallel light source, in certain directions. Artistic users are able to easily rearrange these sticks and view their digital art through the refracted light projection on any planar surface. As we demonstrate in this paper, a user can generate various artistic images using only a single set of sticks. We additionally envision that our pixel art with rearrangeable sticks would have great entertainment appeal, e.g., as an art puzzle.

ABSTRACT The linear blendshape technique has been intensively used for computer animation and games because of its simplicity and effectiveness. However, it cannot describe rotational deformations and deformations because of self collision or scene interaction. In this paper, we present a new technique to address these two major limitations by introducing physical‐based simulation to blendshapes. The proposed technique begins by constructing a mass–spring system for each blendshape target. Each system is initialized in its steady state by setting the rest length of each spring as the edge length of the corresponding target. To begin shape interpolation, we linearly interpolate the rest lengths of the springs according to a given interpolation factorα ∈ [0,1]. The interpolated shape is then generated by computing the equilibrium of the mass–spring system with the interpolated rest lengths. Results from our technique show physically plausible deformations even in the case of large rotations between blendshape targets. In addition, the new blendshape model is able to interact with other scene elements by introducing collision detection and handling to the mass–spring system. Copyright © 2012 John Wiley & Sons, Ltd.

ABSTRACT String‐like objects in our daily lives, for example shoelaces, threads, rubber cords, plastic fiber and spaghetti, have a wide variety of materials. Such string‐like objects also exhibit interesting behaviors such as twisting, tearing (by stretching or twisting), and bouncing back when pulled and released. In this paper, we present a method that enables these behaviors and simulates such materials in traditional string simulation methods that explicitly represent a string by particles and segments. Specifically, we offer the following three contributions. First, we introduce a method for handling twisting effects with both uniform and non‐uniform torsional rigidities. Second, we propose a method for estimating the tension acting on inextensible strings in order to reproduce tearing and flicking (bouncing back), whereas the tension for anextensibleobject can be computed via stretched length. The length of aninextensibleobject is maintained constant in general, and thus, we need a novel approach. Third, we introduce an optimized grid‐based collision detection for accelerating the computation. We demonstrate that our method can produce visually plausible animations of string‐like objects with various material properties, and it is a fast framework for interactive applications such as games. Copyright © 2012 John Wiley & Sons, Ltd.

Character speech animation is traditionally considered as important but tedious work, especially when taking lip synchronization (lip-sync) into consideration. Although there are some methods proposed to ease the burden on artists to create facial and speech animation, almost none is fast and efficient. In this paper, we introduce a framework for synthesizing lip-sync character speech animation in real time from a given speech sequence and its corresponding texts, starting from training dominated animeme models (DAMs) for each kind of phoneme by learning the character's animation control signal through an expectation-maximization (EM)-style optimization approach. The DAMs are further decomposed to polynomial-fitted animeme models and corresponding dominance functions while taking coarticulation into account. Finally, given a novel speech sequence and its corresponding texts, the animation control signal of the character can be synthesized in real time with the trained DAMs. The synthesized lip-sync animation can even preserve exaggerated characteristics of the character's facial geometry. Moreover, since our method can perform in real time, it can be used for many applications, such as lip-sync animation prototyping, multilingual animation reproduction, avatar speech, and mass animation production. Furthermore, the synthesized animation control signal can be imported into 3-D packages for further adjustment, so our method can be easily integrated into the existing production pipeline.

This paper presents a novel technique for seamless stereoscopic image cloning, which performs both shape adjustment and color blending such that the stereoscopic composite is seamless in both the perceived depth and color appearance. The core of the proposed method is an iterative disparity adaptation process which alternates between two steps: disparity estimation, which re-estimates the disparities in the gradient domain so that the disparities are continuous across the boundary of the cloned region; and perspective-aware warping, which locally re-adjusts the shape and size of the cloned region according to the estimated disparities. This process guarantees not only depth continuity across the boundary but also models local perspective projection in accordance with the disparities, leading to more natural stereoscopic composites. The proposed method allows for easy cloning of objects with intricate silhouettes and vague boundaries because it does not require precise segmentation of the objects. Several challenging cases are demonstrated to show that our method generates more compelling results compared to methods with only global shape adjustment.

Graph visualization has been widely used to understand and present both global structural and local adjacency information in relational data sets (e.g., transportation networks, citation networks, or social networks). Graphs with dense edges, however, are difficult to visualize because fast layout and good clarity are not always easily achieved. When the number of edges is large, edge bundling can be used to improve the clarity, but in many cases, the edges could be still too cluttered to permit correct interpretation of the relations between nodes. In this paper, we present an ambiguity-free edge-bundling method especially for improving local detailed view of a complex graph. Our method makes more efficient use of display space and supports detail-on-demand viewing through an interactive interface. We demonstrate the effectiveness of our method with public coauthorship network data.

This paper presents a novel interaction system, PUB (Point Upon Body), to explore eyes-free interaction in a personal space by allowing users tapping on their own arms to be provided with haptic feedback from their skin. Two user studies determine how users can interact precisely with their forearms and how users behave when operating in their arm space. According to those results, normal users can divide their arm space at most into 6 points between their wrists and elbows with iterative practice. Experimental results also indicate that the divided pattern of each user is unique from that of other ones. Based on the design principles from the observations, an interaction system, PUB, is designed to demonstrate how interaction design benefits from those findings. Two scenarios, remote display control and mobile device control, are demonstrated through the UltraSonic device attached on the users' wrists to detect their tapped positions.

Abstract Image blur caused by object motion attenuates high frequency content of images, making post‐capture deblurring an ill‐posed problem. The recoverable frequency band quickly becomes narrower for faster object motion as high frequencies are severely attenuated and virtually lost. This paper proposes to translate a camera sensor circularly about the optical axis during exposure, so that high frequencies can be preserved for a wide range of in‐plane linear object motion in any direction within some predetermined speed. That is, although no object may be photographed sharply at capture time, differently moving objects captured in a single image can be deconvolved with similar quality. In addition, circular sensor motion is shown to facilitate blur estimation thanks to distinct frequency zero patterns of the resulting motion blur point‐spread functions. An analysis of the frequency characteristics of circular sensor motion in relation to linear object motion is presented, along with deconvolution results for photographs captured with a prototype camera.

Abstract In this paper, we propose bipartite polar classification to augment an input unorganized point set ℘ with two disjoint groups of points distributed around the ambient space of ℘ to assist the task of surface reconstruction. The goal of bipartite polar classification is to obtain a space partitioning of ℘ by assigning pairs of Voronoi poles into two mutually invisible sets lying in the opposite sides of ℘ through direct point set visibility examination. Based on the observation that a pair of Voronoi poles are mutually invisible, spatial classification is accomplished by carving away visible exterior poles with their counterparts simultaneously determined as interior ones. By examining the conflicts of mutual invisibility, holes or boundaries can also be effectively detected, resulting in a hole‐aware space carving technique. With the classified poles, the task of surface reconstruction can be facilitated by more robust surface normal estimation with global consistent orientation and off‐surface point specification for variational implicit surface reconstruction. We demonstrate the ability of the bipartite polar classification to achieve robust and efficient space carving on unorganized point clouds with holes and complex topology and show its application to surface reconstruction.

Abstract This paper proposes a novel system that “rephotographs” a historical photograph with a collection of images. Rather than finding the accurate viewpoint of the historical photo, users only need to take a number of photographs around the target scene. We adopt the structure from motion technique to estimate the spatial relationship among these photographs, and construct a set of 3D point cloud. Based on the user‐specified correspondences between the projected 3D point cloud and historical photograph, the camera parameters of the historical photograph are estimated. We then combine forward and backward warping images to render the result. Finally, inpainting and content‐preserving warping are used to refine it, and the photograph at the same viewpoint of the historical one is produced by this photo collection.

Abstract Photo‐realistic rendering of inhomogeneous participating media with light scattering in consideration is important in computer graphics, and is typically computed using Monte Carlo based methods. The key technique in such methods is the free path sampling, which is used for determining the distance (free path) between successive scattering events. Recently, it has been shown that efficient and unbiased free path sampling methods can be constructed based on Woodcock tracking. The key concept for improving the efficiency is to utilize space partitioning (e.g., kd‐tree or uniform grid), and a better space partitioning scheme is important for better sampling efficiency. Thus, an estimation framework for investigating the gain in sampling efficiency is important for determining how to partition the space. However, currently, there is no estimation framework that works in 3D space. In this paper, we propose a new estimation framework to overcome this problem. Using our framework, we can analytically estimate the sampling efficiency for any typical partitioned space. Conversely, we can also use this estimation framework for determining the optimal space partitioning. As an application, we show that new space partitioning schemes can be constructed using our estimation framework. Moreover, we show that the differences in the performances using different schemes can be predicted fairly well using our estimation framework.

This work describes a novel approach to utilizing everyday objects of users as additional, auxiliary, and instant tabletop controllers. Based on this approach, a prototype platform, called iCon, is developed to explore the possible design. Field studies and user studies reveal that utilizing everyday objects such as auxiliary input devices might be appropriate under a multi-task scenario. User studies further demonstrate that daily objects can generally be applied in low precision circumstances, low engagement with selected objects, and medium-to-high frequency of use. The proposed approach allows users to interact with computers while not altering their original work environments.

Abstract Given a complete unoriented point set, we propose a binary orientation tree (BOT) for volume and surface representation, which roughly splits the space into the interior and exterior regions with respect to the input point set. The BOTs are constructed by performing a traditional octree subdivision technique while the corners of each cell are associated with a tag indicating thein/outrelationship with respect to the input point set. Starting from the root cell, a growing stage is performed to efficiently assign tags to the connected empty sub‐cells. The unresolved tags of the remaining cell corners are determined by examining their visibility via the hidden point removal operator. We show that the outliers accompanying the input point set can be effectively detected during the construction of the BOTs. After removing the outliers and resolving thein/outtags, the BOTs are ready to support any volume or surface representation techniques. To represent the surfaces, we also present a modified MPU implicits algorithm enabled to reconstruct surfaces from the input unoriented point clouds by taking advantage of the BOTs.

Realistic rendering of participating media is one of the major subjects in computer graphics. Monte Carlo techniques are widely used for realistic rendering because they provide unbiased solutions, which converge to exact solutions. Methods based on Monte Carlo techniques generate a number of light paths, each of which consists of a set of randomly selected scattering events. Finding a new scattering event requires free path sampling to determine the distance from the previous scattering event, and is usually a time-consuming process for inhomogeneous participating media. To address this problem, we propose an adaptive and unbiased sampling technique using kd-tree based space partitioning. A key contribution of our method is an automatic scheme that partitions the spatial domain into sub-spaces (partitions) based on a cost model that evaluates the expected sampling cost. The magnitude of performance gain obtained by our method becomes larger for more inhomogeneous media, and rises to two orders compared to traditional free path sampling techniques.

In this paper we propose a new video interaction model called adaptive fast-forwarding to help people quickly browse videos with predefined semantic rules. This model is designed around the metaphor of scenic car driving, in which the driver slows down near areas of interest and speeds through unexciting areas. Results from a preliminary user study of our video player suggest the following: (1) the player should adaptively adjust the current playback speed based on the complexity of the present scene and predefined semantic events; (2) the player should learn user preferences about predefined event types as well as a suitable playback speed; (3) the player should fast-forward the video continuously with a playback rate acceptable to the user to avoid missing any undefined events or areas of interest. Furthermore, our user study results suggest that for certain types of video, our SmartPlayer yields better user experiences in browsing and fast-forwarding videos than existing video players' interaction models.

This paper proposes a new approach for video stabilization. Most existing video stabilization methods adopt a framework of three steps, motion estimation, motion compensation and image composition. Camera motion is often estimated based on pairwise registration between frames. Thus, these methods often assume static scenes or distant backgrounds. Furthermore, for scenes with moving objects, robust methods are required for finding the dominant motion. Such assumptions and judgements could lead to errors in motion parameters. Errors are compounded by motion compensation which smoothes motion parameters. This paper proposes a method to directly stabilize a video without explicitly estimating camera motion, thus assuming neither motion models nor dominant motion. The method first extracts robust feature trajectories from the input video. Optimization is then performed to find a set of transformations to smooth out these trajectories and stabilize the video. In addition, the optimization also considers quality of the stabilized video and selects a video with not only smooth camera motion but also less unfilled area after stabilization. Experiments show that our method can deal with complicated videos containing near, large and multiple moving objects.

In traditional image composition methods for cutting out a source object from a source image and pasting it onto a target image, users have to segment a foreground object in a target image when they want to partially hide a source object behind it. While recent image editing tools greatly facilitate segmentation operations, it can be tedious to segment each object if users try to place a source object in various positions in a target image before obtaining a satisfying composition. We propose a method which allows users to drag a source object and slip it behind a target object as shown in Fig. 1, so that users can move a source object around without manually segmenting each part of a target image.

Abstract Woven cloth can commonly be seen in daily life and also in animation. Unless prevented in some way, woven cloth usually frays at the edges. However, in computer graphics, woven cloth is typically modeled as a continuum sheet, which is not suitable for representing frays. This paper proposes a model that allows yarn movement and slippage during cloth tearing. Drawing upon techniques from textile and mechanical engineering fields, we model cloth as woven yarn crossings where each yarn can be independently torn when the strain limit is reached. To make the model practical for graphics applications, we simulate only tearing part of cloth with a yarn‐level model using a simple constrained mass‐spring system for computational efficiency. We designed conditions for switching from a standard continuum sheet model to our yarn‐level model, so that frays can be initiated and propagated along the torn lines. Results show that our method can achieve plausible tearing cloth animation with frayed edges.

Abstract A rendering system for interior scenes is proposed in this paper. The light reaches the interior scene, usually through small regions, such as windows or abat‐jours, which we call portals. To provide a solution, suitable for rendering interior scenes with portals, we extend the traditional precomputed radiance transfer approaches. In our approach, a bounding sphere, which we call a shell, of the interior, centered at each portal, is created and the light transferred from the shell towards the interior through the portal is precomputed. Each shell acts as an environment light source and its intensity distribution is determined by rendering images of the scene, viewed from the center of the shell. By updating the intensity distribution of the shell at each frame, we are able to handle dynamic objects outside the shells. The material of the portals can also be modified at run time (e.g. changing from transparent glass to frosted glass). Several applications are shown, including the illumination of a cathedral, lit by skylight at different times of a day, and a car, running in a town, at interactive frame rates, with a dynamic viewpoint.

Interactive 3D content on Internet has yet become popular due to its typically large volume and the limited network bandwidth. Progressive content transmission, or 3D streaming, thus is necessary to enable real-time content interactions. However, the heavy data and processing requirements of 3D streaming challenge the scalability of client-server delivery methods. We propose the use of peer-to-peer (P2P) networks for 3D streaming, and argue that due to the non-linear access patterns of 3D content, P2P 3D streaming is a new class of applications apart from existing media streaming and requires new investigations. We also present FLoD, the first P2P 3D streaming framework that allows clients of 3D virtual globe or virtual environment (VE) applications to obtain relevant data from other clients while minimizing server resource usage. To demonstrate how FLoD applies to real-world scenarios, we build a prototype system that adapts JPEG 2000-based 3D mesh streaming for P2P delivery. Experiments show that server-side bandwidth usage can thus be reduced, while simulations indicate that P2P 3D streaming is fundamentally more scalable than client-server approaches.

Abstract Annoying shaky motion is one of the significant problems in home videos, since hand shake is an unavoidable effect when capturing by using a hand‐held camcorder. Video stabilization is an important technique to solve this problem, but the stabilized videos resulting from some current methods usually have decreased resolution and are still not so stable. In this paper, we propose a robust and practical method of full‐frame video stabilization while considering user's capturing intention to remove not only the high frequency shaky motions but also the low frequency unexpected movements. To guess the user's capturing intention, we first consider the regions of interest in the video to estimate which regions or objects the user wants to capture, and then use a polyline to estimate a new stable camcorder motion path while avoiding the user's interested regions or objects being cut out. Then, we fill the dynamic and static missing areas caused by frame alignment from other frames to keep the same resolution and quality as the original video. Furthermore, we smooth the discontinuous regions by using a three‐dimensional Poisson‐based method. After the above automatic operations, a full‐frame stabilized video can be achieved and the important regions and objects can also be preserved.

Abstract This paper investigates a new approach for color transfer. Rather than transferring color from one image to another globally, we propose a system with a stroke‐based user interface to provide a direct indication mechanism. We further present a multiple local color transfer method. Through our system the user can easily enhance a defect (source) photo by referring to some other good quality (target) images by simply drawing some strokes. Then, the system will perform the multiple local color transfer automatically. The system consists of two major steps. First, the user draws some strokes on the source and target images to indicate corresponding regions and also the regions he or she wants to preserve. The regions to be preserved which will be masked out based on an improved graph cuts algorithm. Second, a multiple local color transfer method is presented to transfer the color from the target image(s) to the source image through gradient‐guided pixel‐wise color transfer functions. Finally, the defect (source) image can be enhanced seamlessly by multiple local color transfer based on some good quality (target) examples through an interactive and intuitive stroke‐based user interface.

Wedding is one of the most important ceremonies in our lives. It symbolizes the birth and creation of a new family. In this paper, we present a system for automatically segmenting a wedding ceremony video into a sequence of recognizable wedding events, e.g., the couple's wedding kiss. Our goal is to develop an automatic tool that helps users to efficiently organize, search, and retrieve his/her treasured wedding memories. Furthermore, the obtained event descriptions could benefit and complement the current research in semantic video understanding. Based on the knowledge of wedding customs, a set of audiovisual features, relating to the wedding contexts of speech/music types, applause activities, picture-taking activities, and leading roles, are exploited to build statistical models for each wedding event. Thirteen wedding events are then recognized by a hidden Markov model, which takes into account both the fitness of observed features and the temporal rationality of event ordering to improve the segmentation accuracy. We conducted experiments on a collection of wedding videos and the promising results demonstrate the effectiveness of our approach. Comparisons with conditional random fields show that the proposed approach is more effective in this application domain.

This paper presents a method for automatically extracting a scene depth map and the alpha matte of a foreground object by capturing a scene through RGB color filters placed in the camera lens aperture. By dividing the aperture into three regions through which only light in one of the RGB color bands can pass, we can acquir three shifted views of a scene in the RGB planes of an image in a single exposure. In other words, a captured image has depth-dependent color misalignment. We develop a color alignment measure to estimate disparities between the RGB planes for depth reconstruction. We also exploit color misalignment cues in our matting algorithm in order to disambiguate between the foreground and background regions even where their colors are similar. Based on the extracted depth and matte, the color misalignment in the captured image can be canceled, and various image editing operations can be applied to the reconstructed image, including novel view synthesis, postexposure refocusing, and composition over different backgrounds.

In this paper, we present an interactive system to transfer and concatenate motions in different articulated figures, such as human and dog. This system first constructs a union skeleton that contains both bone structures with bone correspondence manually assigned by users, and then aligns the initial poses of these two skeletons. With the initial poses aligned, motion data can be transferred between them. In addition, by blending similar poses of different skeletons, we can produce a seamless transition sequence which can be used to drive a meta-mesh and generate an animated morphing result.

In this paper, we propose a new framework for developing 3D graphic applications on portable devices such as cellular phones or personal digital assistants (PDAs). Our framework is based on J2ME (Java 2 Platform, Micro Edition) environment, an application environment that specifically addresses the needs of developing programs on embedded systems using Java, and we leverage the advantages of J2ME to build our platform-independent 3D graphic library on mobile devices. Adopting this framework, the 3D program developers can not only implement a cross-platform 3D program easily as they did using OpenGL ES2 library but also gain the powerful features including efficient memory usage and shading functionalities.

Creating lifelike, autonomous, and interactive virtual behaviors is important in generating character animation, such as animal crowds, pedestrians, battle scenes, etc. Unfortunately, such task has long been limited to skilled users, since the authoring tools, including script languages and other commercial programs, mostly require lengthy prelearning process or are difficult to use. A novel approach, gesture-based behavior authoring, is proposed to open the interesting experience of creating autonomous animated characters to novice users. The technique enables users to efficiently prototype behaviors of a character, with the potential for further refinements. With several testees, our gesture-based authoring manner has been verified to be beneficial to the addressed problem, and, grounded on the HCI (human-computer interaction) literature, the authoring process is direct, easy, and enjoyable.

This paper presents a new technique for rendering bidirectional texture functions (BTFs) at different levels of detail (LODs). Our method first decomposes each BTF image into multiple subbands with a Laplacian pyramid. Each vector of Laplacian coefficients of a texel at the same level is regarded as a Laplacian bidirectional reflectance distribution function (BRDF). These vectors are then further compressed by applying principal components analysis (PCA). At the rendering stage, the LOD parameter for each pixel is calculated according to the distance from the viewpoint to the surface. Our rendering algorithm uses this parameter to determine how many levels of BTF Laplacian pyramid are required for rendering. Under the same sampling resolution, a BTF gradually transits to a BRDF as the camera moves away from the surface. Our method precomputes this transition and uses it for multiresolution BTF rendering. Our Laplacian pyramid representation allows real-time anti-aliased rendering of BTFs using graphics hardware. In addition to provide visually satisfactory multiresolution rendering for BTFs, our method has a comparable compression rate to the available single-resolution BTF compression techniques.

Summary: A web-based three-dimensional (3D) protein retrieval system is available for protein structure data including all PDB and FSSP dataset. In this system, we use a visual-based matching method to compare the protein structure from multiple viewpoints. It takes less than three seconds for each query with 90% accuracy on an average. Availability: The web-based query interface and downloadable files can be accessed via http://3d.csie.ntu.edu.tw/ProteinRetrieval/ Contact: [email protected] Supplementary information: Further details of the proposed method are available at http://graphics.csie.ntu.edu.tw/~jsyeh/3Dprotein/

In this paper, we present a new method for surface extraction from volume data which preserves sharp features, maintains consistent topology and generates surface adaptively without crack patching. Our approach is based on the marching cubes algorithm, a popular method to convert volumetric data to polygonal meshes. The original marching cubes algorithm suffers from problems of topological inconsistency, cracks in adaptive resolution and inability to preserve sharp features. Most of marching cubes variants only focus on one or some of these problems. Although these techniques could be combined to solve these problems altogether, such a combination might not be straightforward. Moreover, some feature-preserving variants introduce an additional problem, inter-cell dependency. Our method provides a relatively simple and easy-to-implement solution to all these problems by converting 3D marching cubes into 2D cubical marching squares, resolving topology ambiguity with sharp features and eliminating inter-cell dependency by sampling face sharp features. We compare our algorithm with other marching cubes variants and demonstrate its effectiveness on various applications.

When creating a cel animation, the animators often use 3D character models to add some effects on the character or to generate intermediate images between the keyframes. However, it is a troublesome and time-consuming task to create a 3D model. In this paper, we present an easy-to-use approach for creating a set of consistent 3D character models from the user-specified strokes on a 2D image sequence. The created consistent 3D models can be used in cel animation editing systems for adding shadowing effects, textures, etc. Moreover, since the vertices of the consistent 3D models have one-to-one correspondence among the frames, by using 3D morphing techniques, this approach can also be used to generate intermediate images between the key frames.

Conceptual Farm is a virtual reality platform for generating and observing the behaviors of different autonomous characters. By providing (1) descriptions for characters' behaviors and (2) 3D animations and sound, life-like characters in a realistic habitat can be created, modified, and interact with both users and other characters in real time. The flexible, manageable and scalable nature of Conceptual Farm leads to its desirability in zoological research, general education, game and film production, and even decorative arts.

Digital sculpting is a new trend for creating 3D models, but its application in the manipulation of volumetric data raises several issues that need to be addressed. With the extended marching cubes algorithm (EMC), sharp features of 3D models are well preserved. Additionally, the dynamic nature of modifying models in real time needs to be dealt with in sculpting applications: since the sampling of sharp features is implicit, direct modification on cell data will cause problems. A feature refinement strategy is proposed to preserve the dynamically modified model correct, and efficiently. Overall, the proposed methods provide an adaptive resolution and feature-preserved sculpting system that handles dynamic behavior in real-time performance.

In this paper, we propose an appearance representation for general complex materials which can be applied in real-time rendering framework. By combining a single parametric shading function (such as the Phong model) and the proposed spatial-varying residual function (SRF), this representation can recover the appearance of complex materials with little loss of visual fidelity. The difference between the real data and the parametric shading is directly fitted by a specific function for easy reconstruction. It is simple, flexible and easy to be implemented on programmable graphics hardware. Experiments show that the mean square error (MSE) between the reconstructed appearance and real photographs is less than 5%.

We present an efficient method of mesh simplification for geometric 3D models. The transmission of 3D models on the Internet is an important task. The data size of a 3D model is usually large to enable more detail to be represented. Hence, it is necessary to represent the 3D model while keeping the data size small and preserving its features, even if the meshes that constitute the model are unstructured. Although there are many methods for simplifying the meshes, most of them are time-consuming. Our approach is to obtain an adequate simplified model in a short amount of time. Therefore, the model provider can check the simplified result interactively before uploading to the server. After transmitting the simplified model, if the user at the client needs to get more details, by transmitting some necessary information, the progressively increasing model detail and the original model without losses could be reconstructed.

In this paper, we present a physics motivated modeling method for volcanic clouds as a two fluids model. Some previous methods model smoke or clouds as one fluid, but the volcanic clouds can not be treated as one fluid. The volcanic clouds consist of the pyroclasts, the volcanic gas and the entrained air. Since the pyroclasts and the volcanic gas can be treated as one fluid, called magma, the volcanic clouds are regarded as two fluids, the magma and the entrained air. The modeling in the 3D analysis space can be simplified to enhance the performance. Since our approach is physics motivated, it can be used to generate physically reasonable and realistic images of volcanic clouds from the volcanic initial eruption to the equilibrium situation.

Abstract This paper presents a practical approach to the animation of hair at an interactive frame rate. In our approach,we model the hair as a set of particles that serve as sampling points for the volume of the hair, which covers thewhole region where hair is present. The dynamics of the hair, including hair‐hair interactions, is simulated usingthe interacting particles. The novelty of this approach is that, as opposed to the traditional way of modeling hair,we release the particles from tight structures that are usually used to represent hair strands or clusters. Therefore,by making the connections between the particles loose while maintaining their overall stiffness, the hair can bedynamically split and merged during lateral motion without losing its lengthwise coherence. Categories and Subject Descriptions (according to ACM CCS): I.3.7 [Computer Graphics]: Three‐DimensionalGraphics and Realism, I.3.3 [Computer Graphics]: Picture/Image Generation

Developing intuitive and efficient methods for shape editing is one of the most important areas in computer graphics, and free-form deformation (FFD), which is one of such methods, allows the user to deform a model easily by moving a set of control points, collectively called the lattice. Although the FFD method can be used for both global and local deformations, the user must define a suitable lattice manually or use a simple shaped lattice such as a parallelepiped. Therefore, we propose a new FFD method that automatically generates the lattices with which both types of deformations can be achieved. Our method refines a bounding box of the model and generates a set of finer lattices, which hierarchically approximate the shape of the model. Through adjusting the control points of the generated lattices, both global and local deformations of the model can be achieved easily. Moreover, the method allows hierarchical deformation of the model by combining different levels of lattice.

Solid texturing is a well-known computer graphics technology, which is problematic because it consumes too much time if every pixel is calculated on the fly or has a very high memory requirement if all of the pixels are stored at the beginning. Although some methods have been proposed, almost all of them need the support of specific hardware accelerators. Hence, these methods could not be applied to all kinds of machine, especially low-cost ones available over the Internet. Therefore, we present a new method for procedural solid texturing. Our approach could almost render an object with solid texturing in real-time using only a software solution. Furthermore, to demonstrate that our approach is widely applicable we choose pure Java for its implementation, since it does not receive any benefit from hardware and could be executed on the Internet directly.