Food texture plays a crucial role in the overall sensory experience and the functional properties of food, evolving dynamically from the first bite to the final chew. Traditional methods for modifying food texture, such as adjusting cooking parameters, often compromise other key attributes, including appearance and nutritional value, whereas existing computational approaches largely treat texture as a static, monolithic attribute. This paper presents a computational method that modulates the internal structure of food to design temporal food texture experiences. Our method generates a silicone mold based on the desired food texture experience. This approach decouples textural properties from visual and amount attributes, enabling independent tuning of sensory factors during different oral processing stages. We characterize the relationship between internal structure and temporal texture perception, specifically finding that infill pattern and shell thickness control the bite and chewing phases, respectively. We present an interactive design interface that allows users to create food items tailored to specific temporal food textures.

Accurately simulating and designing the vibro-acoustic performance of 3D printed structures is a challenging task due to the lack of material elastic properties, particularly when designing without specialized measurement instruments. To tackle this issue, we propose a numerical-experimental estimation method based on modal impact sound, which only requires a microphone for measurement. Specifically, we established a modal impact sound simulation pipeline that considers the material anisotropy and estimated the material elastic property by minimizing the residuals between the simulated and recorded modal impact sound features. Our method outperforms the state-of-the-art modal-sound approach in terms of convergence speed and accuracy, and achieves an average modal-frequency residual of 0.66% across two materials and six infills. We also designed and manufactured a xylophone using the estimated material elasticity for further demonstration and verification.

This paper presents a shadowless projection mapping system for interactive applications in which a target surface is frequently occluded from a projector with a user's body. We propose a delay-free optical solution for this critical problem. Specifically, as the primary technical contribution, we apply a large format retrotransmissive plate to project images onto the target surface from wide viewing angles. We also tackle technical issues unique to the proposed shadowless principle. First, the retrotransmissive optics inevitably suffer from stray light, which leads to significant contrast degradation of the projected result. We propose to block the stray light by covering the retrotransmissive plate with a spatial mask. Because the mask reduces not only the stray light but the achievable luminance of the projected result, we develop a computational algorithm that determines the shape of the mask to balance the image quality. Second, we propose a touch sensing technique by leveraging the optically bidirectional property of the retrotransmissive plate to support interaction between the user and the projected contents on the target object. We implement a proof-of-concept prototype and validate the above-mentioned techniques through experiments.

Since ancient times, a hug has been one of the most basic ways to express emotions and has played an important role in building relationships between people. On the other hand, social robots that are designed to provide mental health care to patients have been attracting great attention, and hugging between humans and robots is becoming more and more popular. In this study, we propose a huggable robot that allows intimate interactions between humans and robots. Our robot is based on a tensegrity structure, which is composed of rigid elements connected by springs, and the structure allows the robot to flexibly respond to external force from a hugging interaction, and express various emotions through its movements. In addition, we conducted user experiments and explored an interaction design for the affective movement of the proposed robot. Through the experiments, we confirm that a robot with a tensegrity structure can be used for a hug interaction and it has a large possibility for emotional interaction.

We present interiqr, a method that utilizes the infill parameter in the 3D printing process to embed information inside the food that is difficult to recognize with the human eye. Our key idea is to utilize the air space or secondary materials to generate a specific pattern inside the food without changing the model geometry. As a result, our method exploits the patterns that appear as hidden edible tags to store the data and simultaneously adds them to a 3D printing pipeline. Our contribution also includes the framework that connects the user with a data-embedding interface through the food 3D printing process, and the decoding system allows the user to decode the information inside the 3D printed food through backlight illumination and a simple image processing technique. Finally, we evaluate the usability of our method under different settings and demonstrate our method through the example application scenarios.

Abstract Nature evolves structures like honeycombs at optimized performance with limited material. These efficient structures can be artificially created with the collaboration of structural topology optimization and additive manufacturing. However, the extensive computation cost of topology optimization causes low mesh resolution, long solving time, and rough boundaries that fail to match the requirements for meeting the growing personal fabrication demands and printing capability. Therefore, we propose the neural synthesizing topology optimization that leverages a self‐supervised coordinate‐based network to optimize structures with significantly shorter computation time, where the network encodes the structural material layout as an implicit function of coordinates. Continuous solution space is further generated from optimization tasks under varying boundary conditions or constraints for users' instant inference of novel solutions. We demonstrate the system's efficacy for a broad usage scenario through numerical experiments and 3D printing.

An immersive virtual laboratory (VL) could offer flexibility of time and space, as well as safety, for remote students to conduct laboratory activities through online experiential learning. Recording an instructor's demonstration inside a VL is an approach that allows students to learn directly from a demonstration. However, students have to learn from a recording while controlling the playback, which requires attention spent on additional spatial and temporal cues. This additional cognitive load could lead to errors during the laboratory procedure. To address these challenges, we have identified four design requirements to reduce attention load in VLs; namely, organized learning steps, improved student sense of co-presence, reduction of task-instructor split-attention, and learning independent of interpersonal distance. Based on these requirements, we have designed and implemented spatial-temporal assistive toolsets for laboratories in a virtual environment, namely XR-LIVE, to reduce cognitive load and enhance learning in an asynchronous shared-space demonstration, implemented based on the setup of a standard civil engineering laboratory. We also analyzed students' behavior in the VL demonstration to design guidelines applicable to generic VLs.

Stereoscopic projection mapping (PM) allows a user to see a three-dimensional (3D) computer-generated (CG) object floating over physical surfaces of arbitrary shapes around us using projected imagery. However, the current stereoscopic PM technology only satisfies binocular cues and is not capable of providing correct focus cues, which causes a vergence-accommodation conflict (VAC). Therefore, we propose a multifocal approach to mitigate VAC in stereoscopic PM. Our primary technical contribution is to attach electrically focus-tunable lenses (ETLs) to active shutter glasses to control both vergence and accommodation. Specifically, we apply fast and periodical focal sweeps to the ETLs, which causes the "virtual image" (as an optical term) of a scene observed through the ETLs to move back and forth during each sweep period. A 3D CG object is projected from a synchronized high-speed projector only when the virtual image of the projected imagery is located at a desired distance. This provides an observer with the correct focus cues required. In this study, we solve three technical issues that are unique to stereoscopic PM: (1) The 3D CG object is displayed on non-planar and even moving surfaces; (2) the physical surfaces need to be shown without the focus modulation; (3) the shutter glasses additionally need to be synchronized with the ETLs and the projector. We also develop a novel compensation technique to deal with the "lens breathing" artifact that varies the retinal size of the virtual image through focal length modulation. Further, using a proof-of-concept prototype, we demonstrate that our technique can present the virtual image of a target 3D CG object at the correct depth. Finally, we validate the advantage provided by our technique by comparing it with conventional stereoscopic PM using a user study on a depth-matching task.

Intrinsic projector calibration is essential in projection mapping (PM) applications, especially in dynamic PM. However, due to the shallow depth-of-field (DOF) of a projector, more work is needed to ensure accurate calibration. We aim to estimate the intrinsic parameters of a projector while avoiding the limitation of shallow DOF. As the core of our technique, we present a practical calibration device that requires a minimal working volume directly in front of the projector lens regardless of the projector's focusing distance and aperture size. The device consists of a flat-bed scanner and pinhole-array masks. For calibration, a projector projects a series of structured light patterns in the device. The pinholes directionally decompose the structured light, and only the projected rays that pass through the pinholes hit the scanner plane. For each pinhole, we extract a ray passing through the optical center of the projector. Consequently, we regard the projector as a pinhole projector that projects the extracted rays only, and we calibrate the projector by applying the standard camera calibration technique, which assumes a pinhole camera model. Using a proof-of-concept prototype, we demonstrate that our technique can calibrate projectors with different focusing distances and aperture sizes at the same accuracy as a conventional method. Finally, we confirm that our technique can provide intrinsic parameters accurate enough for a dynamic PM application, even when a projector is placed too far from a projection target for a conventional method to calibrate the projector using a fiducial object of reasonable size.

Personalization of eating such that everyone consumes only what they need allows improving our management of food waste. In this paper, we explore the use of food 3D printing to create perceptual illusions for controlling the level of perceived satiety given a defined amount of calories. We present FoodFab, a system that allows users to control their food intake through modifying a food's internal structure via two 3D printing parameters: infill pattern and infill density. In two experiments with a total of 30 participants, we studied the effect of these parameters on users' chewing time that is known to affect people's feeling of satiety. Our results show that we can indeed modify the chewing time by varying infill pattern and density, and thus control perceived satiety. Based on the results, we propose two computational models and integrate them into a user interface that simplifies the creation of personalized food structures.

From full-color objects to functional capacitive artifacts, 3D printing multi-materials became essential to broaden the application areas of digital fabrication. We present Programmable Filament, a novel technique that enables multi-material printing using a commodity FDM 3D printer, requiring no hardware upgrades. Our technique builds upon an existing printing technique in which multiple filament segments are printed and spliced into a single threaded filament. We propose an end-to-end pipeline for 3D printing an object in multi-materials, with an introduction of the design systems for end-users. Optimized for low-cost, single-nozzle FDM 3D printers, the system is built upon our computational analysis and experiments to enhance its validity over various printers and materials to design and produce a programmable filament. Finally, we discuss application examples and speculate the future with its potential, such as custom filament manufacturing on-demand.

The appearance of fabric motion is suggested to affect the human perception of bending stiffness. This study presents a novel spatial augmented reality, or projection mapping, approach that can visually manipulate the perceived bending stiffness of a fabric. Particularly, we proposed a flow enhancement method that can change the apparent fabric motion by using a simple optical flow analysis technique rather than complex physical simulations for interactive applications. Through a psychophysical experiment, we investigated the relationship between the magnification factor of our flow enhancement and the perceived bending stiffness of a fabric. Furthermore, we constructed a prototype application system that allows users to control the stiffness of a fabric without changing the actual physical fabric. By evaluating the prototype, we confirmed that the proposed technique can manipulate the perceived stiffness of various materials (i.e., cotton, polyester, and mixed cotton and linen) at an average accuracy of 90.3 percent.

While virtual reality (VR) tools provide an immersive learning experience for students, it is difficult for an instructor to observe the students' learning activities in a virtual environment (VE). Thus, it hinders interactions that could occur between the instructor and students, which are usually required in a classroom environment to understand how each student learns. Previous work has added virtual awareness cues that can help a small group of students to collaborate in a VE. However, when the number of students increases, such virtual awareness cues can cause visual clutter and confuse the instructor. We propose ObserVAR, a visualization system that allows the instructor to observe students in a VE at scale. ObserVAR uses augmented reality techniques to visualize each student's gaze in a VE and improves the instructor's awareness of the entire class. The visualizations are then optimized to reduce visual clutter in the scene using a force-directed graph drawing algorithm. In designing ObserVAR, we first investigated visualizations that can provide the instructor with an overall awareness of the VE that can be scaled up as the number of users increases. Second, we optimized the visualization of students by leveraging a graph drawing algorithm to reduce the visual clutter in the class scene. We compared the performance of our prototype with some commercially available user interfaces for VE classrooms. In our study, ObserVAR has demonstrated improvement and flexibility in several application scenarios.

In this paper, we propose a different perspective on the use of support material: rather than printing support structures for overhangs, our idea is to make use of its transient nature, i.e. the fact that it can be dissolved when placed in a solvent, such as water. This enables a range of new use cases, such as quickly dissolving and replacing parts of a prototype during design iteration, printing temporary assembly labels directly on the object that leave no marks when dissolved, and creating time-dependent mechanisms, such as fading in parts of an image in a shadow art piece or releasing relaxing scents from a 3D printed structure sequentially overnight. Since we use regular support material (PVA), our approach works on consumer 3D printers without any modifications. To facilitate the design of objects that leverage dissolvable support, we built a custom 3D editor plugin that includes a simulation showing how support material dissolves over time. In our evaluation, our simulation predicted geometries that are statistically similar to the example shapes within 10% error across all samples.

In this paper, we present a method to create re-programmable multi-color textures that are made from a single material only. The key idea builds on the use of photochromic inks that can switch their appearance from transparent to colored when exposed to light of a certain wavelength. By mixing cyan, magenta, and yellow (CMY) photochromic dyes into a single solution and leveraging the different absorption spectra of each dye, we can control each color channel in the solution separately. Our approach can transform single-material fabrication techniques, such as coating, into high-resolution multi-color processes. We discuss the material mixing procedure, modifications to the light source, and the algorithm to control each color channel. We then show the results from an experiment in which we evaluated the available color space and the resolution of our textures. Finally, we demonstrate our user interface that allows users to transfer virtual textures onto physical objects and show a range of application examples.

Shadowless Projector is projection mapping system in which a shadow (more specifically, umbra) does not suffer the projected result. A typical shadow removal technique used a multiple overlapping projection system. In this paper, we propose a shadow-less projection method with single projector. Inspired by a surgical light system that does not cast shadows on patients' bodies in clinical practice, we apply a special optical system that consists of methodically positioned vertical mirrors. This optical system works as a large aperture lens, it is impossible to block all projected ray by a small object such as a hand. Consequently, only penumbra is caused, which leads to a shadow-less projection.

We present fARFEEL, a remote communication system that provides visuo-haptic feedback allows a local user to feel touching distant objects. The system allows the local and remote users to communicate by using the projected virtual hand (VH) for the agency of his/her own hands. The necessary haptic information is provided to the non-manipulating hand of the local user that does not bother the manipulation of the projected VH. We also introduce the possible visual stimulus that could potentially provide the sense of the body ownership over the projected VH.

As an emerging technology, virtual reality (VR) has been used increasingly as a learning tool to explore “outside the classroom experiences” inside the classroom. While VR provides an immersive experience to the students, it is difficult for the instructor to monitor the students' activities in the VR. Thus, it hinders interactions between the instructor and students. To solve this challenge, we investigated a technique that allows the instructor to observe VR users at scale using Augmented Reality. Augmented Reality techniques are used to visualize the gazes of the VR users in the virtual environment, and improve the instructor's awareness.

Recent research has shown how to change the color of existing objects using photochromic materials. These materials can switch their appearance from transparent to colored when exposed to light of a certain wavelength. The color remains even when the object is removed from the light source. The process is fully reversible allowing users to recolor the object as many times as they want. So far, these systems have been limited to single color changes, i.e. changes from transparent to colored. In this paper, we present ColorMod, a method to extend this approach to multi-color changes (e.g., red-to-yellow). We accomplish this using a multi-color pattern with one color per voxel across the surface of the object. When recoloring the object, our system locally activates only those voxels that have the desired color and turns all other voxels off. We describe ColorMod's hardware/software system and its user interface that comes with a conversion tool for 3D printing as well as a painting tool that matches physical voxels with the desired appearance. We also contribute our own material formula for a 3D-printable photochromic ink.

Interactive surfaces and spaces (ISS) research has been advanced by augmented reality (AR) display technologies. Recent trends of computational displays overcome limitations of existing display technologies by optimizing both hardware and software while considering human perception. We plan to held a workshop on computational AR displays (CARD) to explore emerging ISS research issues by promoting communications and interactions between ISS and CARD communities.

Collaboration between a helper and a worker in a 3D collaborative virtual environment usually requires real-time information sharing, since the worker relies on the timely assistance from the helper. In contrast, collaboration between workers requires them to shift their attention between independent tasks and dependent tasks. In worker-worker collaborations, a real-time updating technique could create excess information, which may be a distraction. In this paper, we compare different information sharing techniques and determine an efficient technique for the collaboration between workers. In our user experiment, participants performed a floor plan design task in a designer and engineer pairing on a desktop VR environment. The results showed that the proposed information sharing technique, in which objects are updated based on local users' actions, is more suitable than real-time updates. In addition, we discuss design implications that can be applied to different collaborative scenarios.

This paper presents the extended work on LazyNav, a head-free, eyes-free and hands-free mid-air ground navigation control model presented at the IEEE 3D User Interfaces (3DUI) 2015, in particular with a new application to the head-mounted display (HMD). Our mid-air interaction metaphor makes use of only a single pair of the remaining tracked body elements to tailor the navigation. Therefore, the user can navigate in the scene while still being able to perform other interactions with her hands and head, e.g., carrying a bag, grasping a cup of coffee, or observing the content by moving her eyes and locally rotating her head. We design several body motions for navigation by considering the use of non-critical body parts and develop assumptions about ground navigation techniques. Through the user studies, we investigate the motions that are easy to discover, easy to control, socially acceptable, accurate and not tiring. Finally, we evaluate the desired ground navigation features with a prototype application in both a large display (LD) and a HMD navigation scenarios. We highlight several recommendations for designing a particular mid-air ground navigation technique for a LD and a HMD.

We present SoftAR, a novel spatial augmented reality (AR) technique based on a pseudo-haptics mechanism that visually manipulates the sense of softness perceived by a user pushing a soft physical object. Considering the limitations of projection-based approaches that change only the surface appearance of a physical object, we propose two projection visual effects, i.e., surface deformation effect (SDE) and body appearance effect (BAE), on the basis of the observations of humans pushing physical objects. The SDE visualizes a two-dimensional deformation of the object surface with a controlled softness parameter, and BAE changes the color of the pushing hand. Through psychophysical experiments, we confirm that the SDE can manipulate softness perception such that the participant perceives significantly greater softness than the actual softness. Furthermore, fBAE, in which BAE is applied only for the finger area, significantly enhances manipulation of the perception of softness. We create a computational model that estimates perceived softness when SDE+fBAE is applied. We construct a prototype SoftAR system in which two application frameworks are implemented. The softness adjustment allows a user to adjust the softness parameter of a physical object, and the softness transfer allows the user to replace the softness with that of another object.

Choosing the appropriate soft/hard material is important for designing a product such as sofa or bed, but typically limited by the number of physical materials that the designer owns. Pseudo-haptic feedback is an alternative way that enables designer to roughly simulate material properties (e.g., softness, hardness) by only generating the visual illusion. However, the current technique is limited within video see-through augmented reality, in which the user interact in a real space while looking at a virtual space. This paper explores the possibility to realize pseudo-haptic feedback for touching objects in spatial augmented reality. We investigate and compare effects of visually superimposing a projection graphics onto the surface of a touched object and the fingernail/finger for changing the surface tactile perception. The potential of our method is discussed through a preliminary user study.