This paper introduces Shape2Vibe, a co-design tool for collaboratively designing vibrotactile feedback with people with deafblindness. We outline how a first version of Shape2Vibe was developed and tested. We explain to what extent the tool supports communication during the design process and how it can be used to design meaningful vibrotactile feedback for any given situation. The results from a co-design evaluation study indicate that Shape2Vibe sufficiently supports communication in a co-design session to enable co-design between a designer and people with deafblindness.

Flavobacteria, which can be found in marine environments, are able to grow in highly organized colonies producing vivid iridescent colorations. While much is known about the biology of these organisms, their design potential as responsive media in user interfaces has not been explored. Our paper aims at bridging this gap by providing insights into the type, degree, and duration of change in Flavobacteria’s expression, i.e., their living aesthetics. We present a tool to capture and characterize these changes concerning form, texture and iridescent color. To support the long-term study of their living aesthetics, we designed Flavorium. This bio-digital artifact provides the necessary habitat conditions for Flavobacteria to thrive for a month. Granting insights into the responsive behavior of this organism, this work presents a design space, vocabulary, and application concepts to inspire HCI and design scholars to investigate the complex temporal qualities of living media for future user interfaces.

This paper aims to provide first insights into flash characteristics of bioluminescent microalgae as a potential media for future living light interfaces. A growing number of HCI and interaction design researchers show interest in living material interfaces, which incorporate living organisms for novel responsive behaviour and interaction possibilities in digital and biological hybrids. While much is known about the science of these organisms, their ‘living aesthetics’, i.e., how humans experience the unique temporal changes in a living media, have hardly been explored. To bridge this gap in designing living light interfaces, this paper presents a study of bioluminescent flash characterisation. A DIY shaking device was designed to interact with the liquid living media, providing a range of stimuli including orbital rotation, pulsation and vibration. The living light aesthetics is presented with rich visuals illustrating the intensity variations over time, textural qualities and spatial distribution.

A common strategy for improving model robustness is through data augmentations. Data augmentations encourage models to learn desired invariances, such as invariance to horizontal flipping or small changes in color. Recent work has shown that arbitrary style transfer can be used as a form of data augmentation to encourage invariance to textures by creating painting-like images from photographs. However, a stylized photograph is not quite the same as an artist-created painting. Artists depict perceptually meaningful cues in paintings so that humans can recognize salient components in scenes, an emphasis which is not enforced in style transfer. Therefore, we study how style transfer and paintings differ in their impact on model robustness. First, we investigate the role of paintings as style images for stylization-based data augmentation. We find that style transfer functions well even without paintings as style images. Second, we show that learning from paintings as a form of perceptual data augmentation can improve model robustness. Finally, we investigate the invariances learned from stylization and from paintings, and show that models learn different invariances from these differing forms of data. Our results provide insights into how stylization improves model robustness, and provide evidence that artist-created paintings can be a valuable source of data for model robustness. Code and data are available at: https://github.com/hubertsgithub/style_painting_robustness

In computer graphics, illuminating a scene is a complex task, typically consisting of cycles of adjusting and rendering the scene to see the effects. We propose a technique for visualization of light as a tensor field via extracting its properties (i.e., intensity, direction, diffuseness) from (virtual) radiance measurements and showing these properties as a grid of shapes over a volume of a scene. Presented in the viewport, our visualizations give an understanding of the illumination conditions in the measured volume for both the local values and the global variations of light properties. Additionally, they allow quick inferences of the resulting visual appearance of (objects in) scenes without the need to render them. In our evaluation, observers performed at least as well using visualizations as using renderings when they were comparing illumination between parts of a scene and inferring the final appearance of objects in the measured volume. Therefore, the proposed visualizations are expected to help lighting artists by providing perceptually relevant information about the structure of the light field and flow in a scene.

Product designers extensively use sketches to create and communicate 3D shapes and thus form an ideal audience for sketch-based modeling, non-photorealistic rendering and sketch filtering. However, sketching requires significant expertise and time, making design sketches a scarce resource for the research community. We introduce OpenSketch , a dataset of product design sketches aimed at offering a rich source of information for a variety of computer-aided design tasks. OpenSketch contains more than 400 sketches representing 12 man-made objects drawn by 7 to 15 product designers of varying expertise. We provided participants with front, side and top views of these objects, and instructed them to draw from two novel perspective viewpoints. This drawing task forces designers to construct the shape from their mental vision rather than directly copy what they see. They achieve this task by employing a variety of sketching techniques and methods not observed in prior datasets. Together with industrial design teachers, we distilled a taxonomy of line types and used it to label each stroke of the 214 sketches drawn from one of the two viewpoints. While some of these lines have long been known in computer graphics, others remain to be reproduced algorithmically or exploited for shape inference. In addition, we also asked participants to produce clean presentation drawings from each of their sketches, resulting in aligned pairs of drawings of different styles. Finally, we registered each sketch to its reference 3D model by annotating sparse correspondences. We provide an analysis of our annotated sketches, which reveals systematic drawing strategies over time and shapes, as well as a positive correlation between presence of construction lines and accuracy. Our sketches, in combination with provided annotations, form challenging benchmarks for existing algorithms as well as a great source of inspiration for future developments. We illustrate the versatility of our data by using it to test a 3D reconstruction deep network trained on synthetic drawings, as well as to train a filtering network to convert concept sketches into presentation drawings. We distribute our dataset under the Creative Commons CC0 license: https://ns.inria.fr/d3/OpenSketch.

In this research, a novel experimental setup was built to investigate human beings' sensitivity for lighting properties of a real scene using a gauge object or "probe". Light is a kind of medium that makes objects visible without being visible itself in empty space. The light field determines the appearance of objects and materials in it, and the appearance of objects is the main cue to the light field. Cuttle [2003] suggested to use probes (i.e., gauge objects) to reveal lighting characteristics: the sharpness of a light beam can be estimated from the shadow pattern of a peg on a disk and from the highlight pattern on a black shiny sphere, while the flow of light can be revealed from the shading pattern on a matte white sphere. In a workshop led by Madsen and Donn, white spheres were used as a light flow meter to visually assess spatial and form-giving characteristics of daylight [2006]. By putting white spheres into empty spaces of rendered artificial architecture, the judgments of the light flow improved. However, the shading pattern varied with different perspective views, which made the digital light-flow-meter perceptually ambiguous. Pont and Koenderink [2007] implemented the white sphere as their gauge object for estimating human beings' sensitivity to direction, diffuseness and intensity of illumination and found that human observers confuse illumination direction with illumination diffuseness (diffuseness-direction ambiguity). This explains the problem Madsen encountered and indicates that to implement the light-flow-meter as a quick and simple tool to visually assess the properties of light, alternative probes must be developed. Theoretically, the illuminance flow or texture contrast gradients over a rough sphere gives additional cues about its illumination [2004]. In this study we will address whether observers use such cues so that we can improve the probe by adding roughness. In addition, we will test how human beings perceive the global properties of the light field in a real scene instead of on photographs on a computer screen.

We investigate the estimation of illuminance flow using Histograms of Oriented Gradient features (HOGs). In a regression setting, we found for both ridge regression and support vector machines, that the optimal solution shows close resemblance to the gradient based structure tensor (also known as the second moment matrix). Theoretical results are presented showing in detail how the structure tensor and the HOGs are connected. This relation will benefit computer vision tasks such as affine invariant texture/object matching using HOGs. Several properties of HOGs are presented, among others, how many bins are required for a directionality measure, and how to estimate HOGs through spatial averaging that requires no binning.

Although there has recently been a large increase in commercial 3D applications, relatively little is known about the quantitative perceptual improvement from binocular disparity. In this study we developed a method to measure the perceived relative depth structure of natural scenes. Observers were instructed to adjust the direction of a virtual pointer from one object to another. The pointing data was used to reconstruct the relative logarithmic depths of the objects in pictorial space. The results showed that the relative depth structure is more similar between observers for stereo images than for mono images in two out of three scenes. A similar result was found for the depth range: for the same two scenes the stereo images were perceived as having more depth than the monocular images. In addition, our method allowed us to determine the subjective center of projection. We found that the pointing settings fitted the reconstructed depth best for substantially wider fields of view than the veridical center of projection for both mono and stereo images. The results indicate that the improvement from binocular disparity depends on the scene content: scenes with sufficient monocular information may not profit much from binocular disparity.