Genomic Prediction (GP) methods predict the breeding value of unphenotyped individuals in order to select parental candidates breeding populations of livestock and crop plants. Among models for GP, classical linear models have remained consistently popular, while more complex nonlinear methods such as deep neural networks have shown comparable accuracy at best. In this work, we propose the Genomic Prior-Data Fitted Network (GPFN) as a new paradigm for GP. GPFNs perform amortized Bayesian inference by simulating hundreds of thousands or millions of plant or animal populations. This allows GPFNs to be deployed without requiring any training or tuning, providing predictions in a single inference pass. On three populations of plants across two different crop species, GPFNs perform significantly better than the linear baseline on 13 out of 16 traits. On a challenging between-families structured prediction task on a third crop species, the GPFN matches the performance of the linear baseline while outperforming it in one location. GPFNs represent a completely new direction for the field of genomic prediction, and have the potential to unlock levels of selection accuracy not possible with existing methods, especially in diverse populations.

Comparison is a well-studied task in visual analytics, but there is still little support for comparison of temporal streams such as video. There are a wide range of tasks that involve video comparison, but there are very few systems or techniques to support this kind of analysis. To help address this problem, we have developed new interaction techniques that explicitly support video comparison. We provide techniques for equalizing the reference frame of videos to be compared, juxtaposition techniques for enhancing side-by-side and small-multiples comparisons, superposition techniques for comparing overlaid videos, explicit-encoding techniques that visualize differences between extracted points, and temporal-to-linear techniques that translate between a temporal sequence of frames and a 1D timeline. We built a demonstration system with five different datasets, and evaluated our interaction techniques in two ways: an analysis of steps to show their efficiency, and a preliminary user study to explore learnability, utility, and usability.

Simulating large scale expansion of thin structures, such as in growing leaves, is challenging. Solid-shells have a number of potential advantages over conventional thin-shell methods, but have thus far only been investigated for small plastic deformation cases. In response, we present a new general-purpose FEM growth framework for handling a wide range of challenging growth scenarios using the solid-shell element. Solid-shells are a middle-ground between traditional volume and thin-shell elements where volumetric characteristics are retained while being treatable as a 2D manifold much like thin-shells. These elements are adaptable to accommodate the many techniques that are required for simulating large and intricate plastic deformations, including morphogen diffusion, plastic embedding, strain-aware adaptive remeshing, and collision handling. We demonstrate the capabilities of growing solid-shells in reproducing buckling, rippling, curling, and collision deformations, relevant towards animating growing leaves, flowers, and other thin structures. Solid-shells are compared side-by-side with thin-shells to examine their bending behavior and runtime performance. The experiments demonstrate that solid-shells are a viable alternative to thin-shells for simulating large and intricate growth deformations.

Distribution shifts—where the training distribution differs from the test distribution—can substantially degrade the accuracy of machine learning (ML) systems deployed in the wild. Despite their ubiquity in the real-world deployments, these distribution shifts are under-represented in the datasets widely used in the ML community today. To address this gap, we present WILDS, a curated benchmark of 10 datasets reflecting a diverse range of distribution shifts that naturally arise in real-world applications, such as shifts across hospitals for tumor identification; across camera traps for wildlife monitoring; and across time and location in satellite imaging and poverty mapping. On each dataset, we show that standard training yields substantially lower out-of-distribution than in-distribution performance. This gap remains even with models trained by existing methods for tackling distribution shifts, underscoring the need for new methods for training models that are more robust to the types of distribution shifts that arise in practice. To facilitate method development, we provide an open-source package that automates dataset loading, contains default model architectures and hyperparameters, and standardizes evaluations. The full paper, code, and leaderboards are available at https://wilds.stanford.edu.

Fish Tank Virtual Reality (FTVR) displays provide compelling 3D experiences by rendering view-dependent imagery on a 2D screen. While users perceive a 3D object in space, they are actually looking at pixels on a 2D screen, thus, a perceptual duality exists between the object's pixels and the 3D percept potentially interfering with the experience. To investigate, we conducted an experiment to see whether the on-screen size of the 2D imagery affects the perceived object size in 3D space with different viewing conditions, including stereopsis. We found that the size of on-screen imagery significantly influenced object size perception, causing 83.3% under/overestimation of perceived size when viewing without stereopsis and reducing to 64.7% with stereopsis. Contrary to reality, objects look smaller when the viewer gets closer. Understanding the perceptual duality helps us to provide accurate perception of real-world objects depicted in the virtual environment and pave the way for 3D applications.

Spherical fish tank virtual reality (FTVR) displays attempt to create a virtual "crystal ball" experience using head-tracked rendering. Almost all of these systems have omitted stereo cues, making them easy to build, but it is not clear how much this omission degrades the 3D experience. In this study, we evaluate performance and subjective effects of stereo on 3D perception and interaction tasks with a spherical FTVR display. To control for calibration error and tracking latency, we perform the evaluation on a simulated spherical display in VR. The results of our study provide a clear recommendation for the design and use of spherical FTVR displays: while omitting stereo may not be readily apparent for users, their performance will be significantly degraded (20% - 91% increase in median task time). Therefore, including stereo viewing in spherical displays is critical for use in FTVR.

Time-lapse videos are often navigated by scrubbing with a slider. When networks are slow or images are large, however, even thumbnail versions load so slowly that scrubbing is limited to the start of the video. We developed a frame-loading technique called spread-loading that enables scrubbing regardless of delivery rate. Spread-loading orders frame delivery to maximize coverage of the entire sequence; this provides a temporal overview of the entire video that can be fully navigated at any time during delivery. The overview initially has a coarse temporal resolution, becoming finer-grained with each new frame. We compared spread-loading with traditional linear loading in a study where participants were asked to find specific episodes in a long time-lapse sequence, using three views with increasing levels of detail. Results show that participants found target episodes significantly and substantially faster with spread-loading, regardless of whether they could click to change the load point. Users rated spread-loading as requiring less effort, and strongly preferred the new technique.

Eye-contact is a key aspect of non-verbal human communication in everyday tasks [1]. It provides important social and emotional information that can increase the effectiveness of human communication [8]. In a conversation, eye-contact, or the lack thereof, is constantly evaluated by human brains. Conversation partners derive subjective judgment of others' credibility, focus and confidence [4] from it. Seeking another person's eye-contact is a signal for them that focus is put on that person and the main receptive senses are prepared to receive input from the other person. Likewise breaking eye-contact usually indicates distraction, loss of confidence, loss of interest or shifting focus to a different target.

Fish Tank Virtual Reality (FTVR) displays create a compelling 3D effect with the motion parallax cue using the head-coupled perspective. While the head-coupled viewpoint control provides natural visuomotor coupling, the motion parallax cue has been found to be underutilized with minimal head motion detected when manual input becomes available to users. We investigate whether users can effectively use head-coupling in conjunction with manual input in a mental rotation task involving inspection and comparison of a pair of 3D cubes. We found that participants managed to incorporate the head-coupled viewpoint control with the manual touch input in the task. They used the touch input as the primary input and the head as the secondary input with the input ratio of 4.2:1. The combined input approach appears to be sequential with only 8.63% duration when the head and manual input are co-activated. The result of this study provides insights for designing head-coupled interactions in many 3D interactive applications.

Fish Tank Virtual Reality (FTVR) displays create a compelling 3D spatial effect by rendering to the perspective of the viewer with head-tracking. Combining FTVR with a spherical display enhances the 3D experience with unique properties of the spherical screen such as the enclosing shape, consistent curved surface, and borderless views from all angles around the display. The ability to generate a strong 3D effect on a spherical display with head-tracked rendering is promising for increasing user's performance in 3D tasks. An unanswered question is whether these natural affordances of spherical FTVR displays can improve spatial perception in comparison to traditional flat FTVR displays. To investigate this question, we conducted an experiment to see whether users can perceive the depth and size of virtual objects better on a spherical FTVR display compared to a flat FTVR display on two tasks. Using the spherical display, we found significantly that users had 1cm depth accuracy compared to 6.5cm accuracy using the flat display on a depth-ranking task. Likewise, their performance on a size-matching task was also significantly better with the size error of 2.3mm on the spherical display compared to 3.1mm on the flat display. Furthermore, the perception of size-constancy is stronger on the spherical display than the flat display. This study indicates that the natural affordances provided by the spherical form factor improve depth and size perception in 3D compared to a flat display. We believe that spherical FTVR displays have potential as a 3D virtual environment to provide better task performance for various 3D applications such as 3D designs, scientific visualizations, and virtual surgery.

In this work we describe the design, implementation and initial evaluation of a spherical Fish Tank Virtual Reality (FTVR) display for realistic eye-contact. We identify display shape, size, and depth cues as well as model fidelity as important considerations and challenges for setting up realistic eye-contact and package it into a reproducible framework. Based on the design, we implemented and evaluated the system to assess the effectiveness of the eye-contact. In our initial evaluation participants were able to identify eye-contact with an accuracy of 89.6%. Moreover eye-contact with a virtual character triggered changes in participant's social behavior that are in line with real world eye-contact scenarios. Taken together, these results provide practical guidelines for building displays for realistic eye-contact and can be applied to applications such as teleconferencing and VR treatment in psychology.

A mixed reality experience with a physical display, that situates 3D virtual content within the real world, has the potential to help people work and play with 3D information. However, almost all of such "fish tank virtual reality" (FTVR) systems have been isolated to a single-person experience, making them unsuitable for collaborative tasks. In this paper, we present a display system that allows two people to have unobstructed 3D perspective views into a spherical display while still being able to see and talk to one another. We evaluated the system through qualitative observation at a four-day exhibition and found it was effective for providing a convincing, shared 3D experience.

In this paper, we investigate estimating emergence and biomass traits from color images and elevation maps of wheat field plots. We employ a state-of-the-art deconvolutional network for segmentation and convolutional architectures, with residual and Inception-like layers, to estimate traits via high dimensional nonlinear regression. Evaluation was performed on two different species of wheat, grown in field plots for an experimental plant breeding study. Our framework achieves satisfactory performance with mean and standard deviation of absolute difference of 1.05 and 1.40 counts for emergence and 1.45 and 2.05 for biomass estimation. Our results for counting wheat plants from field images are better than the accuracy reported for the similar, but arguably less difficult, task of counting leaves from indoor images of rosette plants. Our results for biomass estimation, even with a very small dataset, improve upon all previously proposed approaches in the literature.

Head-tracked 3D displays can provide a compelling 3D effect, but even small inaccuracies in the calibration of the participant's viewpoint to the display can disrupt the 3D illusion. We propose a novel interactive procedure for a participant to easily and accurately calibrate a head-tracked display by visually aligning patterns across a multi-screen display. Head-tracker measurements are then calibrated to these known viewpoints. We conducted a user study to evaluate the effectiveness of different visual patterns and different display shapes. We found that the easiest to align shape was the spherical display and the best calibration pattern was the combination of circles and lines. We performed a quantitative camera-based calibration of a cubic display and found visual calibration outperformed manual tuning and generated viewpoint calibrations accurate to within a degree. Our work removes the usual, burdensome step of manual calibration when using head-tracked displays and paves the way for wider adoption of this inexpensive and effective 3D display technology.

We describe an auto-calibrated 3D perspective-corrected spherical display that uses multiple rear projected pico-projectors. The display system is auto-calibrated via 3D reconstruction of each projected pixel on the display using a single inexpensive camera. With the automatic calibration, the multiple-projector system supports a seamless blended imagery on the spherical screen. Furthermore, we incorporate head tracking with the display to present 3D content with motion parallax by rendering perspective-corrected images based on the viewpoint. To show the effectiveness of this design, we implemented a view-dependent application that allows walk-around visualization from all angles for a single head-tracked user. We also implemented a view-independent application that supports a wall-papered rendering for multi-user viewing. Thus, both view-dependent 3D VR content and spherical 2D content, such as a globe, can be easily experienced with this display.

International audience

Real-time games such as first-person shooters (FPS) are sensitive to even small amounts of lag. The effects of net-work latency have been studied, but less is known about local latency, the lag caused by input devices and displays. While local latency is important to gamers, we do not know how it affects aiming performance and whether we can reduce its negative effects. To explore these issues, we tested local latency in a variety of real-world gaming scenarios and carried out a controlled study focusing on targeting and tracking activities in an FPS game with varying degrees of local latency. In addition, we tested the ability of a lag compensation technique (based on aim assistance) to mitigate the negative effects. Our study found local latencies in the real-world range from 23 to 243 ms which cause significant and substantial degradation in performance (even for latencies as low as 41 ms). The study also showed that our compensation technique worked extremely well, reducing the problems caused by lag in the case of targeting, and removing the problem altogether in the case of tracking. Our work shows that local latency is a real and substantial problem -- but games can mitigate the problem with appropriate compensation methods.

We introduce a biomechanical model of oropharyngeal structures that adds the soft-palate, pharynx, and larynx to our previous models of jaw, skull, hyoid, tongue, and face in a unified model. The model includes a comprehensive description of the upper airway musculature, using point-to-point muscles that may either be embedded within the deformable structures or operate exter- nally. The airway is described by an air-tight mesh that fits and deforms with the surrounding articulators, which enables dynamic coupling to our articulatory speech synthesizer. We demonstrate that the biomechanics, in conjunction with the skinning, supports a range from physically realistic to simplified vocal tract geometries to investigate different approaches to aeroacoustic modeling of vocal tract. Furthermore, our model supports inverse modeling to support investigation of plausible muscle activation patterns to generate speech.

In this poster we present Portable-Spheree, an interactive spherical rear-projected 3D-content-display that provides perspective-corrected views according to the user's head position, to provide parallax, shading and occlusion depth cues. Portable-Spheree is an evolution of the Spheree and it is developed in a smaller form factor, using more projectors and a dark-translucent screen with increased contrast. We present some preliminary results of this new configuration as well as applications with spatial interaction that might benefit from this new form factor.

Object handoff - that is, passing an object or tool to another person - is an extremely common activity in collaborative tabletop work. On digital tables, object handoff is typically accomplished by sliding them on the table surface - but surface-only interactions can be slow and error-prone, particularly when there are multiple people carrying out multiple handoffs. An alternative approach is to use the space above the table for object handoff; this provides more room to move, but requires above-surface tracking. We have developed two above-the-surface handoff techniques that use simple and inexpensive tracking: a force-field technique that uses a depth camera to determine hand proximity, and an electromagnetic-field technique called ElectroTouch that provides positive indication when people touch hands over the table. We compared the new techniques to three kinds of surface-only handoff (sliding, flicking, and surface-only Force-Fields). The study showed that the above-surface techniques significantly improved both speed and accuracy, and that ElectroTouch was the best technique overall. This work provides designers with practical new techniques for substantially increasing performance and interaction richness on digital tables.

In this paper, we describe the design of a personal cubic display that offers novel interaction techniques for static and dynamic 3D content. We extended one-screen Fish Tank VR by arranging five small LCD panels into a box shape that is light and compact enough to be handheld. The display uses head-coupled perspective rendering and a real-time physics simulation engine to establish an interaction metaphor of having real objects inside a physical box that a user can hold and manipulate. We evaluated our prototype as a visualization tool and as an input device by comparing it with a conventional LCD display and mouse for a 3D tree-tracing task. We found that bimanual interaction with pCubee and a mouse offered the best performance and was most preferred by users. pCubee has potential in 3D visualization and interactive applications such as games, storytelling and education, as well as viewing 3D maps, medical and architectural data.

We describe a demonstration of four novel interaction techniques for a cubic head-coupled 3D display. The interactions illustrated include: viewing a static scene, navigating through a large landscape, playing with colliding objects inside a box, and stylus-based manipulation of objects. Users experience new interaction techniques for 3D scene manipulation in a cubic display.