Recent advancements in photogrammetry and 3D LiDAR scanners have led to a noticeable increase in the creation of 3D scans of museum artifacts. This trend has opened up new possibilities for museums to create interactive and immersive experiences using virtual reality (VR). In collaboration with the Florida Natural History Museum, we have developed an interactive VR game that utilizes the extensive 3D digital collection of the Florida Museum. The goal is to enhance the traditional museum experience by immersing visitors in dynamic VR environments that showcase 3D museum collections within context. Specifically, our VR game highlights 3D scans of endangered species of underwater creatures in the ocean. Children can swim alongside these sea creatures while an AI conversational agent provides scientific insights. Our VR game was showcased to the public at the Florida Natural History Museum during a public outreach event, attracting visits from three K-12 school trips. We conducted field observations to evaluate children’s interactions with the VR game and conducted semi-structured interviews with children’s guardians as well as museum staff. The findings from our observations emphasized the importance of shared experiences among visitors, which could be facilitated by projecting the VR gameplay on a large screen to mitigate the isolated nature of the HMD VR experience limitations. Additionally, museum staff emphasized the significance of considering visitor traffic when designing VR experiences in museum settings. The findings also highlighted a preference for seated experiences over standing ones due to safety concerns related to children colliding with others and museum artifacts. This paper provides an overview of our design process and the challenges of implementing HMD VR in museum settings, offering valuable insights for future endeavors aimed at designing public VR educational experiences targeting children.

Real-time 3-D reconstruction of the human body has many applications in anthropometry, telecommunications, gaming, fashion, and other areas of human-computer interaction. In this paper, a novel framework is presented for reconstructing the 3-D model of the human body from a sequence of RGB-D frames. The reconstruction is performed in real time while the human subject moves arbitrarily in front of the camera. The method employs a novel parameterization of cylindrical-type objects using Cartesian tensor and b-spline bases along the radial and longitudinal dimension respectively. The proposed model, dubbed tensor body, is fitted to the input data using a multistep framework that involves segmentation of the different body regions, robust filtering of the data via a dynamic histogram, and energy-based optimization with positive-definite constraints. A Riemannian metric on the space of positive-definite tensor splines is analytically defined and employed in this framework. The efficacy of the presented methods is demonstrated in several real-data experiments using the Microsoft Kinect sensor.

Tensors of various orders can be used for modeling physical quantities such as strain and diffusion as well as curvature and other quantities of geometric origin. Depending on the physical properties of the modeled quantity, the estimated tensors are often required to satisfy the positivity constraint, which can be satisfied only with tensors of even order. Although the space [Formula: see text] of 2m(th)-order symmetric positive semi-definite tensors is known to be a convex cone, enforcing positivity constraint directly on [Formula: see text] is usually not straightforward computationally because there is no known analytic description of [Formula: see text] for m > 1. In this paper, we propose a novel approach for enforcing the positivity constraint on even-order tensors by approximating the cone [Formula: see text] for the cases 0 < m < 3, and presenting an explicit characterization of the approximation Σ(2) (m) ⊂ Ω(2) (m) for m ≥ 1, using the subset [Formula: see text] of semi-definite tensors that can be written as a sum of squares of tensors of order m. Furthermore, we show that this approximation leads to a non-negative linear least-squares (NNLS) optimization problem with the complexity that equals the number of generators in Σ(2) (m). Finally, we experimentally validate the proposed approach and we present an application for computing 2m(th)-order diffusion tensors from Diffusion Weighted Magnetic Resonance Images.

Modeling illumination effects and pose variations of a face is of fundamental importance in the field of facial image analysis. Most of the conventional techniques that simultaneously address both of these problems work with the Lambertian assumption and thus fall short of accurately capturing the complex intensity variation that the facial images exhibit or recovering their 3D shape in the presence of specularities and cast shadows. In this paper, we present a novel Tensor-Spline-based framework for facial image analysis. We show that, using this framework, the facial apparent BRDF field can be accurately estimated while seamlessly accounting for cast shadows and specularities. Further, using local neighborhood information, the same framework can be exploited to recover the 3D shape of the face (to handle pose variation). We quantitatively validate the accuracy of the Tensor Spline model using a more general model based on the mixture of single-lobed spherical functions. We demonstrate the effectiveness of our technique by presenting extensive experimental results for face relighting, 3D shape recovery, and face recognition using the Extended Yale B and CMU PIE benchmark data sets.

Human faces are neither exactly Lambertian nor entirely convex and hence most models in literature which make the Lambertian assumption, fall short when dealing with specularities and cast shadows. In this paper, we present a novel anti-symmetric tensor spline (a spline for tensor-valued functions) based method for the estimation of the Apparent BRDF (ABRDF) field for human faces that seamlessly accounts for specularities and cast shadows. Furthermore, unlike other methods, it does not require any 3D information to build the model and can work with as few as 9 images. In order to validate the accuracy of our anti-symmetric tensor spline model, we present a novel approximation of the ABRDF using a continuous mixture of single-lobed spherical functions. We demonstrate the effectiveness of our anti-symmetric tensor-spline model in comparison to other popular models in the literature, by presenting extensive results for face relighting and face recognition using the Extended Yale B database.

In this paper, we present novel algorithms for statistically robust interpolation and approximation of diffusion tensors-which are symmetric positive definite (SPD) matrices-and use them in developing a significant extension to an existing probabilistic algorithm for scalar field segmentation, in order to segment diffusion tensor magnetic resonance imaging (DT-MRI) datasets. Using the Riemannian metric on the space of SPD matrices, we present a novel and robust higher order (cubic) continuous tensor product of B-splines algorithm to approximate the SPD diffusion tensor fields. The resulting approximations are appropriately dubbed tensor splines. Next, we segment the diffusion tensor field by jointly estimating the label (assigned to each voxel) field, which is modeled by a Gauss Markov measure field (GMMF) and the parameters of each smooth tensor spline model representing the labeled regions. Results of interpolation, approximation, and segmentation are presented for synthetic data and real diffusion tensor fields from an isolated rat hippocampus, along with validation. We also present comparisons of our algorithms with existing methods and show significantly improved results in the presence of noise as well as outliers.

Lattice based neural networks are capable of resolving some difficult non-linear problems and have been successfully employed to solve real-world problems. In this paper a novel model of a lattice neural network (LNN) is presented. This new model generalizes the standard basis lattice neural network (SB-LNN) based on dendritic computing. In particular, we show how each neural dendrite can work on a different orthonormal basis than the other dendrites. We present experimental results that demonstrate superior learning performance of the new Orthonormal Basis Lattice Neural Network (OB-LNN) over SB-LNNs.