course Share on Approaches and Challenges in XR and AI for Health Applications Authors: Joaquim Jorge INESC-ID & IST, Universidade de Lisboa INESC-ID & IST, Universidade de LisboaView Profile , João Pereira INESC-ID & IST, Universidade de Lisboa INESC-ID & IST, Universidade de LisboaView Profile , Anderson Maciel INESC-ID & IST, Universidade de Lisboa INESC-ID & IST, Universidade de LisboaView Profile , Catarina Moreira Technical University Sydney Technical University SydneyView Profile , Regis Kopper UNC Greensboro UNC GreensboroView Profile Authors Info & Claims SA '23: SIGGRAPH Asia 2023 CoursesDecember 2023Article No.: 2Pages 1–147https://doi.org/10.1145/3610538.3614640Published:06 December 2023Publication History 0citation52DownloadsMetricsTotal Citations0Total Downloads52Last 12 Months52Last 6 weeks15 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Monitoring and analyzing physical activity is becoming an important task in both clinical and non-clinical settings. To accomplish this desideratum, stick figures are often used as abstractions of human poses and movements by representing body segments as straight lines (sticks). Despite their straightforwardness, this minimalist representation is incomplete as it lacks the segments’ longitudinal rotations, and therefore, is insufficient for applications requiring full 3D kinematic data. We introduce STONES, an advanced machine learning approach for estimating longitudinal body segment rotations of based on stick figures defined from a minimal set of body points. Our approach relies on a recurrent deep neural network, which takes 3D joint positions from a minimalist stick figure representation, such as those acquired by conventional depth camera sensors, and completes it with accurate longitudinal segment rotations. We validated our approach via a test scenario based on exergaming activities (e.g., lunges, squats, and kicks), which are becoming an emerging trend in several healthcare sectors, and our estimations show a fit above 98% and mean errors of approximately 1 ∘. Our deep learning approach effectively surpasses other machine learning-based strategies and closely matches the accuracy of state-of-the-art motion capture systems while running at real-time speeds.

Head-Mounted Displays are useful to place users in virtual reality (VR). They do this by totally occluding the physical world, including users' bodies. This can make self-awareness problematic. Indeed, researchers have shown that users' feeling of presence and spatial awareness are highly influenced by their virtual representations, and that self-embodied representations (avatars) of their anatomy can make the experience more engaging. On the other hand, recent user studies show a penchant towards a third-person view of one's own body to seemingly improve spatial awareness. However, due to its unnaturality, we argue that a third-person perspective is not as effective or convenient as a first-person view for task execution in VR. In this paper, we investigate, through a user evaluation, how these perspectives affect task performance and embodiment, focusing on navigation tasks, namely walking while avoiding obstacles. For each perspective, we also compare three different levels of realism for users' representation, specifically a stylized abstract avatar, a mesh-based generic human, and a real-time point-cloud rendering of the users' own body. Our results show that only when a third-person perspective is coupled with a realistic representation, a similar sense of embodiment and spatial awareness is felt. In all other cases, a first-person perspective is still better suited for navigation tasks, regardless of representation.

Current state-of-the-art point cloud visualization techniques have shortcomings when dealing with sparse and less accurate data or close-up interactions. In this paper, we present a visualization technique called stroke-based splatting, which applies concepts of stroke-based rendering to surface-aligned splatting, allowing for better shape perception at lower resolutions and close-ups. We create a painterly depiction of the data with an impressionistic aesthetic, which is a metaphor the user is culturally trained to recognize, thus attributing higher quality to the visualization. This is achieved by shaping each object-aligned splat as a brush stroke, and orienting it according to globally coherent tangent vectors from the Householder formula, creating a painterly depiction of the scanned cloud. Each splat is sized according to a color-based clustering analysis of the data, ensuring the consistency of brush strokes within neighborhood areas. By controlling brush shape generation parameters and blending factors between neighboring splats, the user is able to simulate different painting styles in real time. We have tested our method with data sets captured by commodity laser scanners as well as publicly available high-resolution point clouds, both having highly interactive frame rates in all cases. In addition, a user study was conducted comparing our approach to state-of-the-art point cloud visualization techniques. Users considered stroke-based splatting a valuable technique as it provides a higher or similar visual quality to current approaches.

Context-aware pervasive applications can improve user experiences by tracking people in their surroundings. Such systems use multiple sensors to gather information regarding people and devices. However, when developing novel user experiences, researchers are left to building foundation code to support multiple network-connected sensors, a major hurdle to rapidly developing and testing new ideas.

Detecting and analyzing protein cavities provides significant information about active sites for biological processes (e.g., protein-protein or protein-ligand binding) in molecular graphics and modeling. Using the three-dimensional structure of a given protein (i.e., atom types and their locations in 3D) as retrieved from a PDB (Protein Data Bank) file, it is now computationally viable to determine a description of these cavities. Such cavities correspond to pockets, clefts, invaginations, voids, tunnels, channels, and grooves on the surface of a given protein. In this work, we survey the literature on protein cavity computation and classify algorithmic approaches into three categories: evolution-based, energy-based, and geometry-based. Our survey focuses on geometric algorithms, whose taxonomy is extended to include not only sphere-, grid-, and tessellation-based methods, but also surface-based, hybrid geometric, consensus, and time-varying methods. Finally, we detail those techniques that have been customized for GPU (Graphics Processing Unit) computing.

We present a framework to manage multimodal applications and interfaces in a reusable and extensible manner. We achieve this by focusing the architecture both on applications' needs and devices' capabilities. One particular domain we want to approach is collaborative environments where several modalities and applications make it necessary to provide for an extensible system combining diverse components across heterogeneous platforms on-the-fly. This paper describes the proposed framework and its main contributions in the context of an architectural application scenario. We demonstrate how to connect different non-conventional applications and input modalities around an immersive environment (tiled display wall).

The authors present Virtual Worlds as an open architecture for large-scale virtual environments with multi-users. It is based on the integration of VRML, Java and the Internet bringing forth a cross-platform solution. Virtual Worlds is furnished with a set of tools and APIs that allow one to create, extend and customise one's own virtual environment.