Phantom pain is a chronic pain that is experienced as a vivid sensation stemming from the missing limb. From traditional mirror box to virtual reality-based approaches, a wide spectrum of treatments using mimic feedback of the amputated limb have been developed for alleviating phantom limb pain. In our previous work, Mixed reality-based framework for MAnaging Phantom Pain (Mr.MAPP) was presented and used to generate a virtual phantom upper limb, in real time, to manage the phantom pain. However, amputation of the lower limb is more common than that of the upper limb. Hence, in this paper, on top of demonstrating the reproducibility of the Mr.MAPP framework for upper limb, we extend it to manage lower limb phantom pain as well. Unlike an upper limb amputee, a patient with lower limb amputated is constrained to perform the training procedure in a sitting posture. Accordingly, virtual training games are designed for lower limb exercises with sitting posture such as knee flexion and extension, ankle dorsiflexion and tandem coordinated movement. Finally, the technical details of the system setup for playing the training games are introduced.

For ensuring safety and for handling emergency situations, most autonomous vehicles use remote human operators by communicating the road condition captured using LiDAR (Light Detection and Ranging) and stereo RGB cameras [10, 11]. Recent research [2, 3] has identified several possible scenarios of forgery attacks during such data communication between autonomous vehicles and human operators. Hence, there is a distinct requirement for forensics research on the multi-modal LiDAR and RGB camera data generated from autonomous vehicles. In this paper, we present a new dataset, ADD-FAR (Attacked Driving Dataset for Forensics Analysis and Research) that contains forged driving scenarios based on KITTI Vision Benchmark Suite [4, 5]. This dataset is created by identifying objects of interest using automated 3D object detection and carrying out the attacks with different levels of risk as defined in [3]. As part of the dataset, we are also providing the scripts used for automatically generating the different types of attacks. These scripts can easily be modified to work with other driving datasets apart from KITTI as well as for creating new forms of attacks.

This article proposes new tools to detect the tampering of video feeds from surveillance cameras. Our proposal illustrates the unique cyber-physical properties that sensor devices can leverage for their cyber-security. While traditional attestation algorithms exchange digital challenges between devices authenticating each other, our work instead proposes challenges that manifest physically in the field of view of the camera (e.g., a QR code in a display). This physical (challenge) and cyber (verification) attestation mechanism can help protect systems even when the sensors (cameras) and actuators (a display, infrared LEDs, color light bulbs) are compromised. In this article, we consider skillful adversaries that can capture the correct challenges (our system is sending) and can re-create them in the response to try fooling our verification system, and we propose new algorithms to detect these powerful attackers. Also, we introduce new visual challenges that make harder for anti-forensics attackers to succeed, and we present experimental results showing how our system is robust against a variety of attacks ranging from naive attacks to more sophisticated anti-forensics attackers.

With the ever-increasing popularity of LiDAR (Light Image Detection and Ranging) sensors, a wide range of applications such as vehicle automation and robot navigation are developed utilizing the 3D LiDAR data. Many of these applications involve remote guidance - either for safety or for the task performance - of these vehicles and robots. Research studies have exposed vulnerabilities of using LiDAR data by considering different security attack scenarios. Considering the security risks associated with the improper behavior of these applications, it has become crucial to authenticate the 3D LiDAR data that highly influence the decision making in such applications. In this paper, we propose a framework, ALERT (Authentication, Localization, and Estimation of Risks and Threats), as a secure layer in the decision support system used in the navigation control of vehicles and robots. To start with, ALERT tamper-proofs 3D LiDAR data by employing an innovative mechanism for creating and extracting a dynamic watermark. Next, when tampering is detected (because of the inability to verify the dynamic watermark), ALERT then carries out cross-modal authentication for localizing the tampered region. Finally, ALERT estimates the level of risk and threat based on the temporal and spatial nature of the attacks on LiDAR data. This estimation of risk and threats can then be incorporated into the decision support system used by ADAS (Advanced Driver Assistance System). We carried out several experiments to evaluate the efficacy of the proposed ALERT for ADAS and the experimental results demonstrate the effectiveness of the proposed approach.

With the rising popularity of handheld virtual reality (VR) devices and depth sensing RGB-D cameras, a variety of VR applications merging these two technologies has been suggested. However, immersive quality of experience in such VR applications is constrained mainly by the large data size and the hardware limitations to handle it. The depth data captured by RGB-D cameras provide a dense sampling of the surface, resulting in a high-poly mesh, which is difficult to be rendered on handheld VR devices due to their limited processing power. To improve the immersive VR experience, a sparse approximation of the depth data is needed. Traditional mesh and point cloud simplification methods are iterative and so are unsuitable for real-time applications. In this paper, we introduce a depth-imagebased approach that is capable of generating a good quality sparse mesh for visualization in real time. We propose a curvaturesensitive surface simplification-CS3operator that assigns an importance measure to each point in the depth image, based on the local curvature. Further, it applies an importance-order-based restrictive sampling to generate a sparse representation that retains the overall shape as well as the finer features of the object. We also modify the 2-D sweep-line-based constrained Delaunay triangulation to generate 3-D meshes from the sparse point sampling obtained using CS3. In addition, the proposed approach preserves key surface properties, such as texture coordinates and materials. We used three different datasets containing dense 3-D models with and without texture, which are scanned using various sensors to validate and compare the robustness, real-time performance, and accuracy of the proposed method over existing approaches. Based on the experimental results, we show that the proposed CS3operator and modified 2-D sweep-line-based triangulation generate sparse meshes from depth image in real time, performing significantly faster than current state-of-the-art methods while maintaining similar visual quality.

With the increasing accessibility of the mobile head-mounted displays (HMDs), mobile virtual reality (VR) systems are finding applications in various areas. However, mobile HMDs are highly constrained with limited graphics processing units (GPUs) and low processing power and onboard memory. Hence, VR developers must be cognizant of the number of polygons contained within their virtual environments to avoid rendering at low frame rates and inducing simulator sickness. The most robust and rapid approach to keeping the overall number of polygons low is to use mesh simplification algorithms to create low-poly versions of pre-existing, high-poly models. Unfortunately, most existing mesh simplification algorithms cannot adequately handle meshes with lots of boundaries or nonmanifold meshes, which are common attributes of many 3D models. In this article, we present QEM 4VR , a high-fidelity mesh simplification algorithm specifically designed for VR. This algorithm addresses the deficiencies of prior quadric error metric (QEM) approaches by leveraging the insight that the most relevant boundary edges lie along curvatures while linear boundary edges can be collapsed. Additionally, our algorithm preserves key surface properties, such as normals, texture coordinates, colors, and materials, as it preprocesses 3D models and generates their low-poly approximations offline. We evaluated the effectiveness of our QEM 4VR algorithm by comparing its simplified-mesh results to those of prior QEM variations in terms of geometric approximation error, texture error, progressive approximation errors, frame rate impact, and perceptual quality measures. We found that QEM 4VR consistently yielded simplified meshes with less geometric approximation error and texture error than the prior QEM variations. It afforded better frame rates than QEM variations with boundary preservation constraints that create unnecessary lower bounds on overall polygon count reduction. Our evaluation revealed that QEM 4VR did not fair well in terms of existing perceptual distance measurements, but human-based inspections demonstrate that these algorithmic measurements are not suitable substitutes for actual human perception. In turn, we present a user-based methodology for evaluating the perceptual qualities of mesh simplification algorithms.

3D LiDAR (Light Imaging Detection and Ranging) data has recently been used in a wide range of applications such as vehicle automation and crime scene reconstruction. Decision making in such applications is highly dependent on LiDAR data. Thus, it becomes crucial to authenticate the data before using it. Though authentication of 2D digital images and video has been widely studied, the area of 3D data forensic is relatively unexplored. In this paper, we investigate and identify three possible attacks on the LiDAR data. We also propose two novel forensic approaches as a countermeasure for such attacks and study their effectiveness. The first forensic approach utilises the density consistency check while the second method leverages the occlusion effect for revealing the forgery. Experimental results demonstrate the effectiveness of the proposed forgery attacks and raise the awareness against unauthenticated use of LiDAR data. The performance analyses of the proposed forensic approaches indicate that the proposed methods are very efficient and provide the detection accuracy of more than 95% for certain kinds of forgery attacks. While the forensic approach is unable to handle all forgery attacks, the study motivates to explore more sophisticated forensic methods for LiDAR data.

Current state-of-the-art mesh quality measures evaluate closed and complete meshes obtained after mesh postprocessing applications, such as mesh simplification or watermarking, and compare them against the corresponding reference mesh. Emerging 3D immersive VR/AR applications use noisy 3D point cloud, typically from single RGB-D camera (such as Microsoft's Kinect) to generate standalone (no reference) 3D human open mesh (with boundaries) in real time, that needs evaluation. A learning-based objective measure is proposed to rate the visual quality by emulating human perception of 3D human open mesh quality. 2-pronged objective evaluation is performed: (a) Global holistic score captures the efficacy of the mesh to represent the human model as a whole, by considering mesh completeness and mesh noise. (b) Local part-based score caters to the need of varying roughness in different parts of the human body, by finding the deviation in the face normals for all the adjacent triangles in that part (segment). Learning technique aligns the objective scores with the subjective user evaluation, in turn combining the concepts of white-box and black-box evaluation for 3D meshes. Experimental results for a database, specifically generated for the purpose proves the efficacy of the proposed method.

Phantom Limb Pain or simply, Phantom Pain is a severe chronic pain that is experienced as a vivid sensation of the pain in missing body part. Epidemiological studies obtained from a large samples indicate that the short-term incidence rate of the phantom limb pain is 72% [13], while long-term incidence rate (6 months after amputation) is 67%, [5, 13]. A wide spectrum of treatments developed for alleviating phantom limb pain includes the traditional mirror box therapy as well as recently developed virtual reality-based methods. Most of the virtual reality-based methods rely on 3D CAD models of the virtual limb, animating them using the motion data acquired either from patient's existing anatomical limb or myoelectric activity at patient's stump (of the amputated limb). Since motion activity is typically captured using body sensors (Electromyography, EMG, or inertial sensors), these methods are considered as invasive approaches. Further, in the case of virtual reality-based methods, the dependency on the pre-built 3D models degrades the immersive experience due to a mismatch in the skin color, clothes, artificial and rigid look and misalignment of the phantom limb.

With the increasing accessibility of the mobile head-mounted displays (HMDs), mobile virtual reality (VR) systems are finding applications in various areas. However, mobile HMDs are highly constrained with limited graphics processing units (GPUs), low processing power and onboard memory. Hence, VR developers must be cognizant of the number of polygons contained within their virtual environments to avoid rendering at low frame rates and inducing simulator sickness. The most robust and rapid approach to keeping the overall number of polygons low is to use mesh simplification algorithms to create low-poly versions of preexisting, high-poly models. Unfortunately, most existing mesh simplification algorithms cannot adequately handle meshes with lots of boundaries or non-manifold meshes, which are common attributes of 3D models made with computer-aided design tools.; [email protected] this paper, we present a high-fidelity mesh simplification algorithm specifically designed for VR. This new algorithm, QEM4VR, addresses the deficiencies of prior quadric error metric (QEM) approaches by leveraging the insight that the most relevant boundary edges lie along curvatures while linear boundary edges can be collapsed. Additionally, our QEM4VR algorithm preserves key surface properties, such as normals, texture coordinates, colors, and materials. It pre-processes the 3D models and generate their low-poly approximations offline. We used six publicly available, high-poly models, with and without textures to compare the accuracy and fidelity of our QEM4VR algorithm to previous QEM variations. We also performed a frame rate analysis with original high-poly models and low-poly models obtained using QEM4VR and previous QEM variations. Our results indicate that QEM4VR creates low-poly, high-fidelity virtual environments for VR applications on devices that are constrained by the low number of polygons they can work with.

3D Tele-Immersion systems allow geographically distributed users to interact in a virtual world using their "live" 3D models. The capture, reconstruction, transfer, and rendering of these models introduce significant latency into the system. Implicit Latency (ℒ') can be estimated using system clocks to measure the time after the data was received from the RGB-D camera, till the request to render the result. The Observed Latency (ℒ) between a real world event and the event being rendered on the display, cannot be accurately represented by ℒ' since ℒ' ignores the time taken to capture, or update the display, etc. In this paper, a Visual Pattern based Latency Estimation (VPLE) approach is introduced to calculate the real world visual latency of a system without the need for any custom hardware. VPLE generates a constantly changing pattern that is captured and rendered by the 3DTI system. An external observer records both the pattern and the rendered results at high frame rates. ℒ is estimated by calculating the difference between the generated and rendered patterns. VPLE is extended to allow ℒ estimation between geographically distributed sites. Evaluations show that the accuracy of VPLE depends on the refresh rate of the pattern, and is within 4ms. ℒ of a distributed 3DTI system implemented on the GPU is significantly lower than the CPU implementation, and is comparable to video streaming. It is also shown that the ℒ' estimates for GPU based 3DTI implementations are off by almost 100% compared to the ℒ.

Rehabilitation for stroke afflicted patients, through exercises tailored for individual needs, aims at relearning basic motor skills, especially in the extremities. Rehabilitation through Augmented Reality (AR) based games engage and motivate patients to perform exercises which, otherwise, maybe boring and monotonic. Also, mirror therapy allows users to observe one's own movements in the game providing them with good visual feedback. This paper presents an augmented reality based system for rehabilitation by playing four interactive, cognitive and fun Exergames (exercise and gaming).

RGB-D cameras have enabled real-time 3D video processing for numerous computer vision applications, especially for surveillance type applications. In this paper, we first present a real-time anti-forensic 3D object stream manipulation framework to capture and manipulate live RBG-D data streams to create realistic images/videos showing individuals performing activities they did not actually do. The framework uses computer vision and graphics methods to render photorealistic animations of live mesh models captured using the camera. Next, we conducted a visual inspection of the manipulated RGB-D streams (just like security personnel would do) by users who are computer vision and graphics scientists. The study shows that it was significantly difficult to distinguish between the real or reconstructed rendering of such 3D video sequences, thus clearly showing the potential security risk involved. Finally, we investigate the efficacy of forensic approaches for detecting such manipulations.

3D Tele-Immersion (3DTI) has emerged as an efficient environment for virtual interactions and collaborations in a variety of fields like rehabilitation, education, gaming, etc. In 3DTI, geographically distributed users are captured using multiple cameras and immersed in a single virtual environment. The quality of experience depends on the available network bandwidth, quality of the 3D model generated and the time taken for rendering. In a collaborative environment, achieving high quality, high frame rate rendering by transmitting data to multiple sites having different bandwidth is challenging. In this paper we introduce a network adaptive textured mesh generation scheme to transmit varying quality data based on the available bandwidth. To reduce the volume of information transmitted, a visual quality based vertex selection approach is used to generate a sparse representation of the user. This sparse representation is then transmitted to the receiver side where a sweep-line based technique is used to generate a 3D mesh of the user. High visual quality is maintained by transmitting a high resolution texture image compressed using a lossy compression algorithm. In our studies users were unable to notice visual quality variations of the rendered 3D model even at 90% compression.

This paper addresses the problem of land-cover map updating by classification of multitemporal remote-sensing images in the context of domain adaptation (DA). The basic assumptions behind the proposed approach are twofold. The first one is that training data (ground reference information) are available for one of the considered multitemporal acquisitions (source domain) whereas they are not for the other (target domain). The second one is that multitemporal acquisitions (i.e., target and source domains) may be characterized by different sets of classes. Unlike other approaches available in the literature, the proposed DA Bayesian classifier based on maximum a posteriori decision rule (DA-MAP) automatically identifies whether there exist differences between the set of classes in the target and source domains and properly handles these differences in the updating process. The proposed method was tested in different scenarios of increasing complexity related to multitemporal image classification. Experimental results on medium-resolution and very high resolution multitemporal remote-sensing data sets confirm the effectiveness and the reliability of the proposed DA-MAP classifier.