Proficient training in Diagnostic Imaging (DI) is crucial for veterinary students. However, the training process faces challenges, including limited access to animals and strict radiation safety regulations. Can Mixed Reality (MR) address these issues? Two co-located collaborative MR teaching systems were implemented to simulate the horse DI procedure with real-time X-ray simulation. One system integrated actual X-ray equipment, while the other was entirely virtual. A user-centered comparative study was conducted, assessing students’ performance and opinions to explore the differences and the necessity of physical versus virtual equipment. The performance of 22 veterinary students and four DI experts was evaluated, focusing on the quality of the readiographs and the time taken to obtain the X-ray images. Students initially underperformed compared to experts using physical equipment but matched expert performance by the final image, while they performed similarly with virtual equipment. Experts took X-rays faster than students in both cases. Both systems offered comparable user experiences, though most preferred virtual equipment for its usability and presentation. Physical equipment remained essential for realistic simulation. The results show that the MR system, combined with real-time X-ray image synthesis, has great potential to overcome obstacles in veterinary DI training and serve as a support tool for both training and assessment.

Learning robotic skills from raw human videos remains a non-trivial challenge. Previous works tackled this problem by leveraging behavior cloning or learning reward functions from videos. Despite their remarkable performances, they may introduce several issues, such as the necessity for robot actions, requirements for consistent viewpoints and similar layouts between human and robot videos, as well as low sample efficiency. To this end, our key insight is to learn task priors by contrasting videos and to learn action priors through imitating trajectories from videos, and to utilize the task priors to guide trajectories to adapt to novel scenarios. We propose a three-stage skill learning framework denoted as Contrast-Imitate-Adapt (CIA). An interaction-aware alignment transformer is proposed to learn task priors by temporally aligning video pairs. Then a trajectory generation model is used to learn action priors. To adapt to novel scenarios different from human videos, the Inversion-Interaction method is designed to initialize coarse trajectories and refine them by limited interaction. In addition, CIA introduces an optimization method based on semantic directions of trajectories for interaction security and sample efficiency. The alignment distances computed by IAAformer are used as the rewards. We evaluate CIA in six real-world everyday tasks, and empirically demonstrate that CIA significantly outperforms previous state-of-the-art works in terms of task success rate and generalization to diverse novel scenarios layouts and object instances.Note to Practitioners—This work aims to study robot skill learning from raw human videos. Compared with teleoperation or kinesthetic teaching in the laboratory, such learning method can flexibly utilize large-scale human videos available on the Internet, thereby improving the robot’s ability to generalize to various complex scenarios. Previous works on learning from videos usually have some issues, including requirements for robot actions, consistent viewpoints, similar layouts and low sample efficiency. To alleviate these issues, we propose a three-stage skill learning framework CIA. Temporal alignment is utilized to learn task priors through our proposed transformer-based model and self-supervised loss functions. A trajectory generation model is trained to learn the action priors. To further adapt to diverse scenarios, we propose a two-stage policy improvement method by initialization and interaction. An optimization method is introduced to ensure safe interaction and sample efficiency, where the optimization objective is guided by the learned task priors. The experimental results show that our CIA outperforms other state-of-the-art methods in task success rate and generalization to novel scenarios.

Given the rapid advancement of robotic technologies, robots will eventually enter our daily lives, performing complex long-horizon tasks. Complex long-horizon tasks, such as furniture assembly, typically contain dozens of subtasks with various scenes. Although complex, assembly is based on several reusable movements. Movement Primitive (MP) is a promising framework for learning reusable movements from demonstrations and adapting the learned movements to the test scenes. The critical step in employing MP methods is categorizing the unlabeled demonstrations into different MPs. However, current MP methods focus on individual MP learning using manually selected demonstrations, neglecting categorization. Manual categorization of demonstrations is easy to fall into suboptimal. If the demonstrations within the same category are too similar, the learned MP cannot be adapted to task scenes with various obstacles. Conversely, a significant distance between demonstrations leads to the MP’s failure in learning. To this end, we propose the following principle for MP categorization: balance the Learnability and Adaptability. Following this principle, we introduce an optimal transportation (OT)-based theoretical framework and a practical solution utilizing an auto-encoder network. We obtain the lower threshold of learnability by OT. Then we increase the adaptability of MP until it reaches the lower threshold of learnability. For complex long-horizon task learning, we propose a balanced MPs-based learning framework that contains four modules, termed BaMPs. BaMPs achieved success rates of 100% and 80%, respectively, in the 12-step and 20-step tasks.

As part of the clinical training during their degree course, veterinary students learn how to safely obtain radiographs in horses. However, this can sometimes be challenging due to ethical considerations related to the use of live animals and the fact that it is strictly dependent on the caseload of the veterinary teaching hospital. This networked setup allows the lecturer to guide students in equine radiographic techniques without requiring actual horses or an X-ray machine. A study involving veterinary students showed promising results regarding the effectiveness of using MR to teach radiation safety while performing radiographic techniques on horses. In addition to performance metrics, we employed questionnaires, including MREQ, VRSQ, and UEQ, to collect demographic data and participant feedback. Participants praised the system’s pedagogical effectiveness and overall user experience. The immersive MR experience created a sense of presence and co-presence, underscoring the potential for broader applications of co-located MR in radiology and other areas.

Light Field Displays (LFD) offer the potential for a true window into a virtual world without any form of headset. However, the vast majority of LFD only provide motion parallax along the x-axis (horizontal) due to its domination of human 3D perception. The additional motion parallax along y and the z-axis are rarely or never achieved and let alone evaluated. This paper proposed the first approach that provided the real full-motion parallax covering the z-axis. Moreover, this generic approach enables on-demand y and z-axis motion parallax on any off-the-shelf LFD. A prototype was created according to the proposed approach, and with the use of it, an experiment with two tasks was carried out among 24 participants. This pioneering study explored the effect of the motion parallax along the additional axes. Subjective and objective metrics were collected to measure participants’ 3D perception on four viewing conditions (LFD with motion parallax along the x-axis, x-and-y-axis, x-and-z-axis, and x-y-and-z-axis). The study manifested three main findings: 1) The additional y-axis and z-axis motion parallax increased viewers’ engagement with the LFD. 2) LFD with full-motion parallax provided the optimal user experience. 3) The additional motion parallax along with the y-axis increased viewers’ user experience more than the z-axis.

Due to the global pandemic, heavily practical based teaching in veterinary education has been hugely impacted. Even before the crisis, an innovative way to provide an assessment that could simulate the examination of a cadaver has been a long-standing dream for veterinary schools and faculty. We designed and implemented a summative assessment system using touchless interaction head-mounted display (HMD) Virtual Reality (VR), and presented a user study that was an initial exploration of the proposed system designed to replace traditional spatial multiple choices questions (MCQs) for veterinary students. The performance relating to the score and time spent of 24 veterinary students was measured in online MCQ and VR MCQ. Questionnaires were sent out to gather demographic data and opinions. Manipulation time in the VR environment was tracked. The results showed that the time spent was significantly shorter using VR as an assessment tool compared to online MCQ. Participants revealed that the VR assessment outperformed the traditional method in terms of satisfaction, distinctiveness, concentration, and nervousness levels. However, they underperformed in the VR examination significantly. The summative assessment in VR has the potential to further support veterinary education, and the usage of HMDs and touchless interaction will boost the process. This evaluation illustrates the gap between students’ perception of VR examination and their MCQ performance that will require future research to untangle if this technology is to be adopted in the anatomy classroom.

Augmented Reality and Virtual Reality become increasingly popular in scientific visualization especially for education where they can support collaborative scientific visualization experiences in the classroom. However, the inherent limitations of head-mounted AR and VR tools are stemming the popularization of these existing content sharing tools. Instead of sharing 3D educational content between AR/VR headsets, this paper proposes a novel prototype Mixed rEaliTy shAring pLatform (METAL) to allow for 3D educational content to be shared between a Microsoft HoloLens 2 and multiple Looking Glass displays which are a type of Light Field (Multi-view Autostereoscopic) display. This platform allows one teacher to use a HoloLens to manipulate and share different 3D contents with multiple student groups via the network, thus each student group can observe the synchronized 3D educational content with autostereoscopic experiences. Therefore, this proposed prototype enables a low-cost one-to-multiple 3D content sharing experience that allows the intuitive 3D model interaction and seamless communication between the students and the teacher.

During the last decade, there has been an increase in the development of training systems based on Virtual Reality (VR) and Augmented Reality (AR). This paper describes work in progress on exploring the challenges in the creation of anatomy simulators for professionals such as Doctors and Veterinarians. Given the current limitations of workstation powered VR headsets which are primarily their lack of mobility both for a given machine and the Head Mounted Display (HMD) which is commonly tethered. This lack of mobility limits the efficiency of communications between the lecturers and students. This work explores using AR-based smartphone version and mobile VR HMD as an alternative. This work-in-progress paper describes the comparison between the mobile platform AR and mobile immersive headset VR in different aspects of Veterinary Education.

Advances in virtual immersive and augmented reality technology, commercially available for the entertainment and gaming industry, hold potential for education and clinical use in medicine and the field of radiology. Radiology departments have begun exploring the use of these technologies to help with radiology education and training. The purpose of this article is to address how skills have been developed in the gaming world and how these can be adopted for radiology education. Radiology is rapidly evolving with the use of AI for more effective diagnostic and prognostic clinical assessment. Advances in computing technology have enabled widespread availability of simulated reality technologies, including virtual reality (VR) and augmented reality (AR). This work in progress paper describes VR and AR technologies as novel means to communicate and have potential for supplementing radiology training; communicating with colleagues, referring clinicians, and patients; and aiding in interventional radiology procedures.

VR remote learning is an environment-friendly approach for anatomy learning compared with the paper-based, physical model or prepared specimens. However, VR devices can potentially be costly pieces of equipment, and inexperienced users can experience unwanted symptoms like motion sickness. To solve these problems, an asymmetrical system has been developed to connect an experienced head-mounted-display user (lecturer) with light field display users (students) through the Internet. The scenes of lecturer’s end and students’ end are adjusted to match the corresponding displays technologies.

With the explosive growth of digitalized data, knowledge discovery from data is attracting more attentions from different domains. Lots of data analysis and mining algorithms have been proposed in machine learning field. But most of these works concentrated on outperforming previous ones on more datasets. Performance on a given application dataset remains unclear. To evaluating all the available algorithms against a particular dataset can be computationally expensive and tedious. As a result, it's difficult for domain experts to choose a suitable data analysis algorithm. The frequently applied methods are always limited to several popular ones. This paper investigates the characteristics of dataset with 23 proposed quantitative indicators. With these quantitative descriptions, a novel framework PepAls is proposed as an initial attempt to recommend a suitable analysis algorithm for a given application dataset. The highest analysis result of the dataset, which could be reached by up to date methods, is also predicted by PepAls. Using PepAls, domain engineers can roughly preview the analysis result and quickly get the suitable algorithm. Newly proposed methods in machine learning can be easily introduced to application. PepAls is a framework, which can serve different kinds of learning problem when embedded different recommendation models. This paper uses multi-label learning problem as an example to illustrate the kernel idea. Elaborated experiments show that our approach is practical and helpful to narrow the gap of algorithm design and application.