Indirect vision through a mirror, while bi-manually manipulating both the mirror and another tool is a relatively common way to perform operations in various types of surgery. However, learning such psychomotor skills requires extensive training; they are difficult to teach; and they can be quite costly, for instance, for dentistry schools. In order to study the effectiveness of VR simulators for learning these kinds of skills, we developed a simulator for training dental surgery procedures, which supports tracking of eye gaze and tool trajectories (mirror and drill), as well as automated outcome scoring. We carried out a pre-/post-test study in which 30 fifth-year dental students received six training sessions in the access opening stage of the root canal procedure using the simulator. In addition, six experts performed three trials using the simulator. The outcomes of drilling performed on realistic plastic teeth showed a significant learning effect due to the training sessions. Also, students with larger improvements in the simulator tended to improve more in the real-world tests. Analysis of the tracking data revealed novel relationships between several metrics w.r.t. eye gaze and mirror use, and performance and learning effectiveness: high rates of correct mirror placement during active drilling and high continuity of fixation on the tooth are associated with increased skills and increased learning effectiveness. Larger time allocation for tooth inspections using the mirror, i.e., indirect vision, and frequency of inspection are associated with increased learning effectiveness. Our findings suggest that eye tracking can provide valuable insights into student learning gains of bi-manual psychomotor skills, particularly in indirect vision environments.

Deep learning models for scoring sleep stages based on single-channel EEG have been proposed as a promising method for remote sleep monitoring. However, applying these models to new datasets, particularly from wearable devices, raises two questions. First, when annotations on a target dataset are unavailable, which different data characteristics affect the sleep stage scoring performance the most and by how much? Second, when annotations are available, which dataset should be used as the source of transfer learning to optimize performance? In this paper, we propose a novel method for computationally quantifying the impact of different data characteristics on the transferability of deep learning models. Quantification is accomplished by training and evaluating two models with significant architectural differences, TinySleepNet and U-Time, under various transfer configurations in which the source and target datasets have different recording channels, recording environments, and subject conditions. For the first question, the environment had the highest impact on sleep stage scoring performance, with performance degrading by over 14% when sleep annotations were unavailable. For the second question, the most useful transfer sources for TinySleepNet and the U-Time models were MASS-SS1 and ISRUC-SG1, containing a high percentage of N1 (the rarest sleep stage) relative to the others. The frontal and central EEGs were preferred for TinySleepNet. The proposed approach enables full utilization of existing sleep datasets for training and planning model transfer to maximize the sleep stage scoring performance on a target problem when sleep annotations are limited or unavailable, supporting the realization of remote sleep monitoring.

During arthroscopic surgeries, surgeons are faced with challenges like cognitive re-projection of the 2D screen output into the 3D operating site or navigation through highly similar tissue. Training of these cognitive processes takes much time and effort for young surgeons, but is necessary and crucial for their education. In this study we want to show how to recognize states of confusion of young surgeons during an arthroscopic surgery, by looking at their eye and head movements and feeding them to a machine learning model. With an accuracy of over 94% and detection speed of 0.039 seconds, our model is a step towards online diagnostic and training systems for the perceptual-cognitive processes of surgeons during arthroscopic surgeries.

Computer sciences and related disciplines evolve around developing, evaluating, and applying algorithms. Typically, an algorithm is not developed from scratch, but uses and builds upon existing ones, which often are proposed and published in scholarly articles. The ability to capture this evolution relationship among these algorithms in scientific literature would not only allow us to understand how a particular algorithm is composed, but also shed light on large-scale analysis of algorithmic evolution through different temporal spans and thematic scales. We propose to capture such evolution relationship between two algorithms by investigating the knowledge represented in citation contexts, where authors explain how cited algorithms are used in their works. A set of heterogeneous ensemble machine-learning methods is proposed, where the combination of two base classifiers trained with heterogeneous feature types is used to automatically identify the algorithm usage relationship. The proposed heterogeneous ensemble methods achieve the best average F1 of 0.749 and 0.905 for fine-grained and binary algorithm citation function classification, respectively. The success of this study will allow us to generate a large-scale algorithm citation network from a collection of scholarly documents representing multiple time spans, venues, and fields of study. Such a network will be used as an instrument not only to answer critical questions in algorithm search, such as identifying the most influential and generalizable algorithms, but also to study the evolution of algorithmic development and trends over time.

With the minute margins of error in endodontic surgery, training in manual dexterity and proper instrument handling are crucial components in the dental curriculum. Important parameters include tool path, tool angulation, and force applied. In this work, we focus on training of correct application of force. This is particularly challenging since the amounts of force used are on the order of tenths of Newtons, requiring a highly refined tactile sense and incorrect force can cause irreversible damage. Too great a force can cause overdrilling or in extreme cases perforation of the tooth. Too small a force can cause thermal irritation possibly resulting in tissue necrosis. Despite the importance of correct use of force, this is the dimension on which students receive the least tutorial feedback since force information is typically not available in traditional training settings. In this paper, we present an approach to using haptic feedback as a means to convey formative feedback on the correct application of force. Feedback is conveyed to the student graphically and the correct amount of force to apply is trained haptically. The simulator is rewound and the student is asked to redo the stage where the error occurred. Preliminary evaluation against a control group of students who received only feedback concerning outcome shows the feedback mechanism to be effective.

Since malaria is prevalent in less developed and more remote areas in which public health resources are often scarce, targeted intervention is essential in allocating resources for effective malaria control. To effectively support targeted intervention, predictive models must be not only accurate but they must also have high temporal and spatial resolution to help determine when and where to intervene. In this paper we take the first essential step towards a system to support targeted intervention in Thailand by developing a high resolution prediction model through the combination of Bayes nets and ARIMA. Bayes nets and ARIMA have complementary strengths, with the Bayes nets better able to represent the effect of environmental variables and ARIMA better able to capture the characteristics of the time series of malaria cases. Leveraging these complementary strengths, we develop an ensemble predictor from the two that has significantly better accuracy that either predictor alone. We build and test the models with data from Tha Song Yang district in northern Thailand, creating village-level models with weekly temporal resolution.

Use of simulation to teach decision making in surgery is challenging partly due to the situated nature of the decisions, with situation awareness playing a critical role in making high quality decisions. Thus simulation systems need to be able to provide the key cues needed in making decisions with high fidelity. In this paper we present the first version of Desitra, a simulation environment for teaching decision making in dental surgery. System design was driven by an observational study of teaching sessions for endodontic surgery in the operating room which identified perceptual cues used in decision making as well as tutorial intervention strategies used by surgeons. Desitra provides an open environment for learning decision making -- students carry out dental procedures and are free to make mistakes. The pedagogical module monitors the student actions and intervenes when students make mistakes, providing as little guidance as necessary to keep students on a productive learning path. The system is implemented to run on Android tablets to be maximally accessible. Preliminary evaluation of the system shows that Desitra effectively captures key perceptual cues.

Numerous VR simulators have been developed as a means of addressing limitations of the traditional apprenticeship approach to dental surgical skill training. Most existing simulators support intra- and extra-coronal procedures such as carries removal. In this paper we address the problem of automated outcome assessment for endodontic surgery. Outcome assessment is an essential component of any system that provides formative feedback, which requires assessing the outcome, relating it to the procedure, and communicating in a language natural to dental students. This paper takes a first step toward automated generation of such comprehensive feedback. Our system automatically computes reference templates based on tooth anatomy, which provides flexibility to adjust parameters such as tolerance and to create new templates on demand. Detailed scores are transformed into the standard scoring language used by dental schools. Preliminary evaluation of our system on fifteen outcome samples with three expert endodontists shows a high degree of agreement with expert scores.

Crowdsensing can provide real time and detailed information about rapidly evolving crisis situations to facilitate rapid response and effective resource allocation. But while challenges such as heterogeneity of data content and quality, asynchronicity, and volume call for robust data integration and interpretation capabilities, situation awareness in crowdsensing for crisis management remains a largely unexplored area of research. In this paper we extend the mobile4D smartphone-based disaster reporting and alerting system with a situation awareness data interpretation and integration layer and demonstrate its application to the problem of tracking cholera outbreaks. The communication workflow in mobile4D-SA supports interaction between crowdsensed information, system predictions, and multifaceted communication between authorities and affected people on the ground.

Probabilistic predictive models are often used in decision optimization applications. Optimal decision making in these applications critically depends on the performance of the predictive models, especially the accuracy of their probability estimates. In this paper, we propose a probabilistic model for revenue maximization and cost minimization across applications in which a decision making agent is faced with a group of possible customers and either offers a variable discount on a product or service or expends a variable cost to attract positive responses. The model is based directly on optimizing expected revenue and makes explicit the relationship between revenue and the customer's response behavior. We derive an expectation maximization (EM) procedure for learning the parameters of the model from historical data, prove that the model is asymptotically insensitive to selection bias in historical decisions, and demonstrate in a series of experiments the method's utility for optimizing financial aid decisions at an international institute of higher learning.

Problem based learning is becoming widely popular as an effective teaching method in medical education. Paying individual attention to a small group of students in medical PBL can place burden on the workload of medical faculty whose time is very costly. Intelligent tutoring systems offer a cost effective alternative in helping to train the students, but they are typically prone to brittleness and the knowledge acquisition bottleneck. Existing tutoring systems accept a small set of approved solutions for each problem scenario stored into the system. Plausible student solutions that lie outside the scope of the explicitly encoded ones receive little acknowledgment from the system. Tutoring hints are also confined to the knowledge space of the approved solutions, leading to brittleness in the tutoring approach. We report a tutoring system for medical PBL that employs the widely available medical knowledge source UMLS as the domain ontology. We exploit the structure of the ontology to expand the plausible solution space and generate hints based on the problem solving context. Evaluation of student learning outcomes led to highly significant learning gains (Mann-Whitney, p<0.001).

We developed an augmented reality (AR) dental training simulator utilizing a haptic (force feedback) device. A number of dental procedures such as crown preparation and opening access to the pulp can be simulated with various shapes of dental drill. The system allows students to practise surgery in the correct postures as in the actual environment by combining 3D tooth and tool models upon the real-world view and displaying the result through a video see-through head mounted display (HMD). The system monitors the important features such as applied forces and tool movement that characterize the quality of the procedure. Automatic performance assessment is achieved by comparing outcome and process features of a student with the best matching expert. Moreover, we incorporated kinematic feedback and hand guidance by haptic device. The result from an initial evaluation shows that the simulator is promising for supplemental training.

We have developed a virtual reality (VR) and an augmented reality (AR) dental training simulator utilizing a haptic device. The simulators utilize volumetric force feedback computation and real time modification of the volumetric data. They include a virtual mirror to facilitate indirect vision during a simulated operation. The AR environment allows students to practice surgery in correct postures by combining the 3D tooth and tool models with the real-world view and displaying the result through a video see-through head-mounted display (HMD). Preliminary results from an initial evaluation show that the system is a promising tool to supplement dental training and that there are advantages of the AR over the VR approach.

Traditional methods of dental surgical skills training and assessment are being challenged by complications such as unavailability of expert supervision and the subjective manner of surgical skills assessment. This paper presents a dental surgical skills training system that provides a virtual reality environment with a haptic device for dental students to practice tooth preparation procedures. The system monitors important features of the procedures, objectively assesses the quality of the performed procedure and provides objective feedback on the user's performance for each stage in the procedure. We evaluated the accuracy of the skill assessment with data collected from novice dental students as well as experienced dentists. The experimental results show high accuracy in classifying users into novice and expert. The evaluation of the system's generated feedback also indicated a high acceptance rate.

In this paper, we present a novel collaborative authoring tool that was designed to allow medical teachers to formalize and visualize their knowledge for medical intelligent tutoring systems. Our goal is to increase the efficiency and effectiveness in creating the domain model - often referred to as the bottleneck in developing intelligent tutoring systems. We incorporate the Unified Medical Language System (UMLS) knowledge base to assist the authors in creating the problem solution collaboratively via the videoconferencing platform. The system consists of a share workspace gathering information visualization and tools necessary for collaborative problem-solving tasks. We found that the authoring tool can be used as a standalone program to effectively elicit the knowledge structure of the domain model. This was achieved in hours compared to months for the conventional paper-based approach.

In this paper, we describe the operation of barter trade exchanges by identifying key techniques used by trade brokers to stimulate trade and satisfy member needs, and present algorithms to automate some of these techniques. In particular, we develop algorithms that emulate the practice of trade brokers by matching buyers and sellers in such a way that trade volume is maximized while the balance of trade is maintained as much as possible. We show that the buyer/seller matching and trade balance problems can be decoupled, permitting efficient solution as well as numerous options for matching strategies.We model the trade balance problem as a minimum cost circulation problem (MCC) on a network. When the products have uniform cost or when the products can be traded in fractional units, we solve the problem exactly. Otherwise, we present a novel stochastic rounding algorithm that takes the fractional optimal solution to the trade balance problem and produces a valid integer solution. We then make use of a greedy heuristic that attempts to match buyers and sellers so that the average number of suppliers that a buyer must use to satisfy a given product need is minimized.We present results on the empirical evaluation of our algorithms on test problems and simulations. Experiments show that our algorithm (MCC + stochastic rounding) runs in a fraction of the time of a commercial mixed integer programming (MIP) package while producing solutions that are always within 0.7% of the MIP solution. We evaluate the effectiveness of our algorithm on maintaining balance and on stimulating trade using two different simulation techniques, both based on transaction history data from a trade exchange. The simulation results support the barter trade exchange rule of thumb that maximizing single-period trade volume while maintaining balance of trade helps to maximize trade volume over the long run.

This paper describes COMET, a collaborative intelligent tutoring system for medical problem-based learning. The system uses Bayesian networks to model individual student knowledge and activity, as well as that of the group. It incorporates a multi-modal interface that integrates text and graphics so as to provide a rich communication channel between the students and the system, as well as among students in the group. Students can sketch directly on medical images, search for medical concepts, and sketch hypotheses on a shared workspace. The prototype system incorporates substantial domain knowledge in the area of head injury diagnosis. A major challenge in building COMET has been to develop algorithms for generating tutoring hints. Tutoring in PBL is particularly challenging since the tutor should provide as little guidance as possible while at the same time not allowing the students to get lost. From studies of PBL sessions at a local medical school, we have identified and implemented eight commonly used hinting strategies. We compared the tutoring hints generated by COMET with those of experienced human tutors. Our results show that COMET's hints agree with the hints of the majority of the human tutors with a high degree of statistical agreement (McNemar test, p = 0.652, Kappa = 0.773).

We present an approach to elicitation of user preference models in which assumptions can be used to guide but not constrain the elicitation process. We show how to encode assumptions concerning preferential independence and monotonicity in a Knowledge-Based Artificial Neural Network. We quantify the degree to which user preferences violate a set of assumptions. We empirically compare the KBANN network with an unbiased ANN in terms of learning rate and accuracy for preferences consistent and inconsistent with the assumptions. We go on to demonstrate how the technique can be used to learn a fine-grained preference structure from simple binary classification data.

AI planning agents are goal‐directed: success is measured in terms of whether an input goal is satisfied. The goal gives structure to the planning problem, and planning representations and algorithms have been designed to exploit that structure. Strict goal satisfaction may be an unacceptably restrictive measure of good behavior, however. A general decision‐theoretic agent, on the other hand, has no explicit goals: success is measured in terms of an arbitrary preference model or utility function defined over plan outcomes. Although it is a very general and powerful model of problem solving, decision‐theoretic choice lacks structure, which can make it difficult to develop effective plan‐generation algorithms. This paper establishes a middle ground between the two models. We extend the traditional AI goal model in several directions: allowing goals with temporal extent, expressing preferences over partial satisfaction of goals, and balancing goal satisfaction against the cost of the resources consumed in service of the goals. In doing so we provide a utility model for a goal‐directed agent. An important quality of the proposed model is its tractability. We claim that our model, like classical goal models, makes problem structure explicit. This structure can then be exploited by a problem‐solving algorithm. We support this claim by reporting on two implemented planning systems that adopt and exploit our model.

We present a first-order logic of time, chance, and probability that is capable of expressing the four types of higher-order probability sentences relating subjective probability and objective chance at different times. We define a causal notion of objective chance and show how it can be used in conjunction with subjective probability to distinguish between causal and evidential correlation by distinguishing between conditions, events, and actions that: 1) influence the agent's belief in chance; and 2) the agent believes to influence chance. Furthermore, the semantics of the logic captures some common sense inferences concerning objective chance and causality. We show that an agent's subjective probability is the expected value of its beliefs concerning objective chance. We also prove that an agent using this representation believes with certainty that the past cannot be causally influenced.

We present an educational tool for bringing the information contained in a Bayesian network to the end user in an easily intelligible form. The BANTER shell is designed to tutor users in evaluation of hypotheses and selection of optimal diagnostic procedures. BANTER can be used with any Bayesian network containing nodes that can be classified into hypotheses, observations, and diagnostic procedures. We present algorithms for determining optimal diagnostic procedures and for explanation generation.>