There have been several attempts to develop machine learning (ML) models to identify human infecting viruses from their genomic sequences, with varying degrees of success. Direct comparison between models is problematic, because these models are typically trained and evaluated on different datasets with alternative data splitting schemes, features, and model performance metrics. In this paper we present a standardized dataset of mammal infecting and non-infecting viral pathogens, refined from the previous work of Mollentze et al. to include the latest literature evidence, roughly doubling the number of curated host-virus records available to the community, and new host target labels, primate and mammal. The new host labels were included for several reasons, including previous reports that classification performance is better at broader taxonomic ranks and the idea that there may be more data for primate infection that might serve as a suitable proxy for zoonotic potential and avoidance of false positives for human infection due to absence of evidence. On this dataset, we report the performance of eight machine learning models for predicting mammal-infecting viruses from their genomic sequences. We find that randomly assigning cases in our improved dataset to training/testing sets, when compared to the original assignments into training/testing in Mollentze et al., increases the overall average ROC AUC of prediction of human infection from 0.663 ± 0.070 to 0.784 ± 0.013, consistent with the reduction in phylogenetic distance between train and test sets (relative entropy change from 3.00 to 0.08). The broadest host category of mammal infection can be predicted most reliably at 0.850 ± 0.020. We share our improved dataset and code to enable standardized comparisons of machine learning methods to predict human host infections. Overall, we have presented preliminary evidence that classification of virus host infection is more tractable at higher taxonomic ranks, that unsurprisingly reducing the phylogenetic distance between training and test sets can improve predictive performance, that peptide kmer features appear to be harmful to out of sample model performance, and we are left with the question of whether models for virus host prediction can reasonably be expected to perform well in out of sample scenarios given the likelihood that viruses do not share a common ancestor. Consistent with this concern, when the data is resampled such that there is no overlap between viral families in training and test sets (relative entropy > 24), models perform no better than random chance at prediction of human infection regardless of whether kmers are included (ROC AUC 0.50 ± 0.08) or not (ROC AUC 0.50 ± 0.04).

Machine learning classifiers have been demonstrated, both empirically and theoretically, to be robust to label noise under certain conditions --- notably the typical assumption is that label noise is independent of the features given the class label. We provide a theoretical framework that generalizes beyond this typical assumption by modeling label noise as a distribution over feature space. We show that both the scale and the \emph{shape} of the noise distribution influence the posterior likelihood; and the shape of the noise distribution has a stronger impact on classification performance if the noise is concentrated in feature space where the decision boundary can be moved. For the special case of uniform label noise (independent of features and the class label), we show that the Bayes optimal classifier for $c$ classes is robust to label noise until the ratio of noisy samples goes above $\frac{c-1}{c}$ (e.g. 90\% for 10 classes), which we call the \emph{tipping point}. However, for the special case of class-dependent label noise (independent of features given the class label), the tipping point can be as low as 50\%. Most importantly, we show that when the noise distribution targets decision boundaries (label noise is directly dependent on feature space), classification robustness can drop off even at a small scale of noise. Even when evaluating recent label-noise mitigation methods we see reduced accuracy when label noise is dependent on features. These findings explain why machine learning often handles label noise well if the noise distribution is uniform in feature-space; yet it also points to the difficulty of overcoming label noise when it is concentrated in a region of feature space where a decision boundary can move.

BACKGROUND: Manual extraction of information from electronic pathology (epath) reports to populate the Surveillance, Epidemiology, and End Result (SEER) database is labor intensive. Systematizing the data extraction automatically using machine-learning (ML) and natural language processing (NLP) is desirable to reduce the human labor required to populate the SEER database and to improve the timeliness of the data. This enables scaling up registry efficiency and collection of new data elements. To ensure the integrity, quality, and continuity of the SEER data, the misclassification error of ML and NPL algorithms needs to be negligible. Current algorithms fail to achieve the precision of human experts who can bring additional information in their assessments. Differences in registry format and the desire to develop a common information extraction platform further complicate the ML/NLP tasks. The purpose of our study is to develop triage rules to partially automate registry workflow to improve the precision of the auto-extracted information. RESULTS: This paper presents a mathematical framework to improve the precision of a classifier beyond that of the Bayes classifier by selectively classifying item that are most likely to be correct. This results in a triage rule that only classifies a subset of the item. We characterize the optimal triage rule and demonstrate its usefulness in the problem of classifying cancer site from electronic pathology reports to achieve a desired precision. CONCLUSIONS: From the mathematical formalism, we propose a heuristic estimate for triage rule based on post-processing the soft-max output from standard machine learning algorithms. We show, in test cases, that the triage rule significantly improve the classification accuracy.

We revisit the problem of computing the path with the minimum cost in terms of the expected number of link layer transmissions (including retransmissions) in wireless mesh networks. Unlike previous efforts, such as the popular ETX, we account for the fact that MAC protocols (including the IEEE 802.11 MAC) incorporate a finite number of transmission attempts per packet. This in turn leads to our key observation: the performance of a path depends not only on the number of the links on the path and the quality of its links, but also, on the relative positions of the links on the path. Based on this observation, we propose ETOP, a path metric that accurately captures the expected number of link layer transmissions required for reliable end-to-end packet delivery. We analytically compute ETOP, which is not trivial, since ETOP is a noncommutative function of the link success probabilities. Although ETOP is a more involved metric, we show that the problem of computing paths with the minimum ETOP cost can be solved by a greedy algorithm. We implement and evaluate a routing approach based on ETOP on a 25-node indoor mesh network. Our experiments show that the path selection with ETOP consistently results in superior TCP goodput (by over 50 percent in many cases) compared to path selection based on ETX. We also perform an in-depth analysis of the measurements to better understand why the paths selected by ETOP improve the TCP performance.

The rising popularity of mobile devices, such as cellular phones and PDAs, has made them a lucrative playground for mobile malware propagation. One common infection vector exploited by these mobile malware is Bluetooth. In this paper, we propose an architecture called Blue-Watchdog that detects Bluetooth worm propagation in public areas based on statistical methods. To achieve fast and accurate Bluetooth worm detection, Blue-Watchdog monitors abrupt changes of average paging rate per Bluetooth device from both temporal and temporal-spatial perspectives. The temporal scheme relies on the CUSUM (Cumulative Sum) sequential test together with the generalized likelihood ratio (GLR), and the temporal-spatial scheme aims to identify spatial regions with abnormally frequent paging attempts. Experimental results show that Blue-Watchdog not only has low false alarm rates, but also effectively detects Bluetooth worms that spread quickly in areas where Bluetooth devices are greatly mixed due to high mobility and also those that propagate relatively slowly in a spatially constrained fashion.

We present PedSims, a suite of mobility models spanning the entire range from the classic random waypoint (RWP), to correlated movement, to trace-level-quality social-activity-based mobility. Instrumenting PedSims at various levels of structural complexity we assess and quantify the impact of real world mobility patterns, such as common interests or synchronous behavior, on ad hoc network performance. Thus, PedSims provides not only a flexible platform for mobility simulation but also a means to strike a trade-off between a tolerable error and computational complexity. Notably, as our study of DSR, AODV and SAFARI finds, neglecting to capture mobility patterns may result in inversion of the ranking of routing protocols as well as in over- and under-estimation of network performance. Further contributions include a bias correction for the sampling of node-node connection time.

We revisit the problem of computing the path with the minimum cost in terms of the expected number of link layer transmissions (including retransmissions) in wireless mesh networks. Unlike previous efforts, such as the popular ETX, we account for the fact that MAC protocols (including the IEEE 802.11 MAC) incorporate a finite number of transmission attempts per packet. This in turn leads to our key observation: the performance of a path depends not only on the number of the links on the path and the quality of its links, but also, on the relative positions of the links on the path. Based on this observation, we propose ETOP, a path metric that accurately captures the expected number of link layer transmissions required for reliable end-to-end packet delivery. We analytically compute ETOP, which is not trivial, since ETOP is a noncommutative function of the link success probabilities. Although ETOP is a more involved metric, we show that the problem of computing paths with the minimum ETOP cost can be solved by a greedy algorithm. We implement and evaluate a routing approach based on ETOP on a 25-node indoor mesh network. Our experiments show that the path selection with ETOP consistently results in superior TCP goodput (by over 50% in many cases) compared to path selection based on ETX. We also perform an in-depth analysis of the measurements to better understand why the paths selected by ETOP improve the TCP performance.

The alarm that worms start to spread on increasingly popular mobile devices calls for an in-depth investigation of their propagation dynamics. In this paper, we study how mobility patterns affect Bluetooth worm spreading speeds. We find that the impact of mobility patterns is substantial over a large set of of changing Bluetooth and worm parameters. For instance, a mobility model under which devices move among a fixed set of activity locations can result in worm propagation speeds four times faster than a classical mobility model such as the random walk model. Our investigation reveals that the key factors affecting Bluetooth worm propagation speeds include spatial distributions of nodes, link duration distributions, degrees to which devices are mixed together, and even the burstiness of successive links.

In this paper, we present queue management algorithms with low implementation complexity that can partition the bandwidth of an outgoing link among flows in a high speed packet switch. These algorithms belong to a family of queue management schemes called sending rate estimate based queue management schemes (SREQM), which try to prevent congestion by effectively limiting flows based on their estimated sending rates. We also present techniques based on sampling and Bloom filters to further reduce the implementation overhead. The capability of the algorithms to protect responsive flows from non-responsive flows are confirmed by exhaustive analysis and simulations.

We propose a rigorous mathematical interpretation of a novel family of active queue management schemes, called sending rate estimate based queue management (SREQM) scheme, that aims to provide fair bandwidth allocation to all the flows in a router by estimating the flow sending rates, while maintaining only minimal per-flow state information. We propose an optimized implementation of SREQM, called fair sending rate estimate based queue management (FSREQM) scheme, and show through comparative simulation that FRESQM is the only scheme among those tested that successfully prevents bandwidth abuse while maintaining high link utilization.

We revisit the problem of computing the path with the minimum cost in terms of the expected number of link layer retransmissions in wireless mesh networks. Unlike previous efforts (such as the popular ETX) we account for the fact that link layer protocols (such as the IEEE 802.11 MAC) incorporate a non-zero but finite number of retransmission attempts per packet. A key observation that motivates this work is that the performance of a path depends not only on the number of links on the path and their qualities, but also on the relative positions of the links on the path. In particular, the closer a lossy link to the destination, the higher is its impact on the performance of that path. We design a new path metricthat captures all of the above factors and we call this metric ETOP. In this paper, we provide a synopsis of the analytical computation of ETOP. We also implement a routing strategy based on ETOP on a 25-node experimental testbed and provide sample results to showcase the performance with ETOP.

Highly penetrating cosmic ray muons constantly shower the earth at a rate of about 1 muon per cm2 per minute. We have developed a technique which exploits the multiple Coulomb scattering of these particles to perform nondestructive inspection without the use of artificial radiation. In prior work [1]-[3], we have described heuristic methods for processing muon data to create reconstructed images. In this paper, we present a maximum likelihood/expectation maximization tomographic reconstruction algorithm designed for the technique. This algorithm borrows much from techniques used in medical imaging, particularly emission tomography, but the statistics of muon scattering dictates differences. We describe the statistical model for multiple scattering, derive the reconstruction algorithm, and present simulated examples. We also propose methods to improve the robustness of the algorithm to experimental errors and events departing from the statistical model.

We present a technique for combining prior knowledge about transformations that should be ignored with a covariance matrix estimated from training data to make an improved Mahalanobis distance classifier. Modern classification problems often involve objects represented by high-dimensional vectors or images (for example, sampled speech or human faces). The complex statistical structure of these representations is often difficult to infer from the relatively limited training data sets that are available in practice. Thus, we wish to efficiently utilize any available a priori information, such as transformations or the representations with respect to which the associated objects are known to retain the same classification (for example, spatial shifts of an image of a handwritten digit do not alter the identity of the digit). These transformations, which are often relatively simple in the space of the underlying objects, are usually nonlinear in the space of the object representation, making their inclusion within the framework of a standard statistical classifier difficult. Motivated by prior work of Simard et al. (1998; 2000), we have constructed a new classifier which combines statistical information from training data and linear approximations to known invariance transformations. When tested on a face recognition task, performance was found to exceed by a significant margin that of the best algorithm in a reference software distribution.