Biological samples are routinely analyzed for microbe concentration. The samples are diluted, loaded onto established host cell cultures, and incubated. If infectious agents are present in the samples, they form circular spots that do not contain the host cells. Each spot is assumed to be originated from a single microbial unit such as a bacterial colony forming unit or viral plaque forming unit. The undiluted sample concentration is estimated by counting the spots and back-calculating. Counting the number of spots by trained technicians is currently the gold standard but it is laborious, subjective, and hard to scale. This paper presents a new automated algorithm for spot counting, Localized and Sequential Thresholding (LoST). Validation studies showed that LoST performance was comparable with manual counting and outperformed several existing tools on images with overlapping spots. The LoST algorithm employs sequential thresholding through a two-stage segmentation and borrows information across all images from the same dilution series to fine-tune the count and identify right censoring. The algorithm increases the efficiency of the spot counting and the quality of the downstream analysis, especially when coupled with an appropriate statistical serial dilution model to enhance the undiluted sample concentration estimation procedure.

Ensemble methods such as random forest works well on high-dimensional datasets. However, when the number of features is extremely large compared to the number of samples and the percentage of truly informative feature is very small, performance of traditional random forest decline significantly. To this end, we develop a novel approach that enhance the performance of traditional random forest by reducing the contribution of trees whose nodes are populated with less informative features. The proposed method selects eligible subsets at each node by weighted random sampling as opposed to simple random sampling in traditional random forest. We refer to this modified random forest algorithm as "Enriched Random Forest". Using several high-dimensional micro-array datasets, we evaluate the performance of our approach in both regression and classification settings. In addition, we also demonstrate the effectiveness of balanced leave-one-out cross-validation to reduce computational load and decrease sample size while computing feature weights. Overall, the results indicate that enriched random forest improves the prediction accuracy of traditional random forest, especially when relevant features are very few.

Comorbidity greatly increases the complexity of managing disease in patients. Approximately 27 percent of the US population have two or more concurrent comorbid conditions. Traditional models for assessing the impact of patient demographic and comorbidity burden on patient health outcomes, represented comorbidity conditions by Charlson Comorbidity Index. In this paper, we develop a novel two-way clustering approach combining model-based and weighted K-means clustering methods for characterizing and summarizing a patient's comorbid conditions. Our two-way approach helps reduce the size of the data to a manageable size, thus being practical for big data applications. Another novel aspect of our approach is the ability to handle weighted observations. Assigning weights to observations helps reduce the size of the dataset, thus addressing the scalability challenge of algorithms when dealing with big data. Using the National Inpatient Sample database for 2008-2013, we evaluate the performance of our approach by the use of logistic regression and support vector machine models by applying them to patients whose primary diagnosis is cardiovascular disease. In addition to evaluating our proposed method using empirical test data, we use asymptotic statistics. Both evaluation methods show that the proposed approach improves the prediction of patient health outcomes; specifically, hospital length of stay.

Forecasting short-term traffic flow has been a critical topic in transportation research for decades, which aims to facilitate dynamic traffic control proactively by monitoring the present traffic and foreseeing its immediate future. In this paper, we focus on forecasting short-term passenger flow at subway stations by utilizing the data collected through an automatic fare collection (AFC) system along with various external factors, where passenger flow refers to the volume of arrivals at stations during a given period of time. Along this line, we propose a data-driven three-stage framework for short-term passenger flow forecasting, consisting of traffic data profiling, feature extraction, and predictive modeling. We investigate the effect of temporal and spatial features as well as external weather influence on passenger flow forecasting. Various forecasting models, including the time series model auto-regressive integrated moving average, linear regression, and support vector regression, are employed for evaluating the performance of the proposed framework. Moreover, using a real data set collected from the Shenzhen AFC system, we conduct extensive experiments for methods validation, feature evaluation, and data resolution demonstration.

Although the random forest classification procedure works well in datasets with many features, when the number of features is huge and the percentage of truly informative features is small, such as with DNA microarray data, its performance tends to decline significantly. In such instances, the procedure can be improved by reducing the contribution of trees whose nodes are populated by non-informative features. To some extent, this can be achieved by prefiltering, but we propose a novel, yet simple, adjustment that has demonstrably superior performance: choose the eligible subsets at each node by weighted random sampling instead of simple random sampling, with the weights tilted in favor of the informative features. This results in an 'enriched random forest'. We illustrate the superior performance of this procedure in several actual microarray datasets.

A new performance evaluation paradigm for computer vision systems is proposed. In real situation, the complexity of the input data and/or of the computational procedure can make traditional error propagation methods infeasible. The new approach exploits a resampling technique recently introduced in statistics, the bootstrap. Distributions for the output variables are obtained by perturbing the nuisance properties of the input, i.e., properties with no relevance for the output under ideal conditions. From these bootstrap distributions, the confidence in the adequacy of the assumptions embedded into the computational procedure for the given input is derived. As an example, the new paradigm is applied to the task of edge detection. The performance of several edge detection methods is compared both for synthetic data and real images. The confidence in the output can be used to obtain an edgemap independent of the gradient magnitude.

A general method for eliminating the bias of nonlinear estimators using bootstrap is presented. Instead of the traditional mean bias we consider the definition of bias based on the median. The method is applied to the problem of fitting ellipse segments to noisy data. No assumption beyond being independent identically distributed is made about the error distribution and experiments with both synthetic and real data prove the effectiveness of the technique.