We present an algorithm for finding temporally consistent occlusion boundaries in videos to support segmentation of dynamic scenes. We learn occlusion boundaries in a pair wise Markov random field (MRF) framework. We first estimate the probability of an spatio-temporal edge being an occlusion boundary by using appearance, flow, and geometric features. Next, we enforce occlusion boundary continuity in a MRF model by learning pair wise occlusion probabilities using a random forest. Then, we temporally smooth boundaries to remove temporal inconsistencies in occlusion boundary estimation. Our proposed framework provides an efficient approach for finding temporally consistent occlusion boundaries in video by utilizing causality, redundancy in videos, and semantic layout of the scene. We have developed a dataset with fully annotated ground-truth occlusion boundaries of over 30 videos (~5000 frames). This dataset is used to evaluate temporal occlusion boundaries and provides a much needed baseline for future studies. We perform experiments to demonstrate the role of scene layout, and temporal information for occlusion reasoning in dynamic scenes.

We present a novel algorithm for estimating the broad 3D geometric structure of outdoor video scenes. Leveraging spatio-temporal video segmentation, we decompose a dynamic scene captured by a video into geometric classes, based on predictions made by region-classifiers that are trained on appearance and motion features. By examining the homogeneity of the prediction, we combine predictions across multiple segmentation hierarchy levels alleviating the need to determine the granularity a priori. We built a novel, extensive dataset on geometric context of video to evaluate our method, consisting of over 100 ground-truth annotated outdoor videos with over 20,000 frames. To further scale beyond this dataset, we propose a semi-supervised learning framework to expand the pool of labeled data with high confidence predictions obtained from unlabeled data. Our system produces an accurate prediction of geometric context of video achieving 96% accuracy across main geometric classes.

In this paper, we present an improved automated system for classification of pathological image data of renal cell carcinoma. The task of analyzing tissue biopsies, generally performed manually by expert pathologists, is extremely challenging due to the variability in the tissue morphology, the preparation of tissue specimen, and the image acquisition process. Due to the complexity of this task and heterogeneity of patient tissue, this process suffers from inter-observer and intra-observer variability. In continuation of our previous work, which proposed a knowledge-based automated system, we observe that real life clinical biopsy images which contain necrotic regions and glands significantly degrade the classification process. Following the pathologist's technique of focusing on selected region of interest (ROI), we propose a simple ROI selection process which automatically rejects the glands and necrotic regions thereby improving the classification accuracy. We were able to improve the classification accuracy from 90% to 95% on a significantly heterogeneous image data set using our technique.

During the analysis of microscopy images, researchers locate regions of interest (ROI) and extract relevant information within it. Identifying the ROI is mostly done manually and subjectively by pathologists. Computer algorithms could help in reducing their workload and improve reproducibility. In particular, we want to assess the validity of the folic acid receptor as a biomarker for head and neck cancer. We are only interested in folic acid receptors appearing in cancerous tissue. Therefore, the first step is to segment images into cancerous and noncancerous regions. We propose to use a spiral intensity profile for segmentation of light microscopy images. Many algorithms identify objects in an image by considering pixel intensity and spatial information separately. Our algorithm integrates intensity and spatial information by considering the change, or profile, of pixel intensity in a spiral fashion. Using a spiral intensity profile can also perform segmentation at different scales from cancer regions to nuclei cluster to individual nuclei. We compared our algorithm with manually segmented image and obtained a specificity of 83.7% and sensitivity of 61.1%. Spiral intensity profiles can be used as a feature to improve other segmentation algorithms. Segmentation of cancerous images at different scales allows effective quantification of folic acid receptor inside cancerous regions, nuclei clusters, or individual cells.