Clinical features found in brain CT scan images are widely used in traumatic brain injury (TBI) as indicators for Glasgow Outcome Scale (GOS) prediction. However, due to the lack of automated methods to measure and quantify the CT scan image features, the computerized prediction of GOS in TBI has not been well studied. This paper introduces an automated GOS prediction system for traumatic brain CT images. Different from most existing systems that perform the prognosis based on pre-processed data, our system directly works on brain CT scan images based on the image features. Our system can also be extended to large dataset with easy adaptation. For each new image of a CT scan series, our proposed system first makes use of sparse representation model that predicts the GOS of each CT image slice using Gabor features. Logistic regression, which integrates the GOS of each CT scan slice with a pre-trained model, is then applied to estimate the GOS score for the new case which contains multiple CT slices. Evaluation of the system has shown promising results in prediction of GOS of traumatic brain injury cases.

Computer aided diagnosis (CAD) in medical imaging is of growing interest in recent years. Our proposed CAD system aims to enhance diagnosis and prognosis of traumatic brain injury (TBI) patients with hematomas. Hematoma caused by blood vessel rupture is the major lesion in TBI cases and is usually assessed using head computed tomography (CT). In our CAD system, we segment the hematoma region from each slice of a CT series, extract features from the hematoma segments, and automatically classify the hematoma types using machine learning methods. We propose two sets of shape based features for each segmented hematoma region. The first set contains primitive features describing the overall shape of a hematoma region. The features in the second set are based on the dissimilarities of the shapes of hematoma regions measured by geodesic distances. After feature extraction, we classify the hematoma regions into three types -- epidural hematoma, sub-dural hematoma, and intracerebral hematoma, using random forest. Each tree of the random forest votes one class for each hematoma, and the random forest takes the class label with the majority votes for the hematoma. As hematomas are volumetric in nature, some hematomas are observed across several consecutive slices in the same CT series. For each class, we add the votes from each hematoma slice that comprises the volumetric hematoma in that class, then we take the class with the majority of the summed votes as the class label for that volumetric hematoma. The overall classification accuracies for hematoma region from each CT slice are 80.7%, 81.3%, and 81.1% using primitive features only, geodesic distance features only, or both sets of features, respectively. For volumetric hematoma classification, the overall accuracies are 80.9%, 81.5%, and 81.5% respectively. The results are promising to radiologists and neurosurgeons specialized in this field of research.

Automatic medical image classification is difficult because of the lacking of training data. As manual labeling is too costly, we provide an automatic labeling solution to this problem by making use of the radiology report associated with the medical images. We first segment and reconstruct the 3D regions of interest (ROIs) from the medical images, and extract pathology and anatomy information from the associated report. We use an anatomical atlas to map the ROIs to the anatomy part(s) and match the pathology information of the same anatomy part(s) from the text. In this way, the ROIs are automatically labeled with pathology types which can be served as class labels, and a training data set of a large number of training instances is generated automatically. We extract the volume, color, location, and shape features of the ROIs, and classify the types of ROIs using these features. The overall evaluation result is promising to doctors and medical professionals. Our experiment is conducted using traumatic brain injury CT images; however, our framework of automatically labeling and classifying medical cases can be extended to medical images in other modality or of other anatomical part.

In this paper, we propose a novel post processing approach for on-line handwriting recognition. Differing from the existing linguistic knowledge-based methods, we make use of domain specific knowledge to improve the performance of recognition. Our system recognizes doctor's handwriting which often poses great challenges in readability, and then enhances the quality of recognized text by analyzing and restoring the text with a medical knowledge model. We show experiments with this approach on a set of medical handwriting data provided by the doctor, and the results are promising.

Large number of medical images are produced daily in hospitals and medical institutions, the needs to efficiently process, index, search and retrieve these images are great. In this paper, we propose a pathology based medical image annotation framework using a statistical machine translation approach. After pathology terms and regions of interest (ROIs) are extracted from training text and images respectively, we use machine translation model IBM Model 1 to iteratively learn the alignment between the ROIs and the pathology terms and generate an ROI-to-pathology translation table. In testing phase, we annotate the ROI in the image with the pathology label of the highest probability in the translation table. The overall annotation results and the retrieval performance are promising to doctors and medical professionals.

In recent years, with the rapid proliferation of digital images, the need to search and retrieve the images accurately, efficiently, and conveniently is becoming more acute. Automatic image annotation with image semantic content has attracted increasing attention, as it is the preprocess of annotation based image retrieval which provides users accurate, efficient, and convenient image retrieval with image understanding. Different machine learning approaches have been used to tackle the problem of automatic image annotation; however, most of them focused on exploring the relationship between images and annotation words and neglected the relationship among the annotation words. In this paper, we propose a framework of using language models to represent the word-to-word relation and thus to improve the performance of existing image annotation approaches utilizing probabilistic models. We also propose a specific language model - the semantic similarity language model to estimate the semantic similarity among the annotation words so that annotations that are more semantically coherent will have higher probability to be chosen to annotate the image. To illustrate the general idea of using language model to improve current image annotation systems, we added the language model on top of the two specific image annotation models - the translation model (TM) and the cross media relevance model (CMRM). We tested the improved models on a widely used image annotation corpus - the Corel 5K dataset. Our results show that by adding the semantic similarity language model, the performance of image annotation improves significantly in comparison with the original models. Our proposed language model can also be applied to other image annotation approaches using word probability conditioned on image or word-image joint probability as well.

Medical text mining has gained increasing interest in recent years. Radiology reports contain rich information describing radiologistpsilas observations on the patientpsilas medical conditions in the associated medical images. However, as most reports are in free text format, the valuable information contained in those reports cannot be easily accessed and used, unless proper text mining has been applied. In this paper, we propose a text mining system to extract and use the information in radiology reports. The system consists of three main modules: a medical finding extractor, a report and image retriever, and a text-assisted image feature extractor. In evaluation, the overall precision and recall for medical finding extraction are 95.5% and 87.9% respectively, and for all modifiers of the medical findings 88.2% and 82.8% respectively. The overall result of report and image retrieval module and text-assisted image feature extraction module is satisfactory to radiologists.