The current study explores an artificial intelligence framework for measuring the structural features from microscopy images of the bacterial biofilms. Desulfovibrio alaskensis G20 (DA-G20) grown on mild steel surfaces is used as a model for sulfate reducing bacteria that are implicated in microbiologically influenced corrosion problems. Our goal is to automate the process of extracting the geometrical properties of the DA-G20 cells from the scanning electron microscopy (SEM) images, which is otherwise a laborious and costly process. These geometric properties are a biofilm phenotype that allow us to understand how the biofilm structurally adapts to the surface properties of the underlying metals, which can lead to better corrosion prevention solutions. We adapt two deep learning models: (a) a deep convolutional neural network (DCNN) model to achieve semantic segmentation of the cells, (d) a mask region-convolutional neural network (Mask R-CNN) model to achieve instance segmentation of the cells. These models are then integrated with moment invariants approach to measure the geometric characteristics of the segmented cells. Our numerical studies confirm that the Mask-RCNN and DCNN methods are 227x and 70x faster respectively, compared to the traditional method of manual identification and measurement of the cell geometric properties by the domain experts.

Scanning electron microscopy (SEM) images play a crucial role in the quantitative analyses of biofilms on materials by providing detailed information about biofilm formation, ultrastructure, cells and their interaction with materials. In addition to some intrinsic limitations of SEM imaging, some of the characteristics of images in SEM volumes often vary in terms of the wide range of magnifications, high resolutions, depth of the field, and the SEM protocols. Quantitative characterization of biofilm morphologies such as the cell size, geometry, and density from these SEM volumes is a challenging problem. This paper presents deep learning based super-resolution (DLSR) as a step towards addressing this problem. Three DLSR approaches based on generative adversarial learning techniques are applied to an SEM biofilm dataset and compared in terms of their a) preservation of the morphological features, and b) their impact on the performance of a deep learning based image segmentation task. Our results show that the DLSR approaches vary in preservation of morphological features. We also show that DLSR can considerably improve the performance of image segmentation outputs of deep learning networks and hence be incorporated in any deep learning pipeline used for quantitative analyses of biofilms based on SEM images.

Accurate detection and segmentation of bacterial cells in the microscopy images of a biofilm is essential to develop technologies to resist microbial corrosion. The traditional approach of manually identifying cell regions in microscopy images is a time-consuming and error-prone task. Nonetheless, many of the existing approaches, including automated systems adopting advanced machine learning models, find it challenging to detect and segment cell instances in clustered biofilms where cells are overlapping and touching each other. In this paper, we develop a method to segment and extract the size properties of all cells. The proposed method consists of two stages, a semantic segmentation stage based on a U-Net architecture followed by a region-based ellipse fitting technique for instance segmentation and size property extraction. We compared the performance of our approach against a widely used object segmentation approach namely Mask R-CNN and found that our algorithm outperformed Mask R-CNN in terms of the segmentation efficiency and cell size estimation for images of Bacillus subtilis biofilms.

Detection of microbially influenced corrosion (MIC) products in biofilm images is an important problem in material science and engineering. Neural network models that can accurately detect biofilm images having corrosion products are likely to have significant and broad implications in the study and development of materials. However, generating and annotating biofilm image datasets in sufficient volumes is a major impediment in developing such networks. A self-supervised learning approach is presented for automatically detecting MIC products on metal surfaces based on analyses of Scanning Electron Microscope (SEM) biofilm images. The proposed approach uses a Simple Siamese (SimSiam) architecture to learn visual image representations from an unlabeled set of biofilm images, which is then fine-tuned using a scarce set of labeled images to build a model to detect biofilms containing corrosion products. The architecture generates two different augmented versions of the input images and learns the representations using an encoder that uses ResNet backbone. The architecture aims to minimize the negative cosine similarity of the outputs from the encoders and hence learns the representations of the images as both augmented versions belong to the same image. In order to improve the dataset quality and volume, input images are contrast enhanced, scaled, and overlapping image patches are generated and used to learn representations and fine-tuning. The performance of the models are analyzed using precision and recall metrics for patches of varying sizes. An overall accuracy of 62% was obtained for the classification of corrosion in the images after finetuning the model with scarce labeled dataset. Our results show that the models built using the proposed self-supervised learning approach can successfully detect corrosion products in biofilm images and that the performance of the models successively improves with increase in the patch size.

The proliferation of information technology infrastructure in recent decades has allowed for unprecedented ease of access to centrally-aggregated scholarly literature and scientific knowledge. This massive aggregation of knowledge requires an information retrieval infrastructure, to include formalized ontologies, that is engineered with careful consideration. A number of domains benefit from the use of hierarchical controlled vocabularies, which may be used to provide a rich set of descriptive terms for characterizing entities in a consistent manner. There are clear benefits to the creation and maintenance of these ontologies: search and retrieval is made easier and analyses of the contained entities are enabled that would not otherwise be possible. However, there may be the opportunity to decrease the manual burden of ontology creation and maintenance with automated methods that leverage natural language processing and other computational techniques. This work presents an automated ontology creation methodology, adapted and expanded from prior work [1], that can produce a topic hierarchy from natural language and may be used to assist in the creation of a novel ontology or the expansion of existing ontologies. The effectiveness of the proposed method is studied using two examples: immunology, an established biomedical domain and a prominent topic in MeSH, and graphene, from the 2D materials domain with wide-ranging biomedical applications, which also has a sparse presence in MeSH

Medical image segmentation is a challenging problem for computer vision approaches where deep learning networks have achieved impressive successes in recent years. In this paper, we propose a directed, fine tuning approach for instance segmentation networks by using feature clustering of predictions along with labeled training instances to improve network performance. The approach directs and limits analyses of predicted instances by experts to similar training instances only and reduces manual overheads by managing the number of instances that need to be examined. Sub-optimal network predictions are handled either by retraining the networks on data augmented with the relevant training instances, correcting training labels, and/or by readjusting network inference parameters. We first develop a state-of-the-art Mask R-CNN based network for instance segmentation of fundus images with retinal lesions and scars caused by Ocular Toxoplasmosis. Then, we show how the proposed approach can be applied to fine tune this network in a directed manner using feature clustering using a pre-trained CNN network. We demonstrate the robustness of our proposed approach with the evaluation results - mask average IoU increased by 7% and mAP under 0.5 IoU threshold increased by 20%. Our experiments also show that fine tuning by analyzing 66% of the predicted instances achieves the same improvement as that obtained by all of the predicted instances, a significant reduction of the manual overheads for fine tuning.

Recent advances in semantic segmentation using deep learning methods have achieved promising results on several benchmark datasets. However, the primary challenge involved in such segmentation approaches is the availability of applicable training data. Since only experts are equipped to effectively annotate (or label) any available data for training semantic segmentation networks, the effort and cost involved can be considerable, especially for larger datasets. In this paper, we aim to address this problem by proposing a Thrifty Annotation Generation approach that records high performance on segmentation networks with minimal expert effort and cost (intervention). We present a deep active learning framework that combines the use of marker-controlled watershed (MC-WS) algorithm to generate pseudo labels for segmentation networks (U-Net) and active learning to significantly minimize effort and cost by selecting only the most impactful training data for labeling. We built the initial U-Net model by generating pseudo labels for the training data using MC-WS. We then make use of the uncertainty information (entropy) of each image provided by the U-Net to determine the most uncertain or effective images for expert labeling. We evaluated the TAG approach using the 2012 ISBI Challenge dataset for 2D segmentation and a novel Biofilm dataset. Our approach achieved promising segmentation accuracy (IoU) and classification accuracy with minimal expert intervention. The results of our experiments also indicate that the TAG approach can be generalized to achieve high-performance segmentation results on any dataset using minimal expert effort and cost.

Ocular Toxoplasmosis (OT) is a widespread infectious chorioretinal disease whose timely diagnosis and treatment are crucial to prevent potential vision loss. Diagnosing OT is a challenging task ranging from tedious analyses of fundus images of the eye to serological clinical tests. An automated approach using convolutional neural networks (CNNs) towards diagnosing OT by analyzing fundus images is described. Fundus images are segmented to patches using a sliding window and are classified into healthy and unhealthy fundus image patches using a CNN model. An OT lesion heat map of a fundus image is generated from these patches. The heat map and patch features are then combined to develop a dual input hybrid CNN model detecting OT fundus images with high accuracy. The approach was applied to a dataset of fundus images involving OT and normal subjects and was highly effective in identifying fundus images having OT lesions.

Integrating of subject matter onto serious games is an important problem that has been shown to impact the learning potential of serious games. A novel approach, inspired by peer-to-peer (P2P) networks, towards designing and deploying a series of problems in game scenarios is described. Given a set of problems involving a set of concepts the proposed approach automatically generates a problem network graph akin to P2P network that can then traversed by a player to collect all the concepts that are necessary to learn a topic of interest. A network traversal algorithm is described, which identifies the relevant problems and produces an efficient route through the network for learning the topic. We also describe an algorithm for mapping the problem network graph onto a game scenario by identifying groups of problems that can be placed in a single location of the game like the level of a building, arcade, or a room, physical barriers that separate, and the conditions for passing through these barriers. The proposed approach has been validated through a quantum cryptography game QuaSim and has been played by over 100 students to learn quantum cryptography basics and cryptography protocols.

Visual distortions play a crucial role in early diagnosis and timely treatment of several eye diseases such as the age-related macular degeneration (AMD). A framework to collect quantitative information about visual distortions, map them to the regions of retina where they may originate, and correlate them to retinal health information obtained using clinical tests, is described in this paper. The resulting composite retinal map can enable physicians to diagnose and treat AMD with minimal manual effort. Our results using the system in practice enabled physicians to study the correlation of distortions to retina lesions with minimal manual overheads.

Spatial pyramid matching (SPM) has achieved impressive successes in analyzing and classifying images across several domains. SPM computes a similarity measure over images by using bag of words similarity score over different levels of coarseness of the images. In this paper we propose a novel, simple approach based on SPM, differential SPM (DSPM) that incorporates finer differences among images while determining image similarity. The approach propagates the differences seen at fine levels to dampen the similarity observed at the coarser levels, thereby highlighting differences among images at small, localized regions. The resulting similarity scores among images can better separate images that match at coarse levels, but have subtle differences. DSPM integrated with K-nearest neighbor classification approaches was used to identify and analyze retinal Optical Coherence Tomography (OCT) images containing normal retinal scans as well as those from subjects with AMD (age-related macular degeneration) and DME (diabetic macular edema). The proposed approach achieved higher classification accuracy with smaller training overheads in comparison to SPM in all cases in our experiments.

Retrieving relevant test cases is a recurring theme in software validation. We present an approach for cluster-based retrieval of test cases for software validation. The approach uses a probabilistic notion of coverage among line-based test profiles and can potentially discover groups of test cases executing a small number of unique lines. The distribution of lines across test profiles are analyzed to determine the number of clusters and generate a clustering structure without any additional user input. We also propose a novel and simple approach to identify test cases that are affected by software changes based on test profiles. It is shown that the clustering structures generated can be used to select affected tests economically to produce high quality regression test suites. The approach is applied to four unix utility programs from a popular testing benchmark. Our results show that the generated number of clusters and their average sizes closely track their estimates based on test profiles. The retrieval of affected tests using the clustering structure is economical and produces a good quality regression test suite.

Samples from clusters of tests are used to automatically predict if tests are to be re-clustered for a modified program. Tests likely to be affected by line level changes are included in the samples by analyzing their execution profiles based on spatial locality. Initial experiments show that the approach can potentially avoid re-clustering in many cases.

Identifying time periods with a burst of activity related to a topic has been an important problem in analyzing time stamped documents. In this paper, we discuss methods to compute a hot spot of a given topic from a time stamped document set. We consider basic topics that contain one or more keywords as well as complex topics that contain topics connected by logical operators and, or, not. We use the temporal scan statistic to assign a discrepancy score to each of the intervals of the time period spanning the given document set. The hot spot of the given topic is the time interval with the highest discrepancy score. We describe efficient algorithms to compute the hot spots of both basic and complex topics. Our preliminary experiments using the SIGMOD/VLDB paper titles data set and the CNN/Reuters news article titles data set collected from the TDT-Pilot Corpus show that our methods to compute the measure and the hot spot of a topic work very well in practice.

Temporal segmentation partitions time series data with the intent of producing more homogeneous segments. It is a technique used to preprocess data so that subsequent time series analysis on individual segments can detect trends that may not be evident when performing time series analysis on the entire dataset.

Abstract Time series segmentation is a promising approach to discover temporal patterns from time‐stamped numeric data. A novel approach to apply time series segmentation to discern temporal information from software version repositories is proposed. Data from such repositories, both numeric and non‐numeric, are represented as item‐set time series data. A dynamic programming algorithm for optimal segmentation is presented. The algorithm automatically produces a compacted item‐set time series that can be analyzed to identify temporal patterns. The effectiveness of the approach is illustrated by analyzing version control repositories of several open‐source projects to identify time‐varying patterns of developer activity. The experimental results show that the segmentation algorithm produces segments that capture meaningful information and is superior to the information content obtained by arbitrarily segmenting software history into regular time intervals. Copyright © 2008 John Wiley & Sons, Ltd.

Time series analysis is a promising approach to discover temporal patterns from time stamped, numeric data. A novel approach to apply time series analysis to discern temporal information from software version repositories is proposed. Version logs containing numeric as well as non-numeric data are represented as an item-set time series. A dynamic programming based algorithm to optimally segment an item-set time series is presented. The algorithm automatically produces a compacted item-set time series that can be analyzed to discern temporal patterns. The effectiveness of the approach is illustrated by applying to the Mozilla data set to study the change frequency and developer activity profiles. The experimental results show that the segmentation algorithm produces segments that capture meaningful information and is superior to the information content obtaining by arbitrarily segmenting time period into regular time intervals.

Constructing time decompositions of time stamped documents is an important step for uncovering temporal relationships and trends of keywords and topics contained in the document set. This paper describes the use of time decompositions to extract temporal information from a small set of PubMed abstracts related to the Wnt signaling pathway. A time decomposition of the document set is constructed to identify temporal information such as keywords/topics significant in some time interval and to also identify temporal progression of the significant keywords. Keywords were assigned temporal significance values using two different measure functions based on notions of entropy and ratio. It is shown how optimal lossy decompositions of the document set are effective in reducing noise both in terms of the number of keywords as well as in terms of smoothing out the temporal progressions of keywords. Several optimal lossy decompositions for the document set are constructed and it is shown that the temporal information captured by an optimal lossy decomposition increases as its size (number of intervals) increases

Constructing time decompositions of time stamped documents is an important first step in extracting temporal information from a document set. Efficient algorithms are described for computing optimal lossy decompositions for a given document set, where the loss of information is constrained to be within a specified bound. A novel and efficient algorithm is proposed for computing information loss values required to construct optimal lossy decompositions. Experimental results are reported comparing optimal lossy decompositions and equal length decompositions in terms of a number of parameters such as information loss. In particular, our results show that optimal lossy decompositions outperform equal length decompositions by preserving more of the information content of the underlying document set. The results also demonstrate that permitting even small amounts of variability in the length of the subintervals of a decomposition results in capturing more of the temporal information content of a document set when compared to equal length decompositions. This paper builds upon our earlier work on time decompositions where the problem of computing optimal lossy decomposition of the time period associated with a document set was first formulated.

Extraction of sequences of events from news and other documents based on the publication times of these documents has been shown to be extremely effective in tracking past events. This paper addresses the issue of constructing an optimal decomposition of the time period associated with a given document set, i.e., a decomposition with the smallest number of subintervals, subject to no or limited loss of information. We introduce the notion of the compressed interval decomposition, where each subinterval consists of consecutive time points having identical information content. We define optimality, and show that any optimal information preserving decomposition of the time period is a refinement of the compressed interval decomposition. We define several special classes of measure functions (functions that compute the significant information from document sets), based on their effect on the information computed as document sets are combined. These classes are used in developing algorithms for computing an optimal information preserving decomposition of the time period of a given document set. We also define the notion of information loss of a time decomposition of a given document set and give an efficient algorithm for computing an optimal lossy decomposition. We discuss the effectiveness of our algorithms on the Reuters-21578, Distribution 1.0 data set and a subset of Medline abstracts.

We claim that a dynamic agent infrastructure can provide a shift from static distributed computing to dynamic distributed computing, and we have developed an infrastructure to realize such a shift. We shall compare this infrastructure with other distributed computing infrastructures such as CORBA and DCOM, and demonstrate its value in highly dynamic system integration, service provisioning and distributed applications such as data mining on the Web. The infrastructure is Java-based, light-weight, and extensible. It differs from other agent platforms and client/server infrastructures in its support of dynamic behavior modification of agents. A dynamic agent is not designed to have a fixed set of predefined functions, but instead, to carry application-specific actions, which can be loaded and modified on the fly. This allows a dynamic agent to adjust its capability to accommodate changes in the environment and requirements, and play different roles across multiple applications. The above features are supported by the light-weight, built-in management facilities of dynamic agents, which can be commonly used by the "carried" application programs to communicate, manage resources and modify their problem-solving capabilities. Therefore, the proposed infrastructure allows application-specific multi-agent systems to be developed easily on top of it, provides "nuts and bolts" for run-time system integration, and supports dynamic service construction, modification and movement. A prototype has been developed at HP Labs and made available to several external research groups.

We claim that a dynamic-agent infrastructure can provide a shift from static distributed computing to dynamic distributed computing, and we have developed such an infrastructure to realize such a shift. We shall show its impact on software engineering through a comparison with other distributed object-oriented systems such as CORBA and DCOM, and demonstrate its value in highly dynamic system integration and service provisioning. The infrastructure is Java-based, light-weight, and extensible. It differs from other agent platforms and client/server infrastructures in its support of dynamic behavior modification of agents. A dynamic-agent is not designed to have a fixed set of predefined functions but instead, to carry application-specific actions, which can be loaded and modified on theory. This allows a dynamic-agent to adjust its capability for accommodating environment and requirement changes, and play different roles across multiple applications. The above features are supported by the light-weight, built-in management facilities of dynamic-agents, which can be commonly used by the "carried" application programs to communicate, manage resources and modify their problem solving capabilities. Therefore, the proposed infrastructure allows application-specific multi-agent systems to be developed easily on top of it, provides "nuts and bolts" for run-time system integration, and supports dynamic service construction, modification and movement. A prototype has been developed at HP Labs and made available to several external research groups.

Commercial distributed database systems generally support an optional protocol that provides loose consistency of replicas, allowing replicas to be inconsistent for some time. In such a protocol, each replicated data item is assigned a primary copy site. Typically, a transaction updates only the primary copies of data items, with updates to other copies deferred until after the transaction commits. After a transaction commits, its updates to primary copies are sent transactionally to the other sites containing secondary copies. We investigate the transaction model underlying the above protocol. We show that global serializability in such a system is a property of the placement of primary and secondary copies of replicated data items. We present a polynomial time algorithm to assign primary sites to data items so that the resulting topology ensures serializability.

No abstract available.