In the context of deep learning research, where model introductions continually occur, the need for effective and efficient evaluation remains paramount. Existing methods often emphasize accuracy metrics, overlooking stability. To address this, the paper introduces the Accuracy-Stability Index (ASI), a quantitative measure incorporating both accuracy and stability for assessing deep learning models. Experimental results demonstrate the application of ASI, and a 3D surface model is presented for visualizing ASI, mean accuracy, and coefficient of variation. This paper addresses the important issue of quantitative benchmarking metrics for deep learning models, providing a new approach for accurately evaluating accuracy and stability of deep learning models. The paper concludes with discussions on potential weaknesses and outlines future research directions.

Image quality is important, and can affect overall performance in image processing and computer vision as well as for numerous other reasons. Image quality assessment (IQA) is consequently a vital task in different applications from aerial photography interpretation to object detection to medical image analysis. In previous research, the BRISQUE algorithm and the PSNR algorithm were evaluated with high resolution (≥ 512×384 pixels), but relatively small image sets (≤4,744 images). However, scientists have not evaluated IQA algorithms on low resolution (≤32×32 pixels), multi-perturbation, big image sets (for example, ≥60,000 different images not counting their perturbations). This study explores these two IQA algorithms through experimental investigation. We first chose two deep learning image sets, CIFAR-10 and MNIST. Then, we added 68 perturbations that add noise to the images in specific sequences and noise intensities. In addition, we tracked the performance outputs of the two IQA algorithms with singly and multiply noised images. After quantitatively analyzing experimental results, we report the limitations of the two IQAs with these noised CIFAR-10 and MNIST image sets. We also explain three potential root causes for performance degradation. These findings point out weaknesses of the two IQA algorithms. The research results provide guidance to scientists and engineers developing accurate, robust IQA algorithms. All source codes, related image sets, and figures are shared on the website (https://github.com/caperock/imagequality) to support future scientific and industrial projects.

Deep learning (DL) classifiers are often unstable in that they may change significantly when retested on perturbed images or low quality images. This paper adds to the fundamental body of work on the robustness of DL classifiers. We introduce a new two-dimensional benchmarking matrix to evaluate robustness of DL classifiers, and we also innovate a four-quadrant statistical visualization tool, including minimum accuracy, maximum accuracy, mean accuracy, and coefficient of variation, for benchmarking robustness of DL classifiers. To measure robust DL classifiers, we create comprehensive 69 benchmarking image sets, including a clean set, sets with single factor perturbations, and sets with two-factor perturbation conditions. After collecting experimental results, we first report that using two-factor perturbed images improves both robustness and accuracy of DL classifiers. The two-factor perturbation includes (1) two digital perturbations (salt & pepper noise and Gaussian noise) applied in both sequences, and (2) one digital perturbation (salt & pepper noise) and a geometric perturbation (rotation) applied in both sequences. All source codes, related image sets, and results are shared on the GitHub website at https://github.com/caperock/robustai to support future academic research and industry projects.

BACKGROUND: We describe a method for extracting data about how biomolecule pairs interact from texts. This method relies on empirically determined characteristics of sentences. The characteristics are efficient to compute, making this approach to extraction of biomolecular interactions scalable. The results of such interaction mining can support interaction network annotation, question answering, database construction, and other applications. RESULTS: We constructed a software system to search MEDLINE for sentences likely to describe interactions between given biomolecules. The system extracts a list of the interaction-indicating terms appearing in those sentences, then ranks those terms based on their likelihood of correctly characterizing how the biomolecules interact. The ranking process uses a tf-idf (term frequency-inverse document frequency) based technique using empirically derived knowledge about sentences, and was applied to the MEDLINE literature collection. Software was developed as part of the MetNet toolkit (http://www.metnetdb.org). CONCLUSIONS: Specific, efficiently computable characteristics of sentences about biomolecular interactions were analyzed to better understand how to use these characteristics to extract how biomolecules interact.The text empirics method that was investigated, though arising from a classical tradition, has yet to be fully explored for the task of extracting biomolecular interactions from the literature. The conclusions we reach about the sentence characteristics investigated in this work, as well as the technique itself, could be used by other systems to provide evidence about putative interactions, thus supporting efforts to maximize the ability of hybrid systems to support such tasks as annotating and constructing interaction networks.

BACKGROUND: Analyzing global experimental data can be tedious and time-consuming. Thus, helping biologists see results as quickly and easily as possible can facilitate biological research, and is the purpose of the software we describe. RESULTS: We present BirdsEyeView, a software system for visualizing experimental transcriptomic data using different views that users can switch among and compare. BirdsEyeView graphically maps data to three views: Cellular Map (currently a plant cell), Pathway Tree with dynamic mapping, and Gene Ontology http://www.geneontology.org Biological Processes and Molecular Functions. By displaying color-coded values for transcript levels across different views, BirdsEyeView can assist users in developing hypotheses about their experiment results. CONCLUSIONS: BirdsEyeView is a software system available as a Java Webstart package for visualizing transcriptomic data in the context of different biological views to assist biologists in investigating experimental results. BirdsEyeView can be obtained from http://metnetdb.org/MetNet_BirdsEyeView.htm.

The seventh annual Midsouth Computational Biology and Bioinformatics (MCBIOS) conference took place February 19 and 20, 2010 at Arkansas State University in Jonesboro, AR, presided over by Daniel Berleant, this year’s President of MCBIOS. Keynote speakers were Elaine Ostrander of NIH, renowned for her work on dog genomics, Clayton Naeve, CIO of St. Jude Children's Research Hospital, and Robert Cottingham, Leader of the Computational Biology and Bioinformatics Group, Oak Ridge National Laboratory. The record-breaking attendance exceeded 200 participants which necessitated parallel talk sessions for the first time. Student oral presentation award winners were Heidi Pagan (first place) of Mississippi State University (MSU), Juliet Tang (second place) of MSU, and Aleksandra Markovets (third) of Mississippi Valley State University. In addition, a record number of posters were also presented. Due to the number of posters, student poster awards were given in two categories, Biological Focus and Computational Focus. Winners were N. Platt and V. Chaitankar (1st place), G. Cooper and L. Pillai (2nd), and M. Ammari and C. Gearheart (3rd). Reflecting the integration of these foci in the field, winners in one category frequently scored high in the other category as well. MCBIOS is also pleased to have this year acquired legal status as a non-profit organization.

MCBIOS 2009 was held February 20–21, 2009 at the Hunter Henry Center in Starkville, Mississippi on the Mississippi State University campus. The conference was hosted by the four research universities in Mississippi comprising the Mississippi Computational Biology Consortium – Jackson State University (JSU), Mississippi State University (MSU), the University of Mississippi (UM), and the University of Southern Mississippi (USM). Dr. Dawn Wilkins of UM and Dr. Susan Bridges of MSU served as conference co-chairs. Keynote speakers were Dr. Laura Elnitski, Head of the Genomic Functional Analysis Section at the National Human Genome Research Institute (NHGRI), Howard Cash, President and CEO of Gene Codes Corporation, and Dr. Cathy Wu, Director of the Protein Information Resource (PIR).

MOTIVATION: The increasingly large amount of free, online biological text makes automatic interaction extraction correspondingly attractive. Machine learning is one strategy that works by uncovering and using useful properties that are implicit in the text. However these properties are usually not reported in the literature explicitly. By investigating specific properties of biological text passages in this paper, we aim to facilitate an alternative strategy, the use of text empirics, to support mining of biomedical texts for biomolecular interactions. We report on our application of this approach, and also report some empirical findings about an important class of passages. These may be useful to others who may also wish to use the empirical properties we describe. RESULTS: We manually analyzed syntactic and semantic properties of sentences likely to describe interactions between biomolecules. The resulting empirical data were used to design an algorithm for the PathBinder system to extract biomolecular interactions from texts. PathBinder searches PubMed for sentences describing interactions between two given biomolecules. PathBinder then uses probabilistic methods to combine evidence from multiple relevant sentences in PubMed to assess the relative likelihood of interaction between two arbitrary biomolecules. A biomolecular interaction network was constructed based on those likelihoods. CONCLUSION: The text empirics approach used here supports computationally friendly, performance competitive, automatic extraction of biomolecular interactions from texts. AVAILABILITY: http://www.metnetdb.org/pathbinder.

The field of robotics is on a growth curve, with most of the growth expected in the areas of personal and service robots. As robots become more prevalent in chaotic home and industrial settings, they will be required to make increasingly independent decisions about how to accomplish their tasks. A key to accomplishing this is the development of techniques to allow robots to handle severe uncertainty. This paper introduces the use of information gap theory as a way to enable robots to make robust decisions in the face of uncertainty, and illustrates this with an example problem.

Abstract Summary: MEDLINE is one of the most important bibliographical information sources for biologists and medical workers. Its PubMed interface supports Boolean queries, which are potentially expressive and exact. However, PubMed is also designed to support simplicity of use at the expense of query expressiveness and exactness. Many PubMed users have never tried explicit Boolean queries. We developed a Java program, PubMed Assistant, to make literature access easier in several ways. PubMed Assistant provides an interface that efficiently displays information about the citations and includes useful functions such as keyword highlighting, export to citation managers, clickable links to Google Scholar and others that are lacking in PubMed. Availability: PubMed Assistant and a detailed online manual are freely available at under a GPL (GNU General Public License). Contact: [email protected]

UNLABELLED: MEDLINE/PubMed is one of the most important information sources for bioinformatics text mining. However, there remain limitations in working with MEDLINE/PubMed citations. For example, PubMed imposes an upper limit of 10,000 for downloading PMID list or citations; and MEDLINE files are too large for most off-the-shelf XML parsers. We developed a Java package, MedKit, to work-around the limitations, as well as provide other useful functionalities, e.g. random sampling. Its four modules (querier, sampler, fetcher and parser) can work independently, or be pipelined in various combinations. It can be used as a stand-alone GUI application, or integrated into other text-mining systems. Text mining researchers and others may download and use the toolkit free for non-commercial purposes. AVAILABILITY: http://metnetdb.gdcb.iastate.edu/medkit CONTACT: [email protected].

Summary: PathBinderH allows users to make queries that retrieve sentences and the abstracts containing them from PubMed. Another aspect of PathBinderH is that users can specify biological taxa in order to limit searches by mentioning either the specified taxa, or their subordinate taxa, in the biological taxonomy. Although the current project requires this function only for plant taxa, the principle is extensible to the entire taxonomy. Availability: www.plantgenomics.iastate.edu/PathBinderH. Source code and databases on request. Contact: [email protected] Supplementary information: A tutorial is at the tool Website. A longer paper is at class.ee.iastate.edu/berleant/s/paperPathBinderHreport.pdf

Characterizing the distribution of times to failure in 2-component systems is an important special case of a more general problem, finding the distribution of a function of random variables. Advances in this area are relevant to reliability as well as other fields, and influential papers on the topic have appeared in the reliability field over a span of many years. Using failure times of 2-component systems as a vehicle, this report begins by reviewing a technique for characterizing distributions of functions of random variables when the dependency relationship between the random variables used as inputs to the function is unknown. The technique addressed is called Distribution Envelope determination (DEnv). Using this review as a foundation, an extension to DEnv is described which applies to cases where means and variances of the input distributions are known, and partial information about dependency is available in the form of a value for correlation. Pearson correlation is used because it is the most commonly encountered correlation measure. This reason is important because the assumption of independence, while common, is frequently problematic. Yet the opposite extreme of no assumption about dependency may mean ignoring available information which could affect the analysis.

Many natural language processing approaches at various complexity levels have been reported for extracting biochemical interactions from MEDLINE. While some algorithms using simple template matching are unable to deal with the complex syntactic structures, others exploiting sophisticated parsing techniques are hindered by greater computational cost. This study investigates link grammar parsing for extracting biochemical interactions. Link grammar parsing can handle many syntactic structures and is computationally relatively efficient. We experimented on a sample MEDLINE corpus. Although the parser was originally developed for conversational English and made many mistakes in parsing sentences from the biochemical domain, it nevertheless achieved better overall performance than a co-occurrence-only method. Customizing the parser for the biomedical domain is expected to improve its performance further.

In this paper we discuss models for systems that support reading.Our account identifies important models and presents a framework for organizing them.To evaluate this account, we show its ability to suggest a wide range of text presentation systems, many of them novel.This evaluation not only provides interesting ideas for future systems, it also shows the usefulness of the account, and further, exemplifies a general approach to evaluating meta-level discussions such as this, namely, evaluating by assessing ability to generate interesting implications.

The ability to discriminate and distinguish among individual documents in the ever-increasing volume of information available through digital networks is becoming more and more difficult. With websites being added to the 100 million installed base by tens of thousands per month, information overload is inevitable (H. Berghel, 1997). There are two basic paradigms for dealing with this information overload: filtering (Information filtering, 1992) information before it reaches the end-user, and customizing the information after it arrives (Berleant & Berghel, 1994a, 1994b). Filtering remains primarily a server side activity since filtering at the client-side would necessitate unnecessary downloads. Information customization is a client-side activity designed to pick up where information filtering leaves off. In this article, we describe our vision of information customization and, along the way, chronicle the development of our proof-of-concept prototype, Cyberbrowser, for customizing information on the Web.

Abstract Every word in the lexicon of a natural language is used distinctly from all the other words. A word expert is a small expert system-like module for processing a particular word based on other words in its vicinity. A word expert exploits the idiosyncratic nature of a word by using a set of context testing decision rules that test the identity and placement of context words to infer the word's role in the passage. The main application of word experts is disambiguating words. Work on word experts has never fully recognized previous related work, and a comprehensive review of that work would therefore contribute to the field. This paper both provides such a review, and describes guidelines and considerations useful in the design and construction of word expert based systems.