Disease named entity recognition (NER) is an important enabling technology to develop various downstream biomedical natural language processing applications. This is a challenging task, which requires addressing potential ambiguities due to variable contextual usage of the disease name mentions in clinical texts. In particular, clinical trial texts have unique complexities compared to patient-focused clinical reports or information-rich biomedical research articles, as they typically define drug testing eligibility requirements for patient cohorts via compound contextual and logical relationships. In this paper, we propose a novel disease NER model for clinical trial texts by using deep contextual embeddings with relevant domain-specific features, word embeddings, and character embeddings in a bidirectional long short-term memory network-conditional random field (BiLSTM-CRF) framework. Experiments and analyses on a clinical trial dataset and the benchmark NCBI scientific article dataset show the effectiveness of the proposed model.

Biomedical Entity Recognition tasks have gained significant importance in the clinical research domain. There has been a lot of prior work on improving entity recognition of biomedical concepts from using rule-based to more context-dependent deep learning-based approaches. However, due to its high domain dependency and distinctive vocabularies, appropriate utilization of contextual knowledge becomes a challenge even for context dependent deep models. To this end, we propose a novel performance-boosting improvement module “BoostER” that can be applied to any existing entity recognition system to boost its performance in terms of precision and F1 scores. The proposed module has been developed on top of a pre-trained BERT model and fine-tuned to give more weights to contextual learning compared to word-specific information. We tested our system with Chemical and Disease Entity Recognition tasks using the BioCreative CDR dataset to demonstrate its effectiveness compared to existing state-of-the-art models.

DNA spectrograms express the periodicities of each of the four nucleotides A, T, C, and G in one or several genomic sequences to be analyzed. DNA spectral analysis can be applied to systematically investigate DNA patterns, which may correspond to relevant biological features. As opposed to looking at nucleotide sequences, spectrogram analysis may detect structural characteristics in very long sequences that are not identifiable by sequence alignment. Alignment of DNA spectrograms can be used to facilitate analysis of very long sequences or entire genomes at different resolutions. Standard clustering algorithms have been used in spectral analysis to find strong patterns in spectra. However, as they use a global distance metric, these algorithms can only detect strong patterns coexisting in several frequencies. In this paper, we propose a new method and several algorithms for aligning spectra suitable for efficient spectral analysis and allowing for the easy detection of strong patterns in both single frequencies and multiple frequencies.

BACKGROUND: The decision environment for cancer care is becoming increasingly complex due to the discovery and development of novel genomic tests that offer information regarding therapy response, prognosis and monitoring, in addition to traditional histopathology. There is, therefore, a need for translational clinical tools based on molecular bioinformatics, particularly in current cancer care, that can acquire, analyze the data, and interpret and present information from multiple diagnostic modalities to help the clinician make effective decisions. RESULTS: We present a platform for molecular signature discovery and clinical decision support that relies on genomic and epigenomic measurement modalities as well as clinical parameters such as histopathological results and survival information. Our Physician Accessible Preclinical Analytics Application (PAPAyA) integrates a powerful set of statistical and machine learning tools that leverage the connections among the different modalities. It is easily extendable and reconfigurable to support integration of existing research methods and tools into powerful data analysis and interpretation pipelines. A current configuration of PAPAyA with examples of its performance on breast cancer molecular profiles is used to present the platform in action. CONCLUSION: PAPAyA enables analysis of data from (pre)clinical studies, formulation of new clinical hypotheses, and facilitates clinical decision support by abstracting molecular profiles for clinicians.

DNA spectral analysis can be applied to systematically investigate DNA patterns, which may correspond to relevant biological features. As opposed to looking at nucleotide sequences, spectrogram analysis may detect structural characteristics in very long sequences that are not identifiable by sequence alignment. Clustering of DNA spectrograms can be used to perform spectral analysis of very large sequences or entire genomes, at different resolutions. Standard clustering algorithms have been used in spectral analysis to find strong patterns in spectra. However, as they use a global distance metric, those algorithms can only detect strong patterns coexisting in several frequencies. In this paper we propose a new method and several algorithms for aligning spectra suitable for efficient spectral analysis and allowing for the easy detection of strong patterns in both single frequencies and multiple frequencies.

Probe design is the most important step for any microarray based assay. Accurate and efficient probe design and selection for the target sequence is critical in generating reliable and useful results. Several different approaches for probe design are reported in literature and an increasing number of bioinformatics tools are available for the same. However, based on the reported low accuracy, determining the hybridization efficiency of the probes is still a big computational challenge. Present study deals with the extraction of various novel features related to sequence composition, thermodynamics and secondary structure that may be essential for designing good probes. A feature selection method has been used to assess the relative importance of all these features. In this paper, we validate the importance of various features currently used for designing an oligonucleotide probe. Finally, a classification methodology is presented that can be used to predict the hybridization quality of a probe.

Recent growth in broadband access and proliferation of small personal devices that capture images and videos has led to explosive growth of multimedia content available everywhere - from personal disks to the Web. While digital media capture and upload has become nearly universal with newer device technology, there is still a need for better tools and technologies to search large collections of multimedia data and to find and deliver the right content to a user according to her current needs and preferences. A renewed focus on the subjective dimension in the multimedia lifecycle, from creation, distribution, to delivery and consumption, is required to address this need beyond what is feasible today. Integration of the subjective aspects of the media itself - its affective, perceptual, and physiological potential (both intended and achieved), together with those of the users themselves will allow for personalizing the content access, beyond today's facility. This integration, transforming the traditional multimedia information retrieval (MIR) indexes to more effectively answer specific user needs, will allow a richer degree of personalization predicated on user intention and mode of interaction, relationship to the producer, content of the media, and their history and lifestyle. In this paper, we identify the challenges in achieving this integration, current approaches to interpreting content creation processes, to user modelling and profiling, and to personalized content selection, and we detail future directions. The structure of the paper is as follows: In Section I, we introduce the problem and present some definitions. In Section II, we present a review of the aspects of personalized content and current approaches for the same. Section III discusses the problem of obtaining metadata that is required for personalized media creation and present eMediate as a case study of an integrated media capture environment. Section IV presents the MAGIC system as a case study of capturing effective descriptive data and putting users first in distributed learning delivery. The aspects of modelling the user are presented as a case study in using user's personality as a way to personalize summaries in Section V. Finally, Section VI concludes the paper with a discussion on the emerging challenges and the open problems.

Differential DNA methylation has been linked to cancer mainly due to silencing of tumor-suppressor genes. Recent surge in research on epigenomics in cancer has produced tremendous amount of published literature, but till date there is no resource that collates all the information and represents it in an easily accessible and immediately use- ful form. In an effort to alleviate this problem, we propose a curated database MDDM, containing information about genes differentially methylated in cancer of various tissues. We present the database, its schema and sample queries on this database which cover the experimental methods, results in terms of sensitivity and specificity on particular tissues and cell lines. We also discuss the MDDM environment that is trying to pull information from various related resources. We propose a workflow and usage scenario for our MDDM environment to support a study for finding the role in gene regulation via DNA methyl binding proteins for breast can- cer. In our usage scenario, we used the MSP primers re- ported in various studies to find the genomic area detected to be differentially methylated. These areas were searched for consensus sequence patterns for various methyl binding proteins to detect their potential binding and hence deter- mine role in regulation of differential expression

The demand for technology that can process biological information is becoming more and more obvious and urgent. Existing research in bioinformatics has been focusing on various types of analysis of DNA sequences and various measurements taken at the protein, RNA transcript and DNA level. In this paper we will show the application of spectral analysis and image processing in analyzing DNA sequences of specific structure. In addition, we extend the framework to visualize long DNA sequences and help in identifying patterns that are visible at high resolution of DNA spectral images.

The need for personalized summaries of media content has been driven by the recent and anticipated explosive growth in the media world. In this paper, we present a methodology and a supporting user study for generating user profiles and content features that can be used to automatically create personalized summaries of broadcast television content. We determined a mapping, from users' personality traits measured by commonly available personality tests, to computable video features that such personality traits appear to prefer. Three common personality profiles (Myers-Briggs, Merrill Reed, and Brain.exe) were elicited from 59 subjects, together with their preferred summary of news, music, and talk show videos. A factor analysis between the personality traits and the features in preferred summaries indicated that only some traits (e.g., gender, extroversion, control orientation, intuitiveness, etc.) and only some features (e.g., faces, reportage, text, chorus, host, etc.) had predictive value. The mapping of personality to feature also differed by genre. However, in general, extroverted users tended to prefer directly experienced content, while introverted users preferred content mediated through analysis. A validation user study is in progress

This paper describes a speaker detection system using cross-modal association methods. Four association approaches are designed using linear and nonlinear association models. Speaker detection experiments were conducted to compare the approaches

We present a system that summarizes the textual, audio, and video information of music videos in a format tuned to the preferences of a focus group of 20 users. First, we analyzed user-needs for the content and the layout of the music summaries. Then, we designed algorithms that segment individual song videos from full music video programs by noting changes in color palette, transcript, and audio classification. We summarize each song with automatically selected high level information such as title, artist, duration, title frame, and text as well as audio and visual segments of the chorus. Our system automatically determines with high recall and precision chorus locations, from the placement of repeated words and phrases in the text of the song's lyrics. Our Bayesian belief network then selects other significant video and audio content from the multiple media. Overall, we are able to compress content by a factor of 10. Our second user study has identified the principal variations between users in their choices of content desired in the summary, and in their choices of the platforms that should support their viewing.

The paper analyzes the interests of users in viewing and organizing consumer videos. It proposes a taxonomy of relevant concepts with three basic dimensions of interests (DOIs) and effective models to predict the user interests in each dimension. The three DOIs correspond to the objects, the scenes and the events. Our conclusions are backed with an extensive study, in which users were asked to annotate and score the importance of each DOI in short clips of diverse and real consumer videos. Analysis of the user study data reveals high consistency (70%) of the scores across different users and independence between objects and events. In addition, we show how heuristic rules and neural networks can accurately predict these scores using camera motion, foreground object and audio information. The automatic and effective prediction of user interests has the potential for improving applications for annotating and summarizing consumer videos.

Existing methods for audiovisual and text analysis of videos perform "blind" recovery of metadata from the audiovisual signal. The film production process however, is based on the original screenplay and its versions. Using this information is like using the recipe book for the movie. High-level semantic information that is otherwise very difficult to derive from the audiovisual content can be extracted automatically by enhancing feature extraction with screenplay processing and analysis. As a test-bed of our approach, we investigated the use of screenplay as a source of information for speaker/character identification. Our speaker identification method consists of screenplay parsing, extraction of time-stamped transcript, alignment of the screenplay with the time-stamped transcript, audio segmentation and audio speaker identification. As the screenplay alignment cannot identify all dialogue sections within any film, we use the segments found by alignment as labels to train a statistical model in order to identify unaligned pieces of dialogue. We find that the screenplay alignment is able to identify the speaker correctly in 30% of lines of dialogue on average. However, with additional automatic statistical labeling for audio speaker ID on the soundtrack, our recognition rate improves significantly.

The emergence of peer-to-peer networking and the increase of home PC storage capacity are necessitating efficient scaleable methods for video clustering, recommending and browsing. Based on film theories and psychological models, color-mood is an important factor affecting user emotional preferences. We propose a compact set of features for color-mood analysis and subgenre discrimination. We introduce two color representations for scenes and full films in order to extract the essential moods from the films: a global measure for the color palette and a discriminative measure for the transitions of the moods in the movie. We captured the dominant color ratio and the pace of the movie. Despite the simplicity and efficiency of the features, the classification accuracy was surprisingly good, about 80%, possibly thanks to the prevalence of the color-mood association in feature films.

Commercial detection plays an important role in various video segmentation and indexing applications. It provides high-level program segmentation so that other algorithms can be applied on the true program material in the broadcast. It is a challenge to have robust commercial detection methodology for various platforms, content formats, and broadcast styles that are used all over the world. Wide deployment of such an algorithm not only requires the development of new algorithms but also updating and tuning of parameters for existing algorithms. We present visual commercial detectors that rely on features including, luminance, letterbox, and keyframe distance. These detectors were developed after a careful study of the various features that can be extracted during MPEG-encoding process in real time. Due to the intermittent nature of the features, and platform restrictions, the commercial detection relies on a set of thresholds to keep the implementation as simple as possible. We evolved these thresholds using genetic algorithms (GAs) to optimize the performance. We show how a scalar genetic algorithm can locate sets of parameters in a multi-objective space (precision and recall) that outperform the values selected by an expert engineer. We present the results of optimizing a commercial detection algorithm for different data sets and parameter sets. In this paper we show that GAs drastically improved our approach and enabled fast prototyping and performance tuning of commercial detection algorithms.

The ability to summarize and abstract information is an essential part of intelligent behavior in consumer devices. However, even for the emerging area of video content analysis, user preferences on summarization have not been explored. This paper reports on a panel which asked: who, why and when summarization is needed; what information should be summarized; and what forms should summaries take. In particular, we investigated the requirements with sensitivity to user needs, user context, media content, device capabilities, and the methods by which user and environment profiles can be assembled and exploited. We organize our findings as a wish list containing four major user questions, and we report the current and near-term states of the art and the major technical challenges in satisfying them. Our study suggests that user preferences should be derived from explicit user statements and from implicit trends inferred from viewing histories and summary usage.

Talking face detection is important for videoconferencing. However, the detection of the talking face is difficult because of the low resolution of the capturing devices, the informal style of communication and the background sounds. In this paper, we present a novel method for finding the talking face using latent semantic indexing approach. We tested our method on a comprehensive set of home video conferencing sessions with a very high detection rate. Our experiments show that the LSI method accuracy degrades gracefully in a noisy environment as opposed to the correlation method which simply fails in presence of noise.

No abstract available.

Multimodal information processing has received considerable attention in recent years. The focus of existing research in this area has been predominantly on the use of fusion technology. In this paper, we suggest that cross-modal association can provide a new set of powerful solutions in this area. We investigate different cross-modal association methods using the linear correlation model. We also introduce a novel method for cross-modal association called Cross-modal Factor Analysis (CFA). Our earlier work on Latent Semantic Indexing (LSI) is extended for applications that use off-line supervised training. As a promising research direction and practical application of cross-modal association, cross-modal information retrieval where queries from one modality are used to search for content in another modality using low-level features is then discussed in detail. Different association methods are tested and compared using the proposed cross-modal retrieval system. All these methods achieve significant dimensionality reduction. Among them CFA gives the best retrieval performance. Finally, this paper addresses the use of cross-modal association to detect talking heads. The CFA method achieves 91.1% detection accuracy, while LSI and Canonical Correlation Analysis (CCA) achieve 66.1% and 73.9% accuracy, respectively. As shown by experiments, cross-modal association provides many useful benefits, such as robust noise resistance and effective feature selection. Compared to CCA and LSI, the proposed CFA shows several advantages in analysis performance and feature usage. Its capability in feature selection and noise resistance also makes CFA a promising tool for many multimedia analysis applications.

Automatic indexing of large collections of multimedia data is important for enabling retrieval functions. Current approaches mostly draw on a single or dual modality of video content analysis. Here we describe a framework for the integration of multimedia content and context information, which generalizes and systematizes current methods. Content information in the visual, audio, and text domains, is described at different levels of granularity and abstraction. Context describes the underlying structural information that can be used to constrain the possible number of interpretations. We introduce a probabilistic framework that combines (a) Bayesian networks that describe both content and context and (b) hierarchical priors that describe the integration of content and context. We present an application that uses this framework to segment and index TV programs. We discuss experimental results on segment classification on six and a half hours of broadcast video. In our experiments we used audio context information. Classification results for financial segments yield 83% and for celebrity segments 89%.

This paper is concerned with the topic of personalized news assembly at the set-top box, based on augmented video. This is video complemented with additional information that is somehow relevant to the semantic video content. We touch upon the technique that is used for video augmentation, which is video subject detection followed by information searches on the subject. The focus of this paper is on subject detection implemented using traditional text analysis tools; video segmentation is based on these results and visual processing. We describe the architecture of such a system and the benefits to the consumer. Further we discuss a preliminary system that shows the viability of the concept.

The importance of video content analysis and retrieval increases as storage capacity grows in consumer devices. Videotext detection is very important for video indexing and retrieval enabling features such as commercial detection, intelligent keyframe extraction, program boundary detection and others. In this paper we propose multi-layered videotext detection tailored to an MP EG encoding scheme. We approached the videotext detection from a perspective of scalability and flexibility for different platforms with varying resources. We propose a three-layered algorithm. The first layer works in compressed domain features such as macroblock type. The second layer works in semicompressed domain such as DCT coefficients. The third layer works in uncompressed domain, i.e. spatial domain. As the next layer gets implemented the complexity increases but so does the precision of the algorithm. On constrained platforms, only the first layer would be implemented. On high-end platforms all three layers could be implemented to enable a full suite of indexing and retrieval applications.

Today, users have to cope with an overwhelming number of TV channels and Web content sources. We introduce automatic content augmentation as a novel approach to contextual information extraction on behalf of the user where the context is provided by the primary content source (i.e. TV channel) and tailored by the user's preferences. A key aspect of this approach is Web information extraction (WebIE) which automatically derives structured information from unstructured Web documents. Our system executes WebIE tasks, each an instantiation of WebIE rules, our generic document processors. We present two WebIE approaches: diffusion WebIE that crawls a wide set of Web pages and extracts information from a subset of the pertinent pages; and laser WebIE that accesses a select set of Web pages and extracts narrowly defined information. We describe the architecture and the implementation details of the system and provide detailed laser WebIE examples.

Currently available personal video recorders find and store whole TV programs. Our system, Video Scouting, not only finds and stores programs; it automatically segments and indexes story segments from the programs according to viewers' profiles. The extracted descriptions serve the viewers' content information requests for program segment selection, e.g., play the three minute interview with Hillary Clinton. To achieve this, the system combines information from the audio, visual, and transcript domains in a probabilistic framework based on Bayesian networks. We describe the overall architecture, a system implementation, and discuss some experimental results.

We describe integrated multimedia processing for Video Scout, a system that segments and indexes TV programs according to their audio, visual, and transcript information. Video Scout represents a future direction for personal video recorders. In addition to using electronic program guide metadata and a user profile, Scout allows the users to request specific topics within a program. For example, users can request the video clip of the USA president speaking from a half-hour news program. Video Scout has three modules: (i) video pre-processing, (ii) segmentation and indexing, and (iii) storage and user interface. Segmentation and indexing, the core of the system, incorporates a Bayesian framework that integrates information from the audio, visual, and transcript (closed captions) domains. This framework uses three layers to process low, mid, and high-level multimedia information. The high-level layer generates semantic information about TV program topics. This paper describes the elements of the system and presents results from running Video Scout on real TV programs.

This paper describes the transcript triggers system and method that automatically retrieves information supplemental to video programming. The generated video transcript is searched for the items of interest and related information is returned. The system performs extraction of transcript, and matching of keywords based on a personal profile, pulling information from additional sources and handling the user interface using a rich set of association rules.

Current personal Vido recorders make it very easy for consumers to record whole TV programs. Our research however, focuses on personalizing TV at a sub-program level. We use a traditional Content-Based Information Retrieval system architecture consisting of archiving and retrieval modules. The archiving module employs a three-layered, multimodal integration framework to segment, analyze, characterize, and classify segments. The retrieval module relies on users personal preferences to deliver both full programs and video segments of interest. We tested retrieval concepts with real users and discovered that they see more value in segmenting non-narrative programs (e.g. news) than narrative programs (e.g. movies). We benchmarked individual algorithms and segment classification for celebrity and financial segments as instances of non-narrative content. For celebrity segments we obtained a total precision of 94.1% and recall of 85.7%, and for financial segments a total precision of 81.1% and a recall of 86.9%.

In this paper we present a novel method for computing superhistograms to represent video segments. We build cumulative histograms for video shots and scenes. The underlying assumption is that a TV program has a consistent color palette, which can be derived as a family of merged individual shot histograms. We present the experimental results computed from several hours of different TV programs such as sitcoms, serials and news. The superhistograms representing episodes of the same sitcom look strikingly similar. The histograms extracted from TV news are not similar to the histograms extracted from sitcoms. However, there are superhistograms extracted from different sitcoms, which show certain resemblance to each other. This method can be used in a variety of applications that need video classification and retrieval methods such as video editing, studio archival, digital libraries, consumer products, and web crawling.

No abstract available.

Methods for video content analysis are necessary for the growing amount of video information delivered to consumers today. In this paper, we present a system for video content analysis called Vitamin, which provides management of a home video library. The system presents the user with a visual table of contents that provides an overview of the video content and direct access to particular points in the stored video. In this process, we apply a computationally inexpensive, simple, yet powerful mechanism for cut detection and keyframe filtering. Our initial implementation and results show that this system can perform video content extraction in real time on a low-end platform that matches a visual table of contents extracted by an expert.

Article Video keyframe extraction and filtering: a keyframe is not a keyframe to everyone Share on Authors: Nevenka Dimitrova Philips Research, 345 Scarborough Rd., Briarcliff Manor NY Philips Research, 345 Scarborough Rd., Briarcliff Manor NYView Profile , Thomas McGee Philips Research, 345 Scarborough Rd., Briarcliff Manor NY Philips Research, 345 Scarborough Rd., Briarcliff Manor NYView Profile , Herman Elenbaas Philips Research, 345 Scarborough Rd., Briarcliff Manor NY Philips Research, 345 Scarborough Rd., Briarcliff Manor NYView Profile Authors Info & Claims CIKM '97: Proceedings of the sixth international conference on Information and knowledge managementJanuary 1997 Pages 113–120https://doi.org/10.1145/266714.266876Online:01 January 1997Publication History 28citation1,003DownloadsMetricsTotal Citations28Total Downloads1,003Last 12 Months12Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Like other types of digital information, video sequences must be classified based on the semantics of their contents. A more-precise and completer extraction of semantic information will result in a more-effective classification. The most-discernible difference between still images and moving pictures stems from movements and variations. Thus, to go from the realm of still-image repositories to video databases, we must be able to deal with motion. Particularly, we need the ability to classify objects appearing in a video sequence based on their characteristics and features such as shape or color, as well as their movements. By describing the movements that we derive from the process of motion analysis, we introduce a dual hierarchy consisting of spatial and temporal parts for video sequence representation. This gives us the flexibility to examine arbitrary sequences of frames at various levels of abstraction and to retrieve the associated temporal information (say, object trajectories) in addition to the spatial representation. Our algorithm for motion detection uses the motion compensation component of the MPEG video-encoding scheme and then computes trajectories for objects of interest. The specification of a language for retrieval of video based on the spatial as well as motion characteristics is presented.

The most prominent difference between still images and moving pictures stems from movements and variations. Thus to go from the realm of still image repositories to video databases, we must be able to deal with motion. Particularly, we need the ability to classify objects appearing in a video sequence based on the movements of each object, as well as their other characteristics and features such as shape or color.